A Kentucky clerk's office on Thursday again refused to issue a marriage license to a gay couple, in defiance of a U.S. Supreme Court ruling that legalized same-sex marriage across the country two months ago.

Rowan County Clerk Kim Davis has refused to issue any marriage licenses, citing her Christian faith and constitutional right to religious freedom, since the landmark decision in June.

On Thursday morning, a deputy clerk in her office refused to issue a marriage license to William Smith Jr. and James Yates. It was their third attempt to get a license.

They said they will not give up.

"They just don't like gay people, they don't want us to get married," Yates said. "And they'd rather burn the earth and not let straight people in Rowan County get married either."

The action Thursday came just a day after a federal appeals court upheld a ruling ordering the clerk in rural Rowan County to issue marriage licenses to same-sex couples.

U.S. District Judge David Bunning had already ordered Davis to issue marriage licenses two weeks ago. He later delayed that ruling until Aug. 31 or until the U.S. 6th Circuit Court of Appeals issued a ruling. The appeals court did so on Wednesday, denying Davis' appeal.

But a deputy clerk in Davis' office told Smith and Yates on Thursday that the office believes Bunning's delay remains in effect until Aug. 31. He refused to give his name or give them a license.

Davis, meanwhile, sat in her office with the door closed. She talked on the phone, ignoring the commotion as the couples, trailed by activists and reporters, poured in through the door and demanded answers.

The American Civil Liberties Union sued her last month on behalf of April Miller and Karen Roberts, a second gay couple and two straight couples. A U.S. district judge ordered Davis to issue the marriage licenses, but later delayed his order so that Davis could have time to appeal to the 6th circuit.

Miller said they felt vindicated by the federal court ruling. They got out the boxes holding their matching wedding bands, bought days after the Supreme Court's decision in June. They are simple white gold bands, ringed in diamonds.

"One step closer," Miller said. "We might be able to get married in September."

Mat Staver, an attorney for Davis, said he was disappointed with the ruling. He said he plans to discuss options with Davis, including an appeal to the U.S. Supreme Court.

"The court of appeals did not provide any religious accommodation rights to individuals, which makes little sense because at the end of the day it's individuals that are carrying out the acts of the office," Staver said. "They don't lose their individual constitutional rights just because they are employed in a public office."

It's unclear how Davis would react if she were to ultimately lose her appeals. She testified in federal court last month she would "deal with that when the time comes." Saturday, she spoke to thousands of supporters at a religious freedom rally at the state capitol, saying: "I need your prayers ... to continue to stand firm in what we believe."

"Regardless of what any man puts on a piece of paper, the law of nature is not going to change," Davis told the crowd.

Davis has said she will not resign. She can only be removed from office if the state legislature impeaches her, which is unlikely. If she continues to defy a federal court order, a judge could hold her in contempt and order hefty fines or jail time.

"Certainly none of those are appealing to my client," Staver said. "No one wants to be fined or go to jail and she's always been a law-abiding citizen. She's just caught in a very difficult situation."

William Sharp, legal director of the ACLU of Kentucky, said he believes the case "is simple.

"Religious liberty certainly does not allow public officials to deny government services to the public based on their personal beliefs," Sharp wrote in a statement. "All that Davis is required to do in her official capacity as clerk is issue a form. In no way is she being forced to endorse anyone's marriage or beliefs."

Miller and Roberts said they know the legal fight will stretch on. Davis continued to refuse to issue marriage licenses even after Bunning ordered her two weeks ago to do so. The couple expects she will continue to refuse.

"We get all excited. But we know a letdown is coming again," Miller said. "It's going to keep going. It's gonna be a long haul."

"But it felt so good for a minute," Roberts chimed in.