Authorities in Florida released three men detained Friday for more than 17 hours for a terror scare that police say was apparently a hoax.

"If this was a hoax, they will be charged," Collier County Sheriff Don Hunter said angrily after an all-day search of the men's two cars turned up no sign of explosives.

It was unclear what charges, if any, the men might face in Florida or Georgia, where a woman told authorities she heard them plotting at a restaurant Thursday morning.

After their release, the three drove to a rest stop, where they told reporters they were medical students heading to Miami for training and denied making any comments or jokes about terrorism. Police declined to say what the men told them during questioning.

At the rest stop, Ayman Gheith, who has a long beard and wore a skull cap, said the woman may have been influenced by his appearance.

"She saw obviously the way I was dressed and maybe she put a little salt and pepper into her story," he said.

The men later told CNN they were unaware of any problems in the restaurant. "The words 9-11, the words September weren't even mentioned in the conversation. Or September 13th. We were talking about what we were going to do in Miami," Gheith said.

The cars were stopped after the Georgia woman reported overhearing three men who appeared to be of Middle Eastern descent making "alarming" comments during breakfast at a restaurant in Calhoun, Ga., said Mickey Lloyd of the Georgia Department of Public Safety.

Eunice Stone, a 44-year-old nurse, told Fox News she was at a Shoney's restaurant in Calhoun with her son, Joshua, on Thursday morning when three men at another table "started talking and laughing about 9/11."

She said they joked about how they had celebrated the terror attacks at a party the night before, and that they spoke ominously about a terror attack planned for Friday.

"It really scared me, I was really scared," Stone said. "Why were they sitting there laughing about 9/11?"

Georgia officials issued an alert based on the woman's report and the cars were stopped at 1 a.m. after one went through the Interstate 75 tollbooth east of Naples, authorities said. The men told CNN they paid the toll, but that the attendant was confused about whether they had.

The men were detained in a van while authorities used dogs and a robot to go through the cars.

"The whole time I kept asking, 'Why are we being pulled over? Why is this happening?"' Kambiz Butt said during the TV interview.

Police did not tell them why they had been detained until shortly before their release, Omar Chaudhary added.

A federal law enforcement official in Washington, who spoke on condition of anonymity, said the men were of Jordanian, Iranian and Pakistani descent. Authorities said one is a U.S. native, another a naturalized citizen and the third has a valid visa.

Relatives of the men criticized the investigation, suggesting they had been singled out because of their heritage.

"I don't know what the lady in the restaurant heard or assumed. She must have had some kind of prejudice," Javed Chaudhary said from his home in Kansas City, Mo. "My son was born and raised here. I feel like we don't have freedom here anymore. Anybody can call anybody to make any kind of accusation. And the authorities treat you like you are a criminal."

Hana Gheith of suburban Chicago also said she didn't believe the report about her brother, 27-year-old Ayman Gheith. She said he was driving to Miami with friends to find an apartment before starting a training program at a hospital.

"My brother doesn't joke about these matters," she said, her voice at times shaking with anger. "A lot of Muslims suffered in 9/11."

Stone said later that her son told her at the time, "Mama, they're just messing with you."

But she told Fox News that whether or not it was a joke, she still believed she did the right thing.

Neighbors in Calhoun, a town of 10,000 in rural north Georgia, said they were proud of Stone for calling authorities.

"I appreciate someone like her with the courage to do it," said neighbor Eric Finch. "For anyone to sit around and joke over a cup of coffee about a couple of thousand people being killed -- they should be prosecuted just for that."

Florida Gov. Jeb Bush said the information that triggered the threat came from credible witnesses.

"Making a mockery or laughing about 9-11 and saying ... 'If you thought 9-11 was bad wait until you see what happens on September 13th' gives us all some concern," he said.

The Associated Press contributed to this report.