A travel ban on flights to and from three African countries ravaged by Ebola could be the only way to keep stricken passengers off America-bound planes, because while the State Department can stop visa holders from boarding U.S.-bound planes, keeping holders of American passports off is far more complicated, sources told FoxNews.com.

Scores of congressmen and senators have called for the Obama administration to impose a travel ban on the three African countries ravaged by Ebola -- Guinea, Liberia and Sierra Leone. While critics say such a ban, which would likely not apply to humanitarian assistance, could be next to impossible to impose, it could be the only way to keep American citizens who visit the countries from coming to the U.S., even if there is reason to believe they pose an Ebola risk.

That's because while the State Department can cancel a visa, it does not have the power to nullify a valid passport. The Department of Homeland Security can in some cases, though it is against UN convention to render anyone stateless.

U.S. passport holders would likely be allowed to fly home, then face possible quarantine in the U.S., a source said.

“There’s two options to quarantine,” a Capitol Hill source told FoxNews.com. “They can go on to their final destination and immediately be put into quarantine or they can be put under watch at the point of origination. But that would not be likely.”

The source said quarantining an American citizen in a West African nation would require a high level of cooperation from that country, which, unlike the case of a visa holder, would not be expended on a native citizen. The U.S. does not typically have ready facilities in which to quarantine its citizens in foreign countries.

The technicality is one reason several lawmakers, including House Speaker John Boehner, R-Ohio, have said a ban should be on the table.

"A temporary ban on travel to the United States from countries afflicted with the virus is something that the president should absolutely consider, along with any other appropriate actions as doubts about the security of our air travel systems grow," Boehner said.

Several African nations, as well as Great Britain and France, have instituted travel bans involving Liberia, Sierra Leone and Guinea.

The Obama administration has been adamant that it will not ban flights to and from the three nations, claiming it would be nearly impossible to implement, ineffective at fighting the disease and make it more difficult to provide aid to the affected countries.

“We are not imposing any sort of ban,” a State Department official told FoxNews.com. “We need to allow for this to be a global response and make sure that there is proper care and training throughout the region."

In addition, the World Health Organization has previously advised against issuing travel bans, noting that air-travel poses a low risk for Ebola transmissions.

"On the small chance that someone on the plane is sick with Ebola, the likelihood of other passengers and crew having contact with their body fluids is even smaller," the organization said in August.

Other opponents of the bans say implementing them requires an unrealistic level of international cooperation. They note that Thomas Eric Duncan, the Liberian who brought Ebola to the U.S. and ultimately infected at least two nurses, flew to Texas via Brussels.

CDC Director Tom Frieden said the system of screening at airports in Africa is working, and that 77 people have been stopped from leaving the affected countries since this summer, when questioning and screening of travelers leaving airports in Guinea, Liberia and Sierra Leone began.