Updated

President Obama has canceled a visit to India's famed Golden Temple following concerns that he would have to wear a headscarf that could make him look Muslim, according to several Indian news outlets.

Though the Golden Temple is a Sikh temple -- and Sikhs are not Muslims -- the Indian Express first reported that the visit was thrown into doubt following an impasse between Obama's aides and temple authorities.

The president would have been expected to wear some kind of head cover, but the White House team ruled out a traditional headscarf. When Obama's team proposed the U.S. president wear a "modified" baseball cap, temple officials said a baseball cap would not be permitted, according to the Indian Express.

A White House official would not confirm the president's final itinerary to FoxNews.com, but said there wouldn't be enough time for Obama to visit every site he wants during the three-day visit, part of a broader Asian trip.

"There are more events that he'd like to do than he'd have time to do in a three-day visit," the official said. "Advance teams visit dozens of places ... with any trip. They don't all end up on a final itinerary."

Other Indian newspapers have reported that the visit was in fact canceled.

Sikh leaders have since appealed to the White House to reconsider its decision -- Obama would otherwise confine his trip, his first official visit to India as president, to stops in Mumbai and New Delhi.

One Indian official told The Times of India that U.S. officials agreed last month that Obama could wear a skull cap instead.

"We don't know what happened after that," the official said.

CNN in India reported Wednesday that the Sikh religion's governing body has since relented on that

Kewal Dhillon, a member of the state legislature of Punjab where the temple is located, urged a compromise, saying it would be "truly a huge loss" to the Sikh community if Obama did not visit, according to the Indian Express.

Sikhs number approximately 2 percent of India's nearly 1.2 billion population, or roughly 19 million people. The U.S. Census Department reported 78,000 self-identified Sikhs in the U.S. in 2008.

White House officials reportedly are concerned that an image of the president in such unconventional head cover could give the impression of him as a Muslim. The president is Christian and has battled rumors about his faith since the 2008 presidential campaign.

His Kenyan father was born a Muslim but later became an atheist, Obama has said.