Police on Wednesday detained about 40 members of a Turkish nationalist party who occupied one of Istanbul's most famous buildings, the Haghia Sophia, to protest Pope Benedict XVI's visit next week.

The protesters belong to the Great Unity Party, a far right-wing group that has previously staged demonstrations against the planned Nov. 28-Dec. 1 visit.

Video footage showed several dozen men entering the 6th century former Byzantine church and mosque, shouting in unison, "Allahu akbar!" — "God is great!" — and then kneeling to perform Islamic prayers.

They also repeatedly shouted a warning to Benedict: "Pope, don't make a mistake, don't wear out our patience."

A group leader read a statement saying Benedict had offended Muslims with his comments linking violence and Islam, but the reading was interrupted by police.

When the protesters refused to surrender, a policeman used pepper spray on them. Police later rounded up the protesters against a wall outside a door of the Haghia Sophia.

The protesters were loaded into buses and taken to a police station for questioning, police said.

The Vatican downplayed the protest.

"I continue to consider these events sporadic and limited, and as such they do not put into question the substance and climate of the visit, which we expect will be serene," Vaticanspokesman Federico Lombardi was quoted as telling the ANSAnews agency.

"These facts do not give rise to particular worries, although they do cause regret," he said, according to the report. However, they are not a surprise since we know that there are groups who are not in favor of the pope's visit. But these are facts that should not be overestimated."

Culture Minister Atilla Koc condemned the protest, saying "I cannot approve a raid on a cultural center or a place of worship, whatever the reasons."

"It is an ugly incident," he said.

Benedict is scheduled to tour the Haghia Sophia, which is a source of religious sensitivity in Turkey. It was one of the world's greatest churches for more than 1,000 years, but was converted into a mosque after the conquest of Istanbul by Ottoman Turks in 1453. Today, the Haghia Sophia is a museum, and public religious ceremonies inside are forbidden.

There has been intense debate in Turkey about whether the pope will pray inside the Haghia Sophia, with some saying that doing so would be a political act and an attempt to reclaim its status as a great Christian church.

Benedict's first trip to a Muslim nation comes at a time of heightened tension between the West and Islam.

The Muslim world erupted in protest after Benedict delivered a speech in September in which he quoted a Byzantine emperor who characterized some of the teachings of the Prophet Muhammad as "evil and inhuman," particularly "his command to spread by the sword the faith."

Benedict has offered his regrets that his speech caused offense and has stressed that the quotes did not reflect his personal opinion. He has also expressed esteem for Islam.

Click here to go to FOXNews.com's Europe Center.