A Muslim-American advocacy group has called on radio talk show host Dennis Prager to be removed from the governing board of the federally funded U.S. Holocaust Memorial Museum after Prager last week blasted a representative-elect for planning to use the Koran at his swearing-in next month.

Prager, who is Jewish, slammed Minnesota Democrat Keith Ellison, the first Muslim to be elected to the U.S. Congress, after Ellison announced he plans to have his oath of office photo taken with the Koran instead of the Christian Bible, which is traditional.

"Insofar as a member of Congress taking an oath to serve America and uphold its values is concerned, America is interested in only one book, the Bible," he wrote in a column titled, "America, Not Keith Ellison, Decides What Book a Congressman Takes His Oath On."

"If you are incapable of taking an oath on that book, don't serve in Congress," Prager wrote, adding that using the Koran "undermines American civilization."

The Washington, D.C.-based Council on American-Islamic Relations (CAIR) issued a statement Monday calling for Prager's ouster from the U.S. Holocaust Memorial Council, the museum's advisory council.

"No one who holds such bigoted, intolerant and divisive views should be in a policy-making position at a taxpayer-funded institution that seeks to educate Americans about the destructive impact hatred has had, and continues to have, on every society," CAIR wrote in its letter to the museum council's president, Fred Zeidman.

"As a presidential appointee, Prager's continued presence on the council would send a negative message to Muslims worldwide about America's commitment to religious tolerance," CAIR wrote.

Prager was not immediately available for comment. Museum spokesman Andrew Hollinger told FOXNews.com that he had not yet spoken with Zeidman and could not comment on the letter.

Prager is one of 52 council members, not including 10 House and Senate members and five ex-officio members. Hollinger said each council member is appointed by the White House to serve on the museum's governing board. The federal government pays for about 60 percent of the museum's annual budget.

Last week, the Anti-Defamation League, a Jewish advocacy group, called Prager's opinions "flat-out wrong," as well as "intolerant, misinformed and downright un-American."

"If Prager were merely a blogger and radio talk-show host trying to be relevant and provocative, these views might not merit a response. But as a newly appointed member of the United States Holocaust Memorial Council, Prager and his views must be held to a higher standard," the group said.

Ellison's chief of staff, Kari Moe, said Friday "the criticism is being flamed by the politics of division that were rejected in the '06 election cycle."

She also said the tradition of using the Bible relates to the photo-op ceremony for individual lawmakers that follows the official swearing-in of all members of Congress together on the House floor. At the official oath of office, newly elected members raise their right hands.

After that explanation, Prager said the ceremonial oath is no less significant than the actual swearing-in.

"Oh, that's the whole point. It's exactly because it's ceremonial that it matters to me," he said. "Ceremonies matter. Ceremonies are exceedingly important. That is the way a society states what is most significant to it."

Prager also said that he would like to see Jews swear their oaths of office on the Christian Bible — the Jewish Bible does not include the New Testament that is central to Christian beliefs.

Prager said he doesn't think Ellison should serve if he does not take the oath on the Bible but he doesn't prefer legislative action be taken to prevent him from serving.

Speaking last week with FOX News, Prager said his feelings have nothing to do with Ellison's being a Muslim.

"It has to do with the fact that it's not the Bible," he said. "If a Mormon-American decided to substitute the Book of Mormon for the Bible, I would have the same problem. If a scientologist wanted L. Ron Hubbard's 'Dyanetics' to be the book, I would have a problem," Prager said.

Prager said the Bible is the most important book in American history and that's why it should be used.

"George Washington began this movement. He was the one to bring a Bible and swear on it. Every president except one, Teddy Roosevelt, because it was right after the assassination of James McKinley and they didn't have time for a Bible. And every president has done this," Prager said, adding that congressional members have followed the tradition.

FOXNews.com's Greg Simmons and the Associated Press contributed to this report.