Daniels, 67, is scheduled to perform Saturday in Dearborn, the center of southeastern Michigan's 300,000-member Arab-American community.
After the Sept. 11 terror attacks (search), Daniels wrote and recorded the song, which became a country hit.
It begins: "This ain't no rag, it's a flag and we don't wear it on our heads. It's a symbol of the land where the good guys live. Are you listening to what I said?"
On Saturday, the Charlie Daniels Band will perform at the city-sponsored Homecoming Festival.
"Someone who is trying to get popular through bigotry, bias and hatred should not be welcome, especially in a city like Dearborn with its rich diversity," Imad Hamad, regional director of the American-Arab Anti-Discrimination Committee, told the Detroit Free Press.
City spokeswoman Mary Laundroche said no formal complaints have been received about the performance and the show will go on.
"Ragheads" is a slur used against Arabs, Muslims and others who wear turbans or cover their heads.
Daniels says the song is not directed at Arabs and Muslims in general, just at turbaned terrorists like Usama bin Laden (search).
"It's not anti-Arab or anti-anything," he said Wednesday by phone from Tennessee, where he lives. "The only thing it's `anti' is the people who bombed us on 9/11. I have people who say you're putting down people who wear turbans. I'm not."
"There are good Arabs and bad Arabs, good Greeks and bad Greeks, good people and bad people in any race," Daniels said. "I'm not a racist person. I came up during the old Jim Crow days. I know what racism is."