The national debate on health care reform has turned ugly, with some leveling charges of racism against opponents.

A Congressional Black Caucus member said Tuesday fears the revival of America's worst days of race hatred.

"I guess we'll probably have folks putting on white hoods and white uniforms again and riding through the countryside, intimidating people," Rep. Hank Johnson, D-Ga., said when asked whether Rep. Joe Wilson's shout to President Obama that "You lie," reflected the sentiment of racists. 

"And you know that's the logical conclusion if this kind of attitude is not rebuked," Johnson said.

New York Times columnist Maureen Dowd also found Wilson's heckling of Obama to be racially motivated, writing that Wilson really meant, "You lie, boy."

On Tuesday evening, former President Jimmy Carter said in an interview with NBC that "an overwhelming portion" of those who demonstrate against Obama are doing so because the president is a black man.

The House voted Wednesday to admonish Wilson, who refused to apologize a second time after apologizing privately to Obama last week.

The White House declined FOX News' request for an interview, but a spokesman for the president said recently his boss sees no racial motives at play.

"I don't think the president believes that people are upset because of the color of his skin," White House press secretary Robert Gibbs said.

Rep. Eric Cantor of Virginia, the No. 2 Republican in the House, also declined to be interviewed on this topic, calling it "too sensitive." When pressed during an interview on FOX News, Cantor largely ducked the issue.

The race card started flying last month when New York Gov. David Paterson claimed in a radio interview that he and other prominent black politicians are being judged by the color of their skin.

Soon after that came Rep. Diane Watson's racially-charged statement on opponents of Obama's health reform plan.

"They are spreading fear and they're trying to see that the first president who looks like me fails," the California Democrat said.

Then Rep. Charles Rangel, D-N.Y., added his two cents: "Some Americans have not gotten over the fact that Obama is president of the United States. They go to sleep wondering, 'How did this happen?'"

FOX News' James Rosen contributed to this report.