Updated

Political commentator Rush Limbaugh (search) quit his job as an ESPN football analyst Wednesday over comments he made about a professional football quarterback, the national media and race.

Limbaugh late Wednesday said he was resigning from his position at ESPN's "Sunday NFL Countdown" because he regretted causing "discomfort" to his colleagues. But Limbaugh stood by his statement that Philadelphia Eagles quarterback Donovan McNabb (search) was overrated because the media wanted to see a black quarterback succeed.

On Sunday, Limbaugh said, "I think what we've had here is a little social concern in the NFL. The media has been very desirous that a black quarterback do well. ... There is a little hope invested in McNabb, and he got a lot of credit for the performance of this team that he didn't deserve. The defense carried this team."

Limbaugh was "way off base with that comment," FOX NFL Sunday host James Brown told Fox News. "I've not seen any examples of the media pandering to a quarterback because he is black."

Radio talk-show host and columnist Armstrong Williams told Fox News it was unfortunate that Limbaugh inserted race into the discussion, but said that “If Rush was black he would’ve been saluted for spiking the ratings, he would’ve been given a raise and certainly wouldn’t have resigned. ... The media does go out of its way to celebrate black quarterbacks and coaches.”

• Video: Rush Under Fire

Leo Terrell, civil-rights attorney and fellow radio talk-show host, countered that the resignation was necessary.

"He made a racist comment and should resign," Terrell said, adding that pro football is a color-blind activity.

"You think the Philadelphia Eagles are going to put millions of dollars into a player who isn’t performing?" he asked.

Seven black quarterbacks started games last weekend. Two other black quarterbacks who regularly start were out with injuries.

Limbaugh, on his radio program, said people were angry with him because he was correct.

"If I wasn't right, there wouldn't be this cacophony of outrage that has sprung up in the sports writer community,” he said.

During a speech Thursday at the National Association of Broadcasters (search) convention in Philadelphia, the city where McNabb plays, Limbaugh said he’d been making pointed social commentary for years. The problem, he said, was not his.

“It’s not people like me who have put codes on speech and codes on thought, but they [his critics] band together and apparently they can ruffle enough feathers and scare enough people,” Limbaugh said. “On the radio program, none of this affects me.”

Limbaugh's initial comments sparked outrage from politicians, the NAACP and McNabb himself. Eagles owner Jeffrey Lurie (search) on Thursday accused ESPN of "institutional racism" for hiring Limbaugh in the first place.

Democratic presidential candidates Wesley Clark, Howard Dean and the Rev. Al Sharpton had called for ESPN to fire Limbaugh. Clark, a retired Army general, called the remarks "hateful and ignorant speech." Dean, a former Vermont governor, labeled them "absurd and offensive."

The NAACP called the remarks "bigoted and ignorant," and called for the network to fire Limbaugh or at least provide an opposing point of view on the show.

McNabb had said earlier Wednesday that he didn't mind criticism of his performance, but was upset that Limbaugh made his race an issue and said it was too late for an apology.

"It's somewhat shocking to hear that on national TV from him," McNabb said. "It's not something that I can sit here and say won't bother me."

Chris Berman, who anchors the ESPN show, said he did not believe Limbaugh's tone or intent was malicious.

"As cut and dry as it seems in print, I didn't think so when it went by my ears," Berman said. "I probably should have looked to soften it."

Limbaugh issued a statement on the remarks Wednesday, saying his comments were not racially motivated but instead were directed at the media.

"I offered an opinion. This opinion has caused discomfort to the crew, which I regret,” Limbaugh said. "I love 'NFL Sunday Countdown' and do not want to be a distraction to the great work done by all who work on it. Therefore, I have decided to resign."

Fox News' Orlando Salinas, Amy C. Sims and The Associated Press contributed to this report.