Press Secretary Robert Gibbs responded for the first time on-camera Wednesday saying he wouldn't resign amidst the backlash over comments he made about what he called "the professional left" not being happy enough with Obama administration.

Gibbs said, "I don't plan on leaving" when questioned about charges he should resign.

The press secretary was under a political firestorm from those within his own party and for the second time this summer has had to defend comments he made about Democrats.

Rep. Keith Ellison,D-Minn., started chatter Tuesday that he should possibly step down, saying that it would be an appropriate response.

Rep. Alan Grayson, D-Fla., took it even further that he should be given the ax. "I don't think he should resign. I think he should be fired. He's done a miserable job, said Grayson on MSNBC Wednesday.

"People I know refer to him as ‘Bozo the Spokesman.' ....He's so far in over his head he'd have to reach up to touch his shoes," he added.

Gibbs said he hasn't made any apology calls, and did not speak about the issue with the president.

The press secretary initially told "The Hill" newspaper that the "professional left" wasn't recognizing accomplishments the Obama administration has made. He said, "They will be satisfied when we have Canadian healthcare and we've eliminated the Pentagon. That's not reality."

Gibbs was absent from Tuesday's White House briefing because he said he was suffering from a cold.

Wednesday he was peppered with questions from reporters asking if he had any regret. Gibbs largely stood by the interview, but noted again he watches a lot of cable TV and that the comments came from frustration.

He gave the example that many times he reads transcripts from briefings and thinks he could have said things slightly differently.

When asked to define the "political left," Gibbs did not elaborate on who fit into that definition.He brushed off claims that there is too much pressure from the left and noted things like health care and drawing down Iraq as markers of success from President Obama.

Gibbs said he hadn't read the claims that at least the White House could have greater communication with the left and at least attempt to explain why they can't get everything done, but they are trying.The initial interview was layered with more than just one or two criticisms -- he made several charges, including, "I hear these people saying he's like George Bush. Those people ought to be drug tested," Gibbs said. "I mean, it's crazy."

Blogs posts and political commentary erupted soon after, and Gibbs was having to backtrack his statements, saying that he watches too much cable news and was "inartful" and called for greater party unity.

MSNBC liberal host Keith Olbermann weighed in saying that on Tuesday, "The White House has seemed more like the amateur left."

Olbermann said the administration should rather be looking at the "professional right." He noted he wasn't sure if he was part of the "professional left," and doesn't necessarily want to be, but happens "when others don't do their jobs."

The press secretary said he didn't see the show, and was instead watching the Atlanta Braves baseball game online and monitoring primary returns on his blackberry.

Gibbs had also mentioned in the interview that liberals would never be happy, saying, "They wouldn't be satisfied if Dennis Kucinich was president," Gibbs said.

Rep. Dennis Kucinich, D-Ohio, responded to Fox News Radio and said of the comments, "From a political perspective it wasn't very smart." He didn't call for his resignation and didn't seem offended -- but added, "It's important for the President's spokesperson to understand that his remarks are taken quite seriously and when he speaks I would hope that he speaks with the purpose of unifying."Earlier this summer Gibbs had to walk back and clarify comments after disappointing Democrats saying on NBC's Meet the Press that his party had a good chance of losing the House in November.