Updated

This is a rush transcript from "Your World," May 1, 2013. This copy may not be in its final form and may be updated.

NEIL CAVUTO, HOST: Benghazi wasn't the only crisis the administration was dealing with today, and Jay Carney was getting questions on today. There was this other little issue of the IRS. Have a look.

(BEGIN VIDEO CLIP)

JAY CARNEY, WHITE HOUSE PRESS SECRETARY: We certainly find the actions taken, as reported, to be inappropriate. And we would fully expect the investigation to be thorough, and for corrections to be made in a case like this. And I believe the IRS has addressed that and has taken some action. And there's an investigation ongoing. But it certainly does seem to be, based on what we have seen, to be inappropriate action that we would want to see thoroughly investigated.

(END VIDEO CLIP)

CAVUTO: All right. Inappropriate or illegal. If you guys aren't aware of this, let me bring you up to speed what the IRS has already admitted to doing -- inappropriately flagging conservative groups last year for additional review, in other words, reviewing their returns, their tax exempt status, seeing if they were doing anything out of the normal, whether they were violating things, their tax status, you name it.

The IRS actually singling them out and now apologizing for it, essentially saying: "That was wrong. That was absolutely incorrect, it was insensitive, and it was inappropriate. That is not how we go about selecting cases...The IRS would like to apologize for that." To Tea Partier Herman Cain, who says, apology not accepted.

You know, Herman, I'm no lawyer, but that is not only inappropriate, that's a hell of a lot worse. I mean, what do you think of that?

HERMAN CAIN, JOBCREATORSNETWORK.COM: I like your word, illegal. It is illegal to harass an individual and it should be illegal to harass an organization itself. When it comes to the IRS, they have gotten away with it, Neil, for decades. What surprised me most wasn't the fact that they had done this. But what surprised me most, they apologized for it. This is unprecedented in itself.

The other issue here is that there's no disincentive for the IRS at the present time to do this again. Neil, I had it happen to me in the early 1990's when I was -- when I criticized Bill Clinton relative to "Hillary- care." I mean, I'm not crying spilt milk at this point, what I'm saying is, I have talked to a lot of people, probably like you have, where this goes on all the time.

So when the president says, through Jay Carney, that there would be a thorough investigation, I'm not convinced. Where was a thorough investigation on Benghazi? And every time that Congress continues to push on Benghazi, we find out that the investigation wasn't so thorough, and they -- and Jay, in his comments, is still trying to ignore the testimony given by three very credible witnesses just a couple of days ago.

CAVUTO: You know, I'm looking at what the IRS has said in this apology. But when it starts singling out conservative groups, to just sort of mess with them and their tax status, and whether they're doing everything right, anyone who has gotten a letter from the IRS knows -- whether you're Republican or Democrat, it's from the IRS, you're scared.

And that is intimidating. But obviously someone higher up has to make the call. That could be a high-up at the IRS who is, you know, a liberal and hates conservative groups, just as it was during the Nixon years when this sort of stuff went on, and it was a conservative who didn't like liberal groups.

Notice the IRS is supposed to be an independent agency, an arbiter of all things green, not red or blue, doing this. I mean, that, again, goes way beyond what is inappropriate or unwise. That is like the least taste of weird -- weird thing, you know?

CAIN: Neil, I agree with you 100 percent. I just finished talking about the book, "1984" by George Orwell, where the Big Brother controlled everything. The IRS is the epitome, based not only on that incident, of Big Brother at its best. I even had --

CAVUTO: But, Herman, think of this, think of this. And my producer is reminding me of this. The IRS is going to be in charge of enforcing a number of these health care provisions. So it will have a much larger -- they're hiring thousands of IRS agents as we speak. And I can just see my policy right now, I'm dead.

But my point is that they're going to have this role that is going to be far greater, grander, more intrusive potentially than ever before. Timing, shall we say, bad.

CAIN: Timing is terrible. According to reports, the IRS hired an additional 16,000 agents just to focus on being the health care police. They have taken -- the ObamaCare legislation, which was shoved done our throats, has now taken Big Brother, the IRS, and made it even bigger and more intrusive in our lives.

What needs to happen, Neil, is two things. Number one, Congress needs to intervene and put up some sort of safeguards to help the American people such that this harassment does not happen. But the long-term solution, as you know, I have been talking about a long time, replace the tax code such that they don't have nearly 80,000 pages of ways that they can harass individuals and harass organizations that appear to be conservative.

CAVUTO: All right. Thank you, Herman Cain.

Content and Programming Copyright 2013 Fox News Network, LLC. ALL RIGHTS RESERVED. Copyright 2013 CQ-Roll Call, Inc. All materials herein are protected by United States copyright law and may not be reproduced, distributed, transmitted, displayed, published or broadcast without the prior written permission of CQ-Roll Call. You may not alter or remove any trademark, copyright or other notice from copies of the content.