Confessions of a former IRS commissioner

This is a rush transcript from "On the Record," May 29, 2013. This copy may not be in its final form and may be updated.

GRETA VAN SUSTEREN, FOX NEWS HOST: Former IRS commissioner calling the IRS failures catastrophic. Mark Everson ran the IRS under President George W. Bush. He joins us. Nice to see you, sir.


VAN SUSTEREN: All right, what we've been talking about all day long is the number of visits that Mr. Shulman, as head of the IRS, made to the White House, a number of visits. How many visits did you make to the White House when you were commissioner?

EVERSON: Oh, I don't have an exact figure in mind. But what I said last year, when I testified before the same House committee chaired by Mr. Issa, was that to my recollection, I only went over there once to discuss policy matters on an interagency basis.

The contrast being -- what I was getting to at the time, Greta, was that I was concerned about the major role that the service plays in the Affordable Care Act, and the fact that it could potentially compromise the independence of the agency when you have the commissioner and his subordinates having to meet on a regular basis with officials over in that part of the government.

VAN SUSTEREN: All right, so the fact that Mr. Shulman met 118 times - - at least, that's the report we have -- at the White House when he was the commissioner, I guess part of the reason is, is because of the unique aspect that "Obama care" was being essentially implemented in part through the IRS. Is that a fair description of it?

EVERSON: That's fair. And I think that the commissioner should have answered the question differently when posed because that's totally legitimate. Now, you may say, as I said at the hearing last year, I expressed my concern about the fact that the service was given such significant responsibilities under the Affordable Care Act.

But I kind of got it wrong, Greta. I said that I thought that the controversy of the Affordable Care Act would potentially damage tax administration as people filled out complicated returns and did all the things they had to do this terms of that legislation. But it now appears, frankly, that the Affordable Care Act is being made more controversial because of what's happened most recently at the service.

VAN SUSTEREN: What has happened to the IRS? I mean, how do we have the situation where they are now being accused -- and the evidence has been quite -- you know, is quite powerful that there's some targeting, and there's a lot of question either whether it's simply low level or whether it went higher up. But how does this happen?

EVERSON: Well, let me first say that my experience at the service was that from top to bottom, people try to do their job impartially. They try to get to the right answer, and certainly, to steer clear of politics. So I am shocked and saddened by all these events.

But having said that, it's very serious because, one, for the service, and secondly for the whole country, because the IRS interacts with so many citizens and it's so -- such a fundamental part of the government.

There are three things, though, to get to your question. First, I think there were some people down in Cincinnati who just sort of, for lack of a better term, lost their minds. And they used criteria to review applications for exempt status that were just wrong and they were biased.

Secondly, at the intermediary level, where Lois was supervising the exempt organizations, it took two years to get this right. That's unconscionable. There were clear problems starting in March of 2010, and they should have been resolved promptly.

And then the third thing, what was happening in the front office? There was no secret here. Greta, I was getting calls in '11 and '12 from people saying, What's going on at the IRS with the (c)4s? I declined to answer that because I didn't know much about it.

But obviously, it was of great interest to many people, with congressional letters going in. The service -- the front office should have learned about it. Shulman's testimony before Ways and Means should have been different, and then the record should have been corrected as soon as he knew that his testimony had been incorrect.

VAN SUSTEREN: Well, what I think is so appalling about this -- I actually think he did know, and I'll tell you why. I'm looking at his testimony and he says, I think there's been a lot of press about this, meaning the targeting. And later on, he says, This is what happens when -- sort of normal back and forth that happens when someone makes an application for a tax extension.

If I'm the commissioner of the IRS and I go to a lunch and I hear people complaining about some unfair aspects of the IRS, I'd go back and I would think -- I'd find out what in the world was going on. He admits he knew about it from the press. So that's notice at the very top. And you would think that he would have had enough sense to say, What in the world is going on?

I mean, I think that's his obligation. And I don't think he just sits and waits for it to sort of trickle up to him.

EVERSON: No, I agree entirely with what you just said. When you -- when you're running a component of government like that, you set some priorities. But you also, of course, respond to events. You are accountable to the Congress. The Congress was raising issues, and frankly, you're foolish if you think that the media, when it raises issues, isn't going to figure out what's going on. So it's very much in your own interests and your agency's interests to get on top it, and that apparently did not happen here.

VAN SUSTEREN: But it's not even just on your own interests. It sort of is the job. You know, it's not even -- it shouldn't just be sort of...

EVERSON: No, you're right.

VAN SUSTEREN: ... a selfish thing. It shouldn't be just what he's worried about himself. If he hears -- I mean, there's nothing, you know, you can think of is more fatal to the IRS than the idea that they're going about targeting people. I mean, there's nothing sort of -- I mean, nothing more vicious, trying to scare people and their livelihoods and everything else.

He heard about it in the press, and he -- and at best, he can says it's sort of the normal back and forth. And when I read that, you know, my head almost exploded, you know, because didn't he care?

EVERSON: Well, what you do if you're in a situation where Congress is asking you difficult questions and you're not sure, you say, You're right. This is an important issue. Let me look at it and come back to you.

But what was issued was an adamant denial that there were any problems, and then subsequently, in a very short period of time, it was known that there were problems. And what should have happened there, Greta, was the record should have one corrected.

And the idea there was an IG audit under way, again, as Russell George has said, not an investigation, that does not excuse inaction by the IRS because I can just tell you, if you don't get into managing the agency in the areas that TGDA or GAO have under audit, they have 20 or 30 audits under way at any given point in time. You're not the going to be doing your job, as you said. So he should have taken a firm hand on this.

VAN SUSTEREN: I'm looking at Mr. Shulman's answer -- and this was some time ago. This was back in March of 2012. And in the same answer, he says -- he's asked about the political targeting, the targeting of certain groups with political leanings. He says, Yes, I think there's been a lot of press about it. He also says, Well, that's the sort of normal back and forth that happens with the IRS, and then he goes on to say, There's absolutely no targeting.

Nothing could be dumber or wrong, more incorrect than what he says, that's the head -- that -- that's what he was doing to a lot of Americans, as the head guy. So I don't think he can just sort of dump this one off on some rogue employees in Cincinnati. You know, this went all the way to the top!

EVERSON: Well, I think, as I've indicated, I think there were three levels of problems. There were problems at the bottom. There were problems at the intermediary level and there were problems at the top.

What disappoints me, Greta, is -- and you may remember this -- we had a very active program in 2004 and 2006 that I launched to look at churches and charities that were accused of politicking in terms of the '04 and '06 election cycle, (c)3s this revolved around mostly. They can't help a candidate or oppose a candidate.

We set up a unit also within TEGE, this governmental tax exempt group. It was out of Dallas. It wasn't at the front end of the process, where the people are doing the work in Cincinnati looking at the applications, if you will. But we did it and wed id it quickly.

And then right before the election, the NAACP launched a broadside saying that we were trying to quash the black vote. I called up this same Russell George and I said, Russell, can you take a look at this? I think we've done it correctly. Nobody will believe, though, that we did.

He did an audit of this, same kind of thing he did on this most recent problem, and he came back in February of 2005 and he said, Well, the service could have moved quicker here and they could have had better guidance, but in fact, there was no bias here, and it was balanced as to how they selected organizations for follow-up.

So I don't get it because I know the service can do this correctly. They did when we did that work in '04 and '06.

VAN SUSTEREN: Actually, I would say that the commissioner did it by asking for the investigation. I don't see that happening here with this one. So anyway, thank you, sir.

EVERSON: Thank you.