Updated

This is a rush transcript from "On the Record," May 30, 2013. This copy may not be in its final form and may be updated.

GRETA VAN SUSTEREN, FOX NEWS HOST: Karl Rove joins us now. Karl, let me pick up where I was talking with Congressman Allen West, on whether or not you think that Attorney General Eric Holder will survive.

KARL ROVE, FOX NEWS CONTRIBUTOR/FORMER BUSH SENIOR ADVISER: You know, I'm not certain he will. This meeting he has tomorrow is clearly an attempt to -- you know, it's a PR gesture more than it's a substantive gesture, and I don't think it's going to repair his relationship with the press, nor is it going to give the American people greater confidence in his veracity.

I mean, we -- if this was the first time that we had had questions about his ability to do the job, it'd be one thing. But it's not. We've had many questions, from Fast and Furious to his handling of the New Black Panther Party case, to the handling by the Justice Department of the St. Paul housing disparity case, and now this and a variety of other issues.

And I just -- it is -- President Obama is ill served by keeping him there. They need to find a way to have him go.

VAN SUSTEREN: You know, it's so disappointing for me to watch him because I will tell you -- and I knew Eric Holder back 20-some years ago, when he was a judge on the superior court of the District of Columbia. And I will tell you he was a very well respected judge. He was always courteous to both sides, the lawyers, the prosecutors, defense lawyers. He was respectful to clients. I mean, he actually had an incredible reputation as a judge.

And now fast-forward until now, and it seems like it's one blunder after another.

ROVE: Yes. Well, sometimes people are in the wrong place, and I think he's in the wrong place. He may have been an admirable judge, but he has been an ineffective, disorganized attorney general who has made very bad decisions and demonstrated very bad judgment.

And this meeting tomorrow is -- look, this is a real problem. I mean, first of all, you're supposedly trying to meet with news organizations to discuss the Justice Department guidelines regarding investigating journalists, and you will not allow them to bring their lawyers? You only want to meet with the bureau chiefs who affect news, you know, not the management.

You know, there was a very interesting piece today by Ron Fournier, who is -- who at the Associated Press covered the Bush White House, a very tough but fair reporter now at The National Journal, and he made some very good points about this -- the meeting, that it wasn't in the interests of either Holder, nor was it in the interests of the media to attend. He said, Look, anybody who shows up at this is just going to simply add to the sense that the media are lapdogs in the tank for the administration.

What kind of conflict is it going to be to have the editor of -- of -- the bureau chief there, with reporters not there who are the people charged with finding out what's in the meeting? You know, the irony of the attorney general saying, you know, I'm going to investigate myself, and I want you to come in and talk to me about the guidelines, but we got to keep it all secret.

So he had some very tough language about how no self-respecting journalist ought to be attending this meeting.

VAN SUSTEREN: I'll tell you what I think is -- is -- you know -- you know, we all sort of look at it sort of from different -- you know, different perspective. I think what's particularly deeply disturbing to me is that on May 13th, the information came out about the Associated Press records. Two days later, he testifies before the committee, and he makes a statement that at best is misleading, at worst was just a lie, because then we learn a couple days later about James Rosen.

That to me, for the attorney general of the United States, without further explanation, you know, is deeply disturbing. He needs to -- he needs to make a real fast trip back to Capitol Hill. And if there's an explanation, he needs to lay it out for the American people...

ROVE: Well...

VAN SUSTEREN: ... because -- and -- you know, he's...

ROVE: Yes, Greta, you're right, there was a -- (INAUDIBLE) time for a fast return to the White House, but we are now on the 30th of May! You're right, he went up in front of the House Judiciary Committee on the 15th, basically said, I've never been involved in anything like this and it'd be bad policy.

It was that weekend that NBC with Mike Isikoff broke that he had been the person who had authorized the subpoena on Rosen. What he should have done is, gone back up on Monday and say, I need to set the record straight.

But instead, what we get is we get an entire week in which he's silent. Then we get him supposedly reading The Washington Post story this Monday and showing, quote, "remorse" and starting to feel bad about it, and we have yet to hear him clean it up with the Congress!

And not only that, but he's also dragging the White House down with him. We had this thing yesterday with Jay Carney, who had to sit there and say -- who had to sit there and say that he saw no conflict between what Holder said on the 15th and his actions earlier in the Rosen affair. And then the classic was -- he got pressed earlier in the briefing on it, and he said, You're conflating a subpoena with prosecution.

Now, think for a moment about what he's trying to suggest there. He's trying to suggest, Yes, we can in seeking a subpoena declare that somebody is a criminal, that they're part of a criminal conspiracy, but we really didn't mean it because the only way we'd really mean it is if we actually accused him of that in a prosecution.

Now, you know, in essence, what he's saying was, We were either lying to the judge in order to get a subpoena we shouldn't otherwise approved -- get it approved that we shouldn't have otherwise gotten, or we were just -- we were getting ready to, but you shouldn't hold us responsible unless we actually decided to follow through on it by prosecuting James Rosen.

So I mean, the White House is being forced to defend this guy. And at some point, the president's got to say, Look, I love Holder, he's my friend, he's my compadre, he's my running buddy, but he's hurting me and he's hurting out administration and he's hurting the country and it's time for him to go!

VAN SUSTEREN: And I guess -- and -- and as I'm stuck on -- he needs to tell us, you know, did he -- did he lie on -- you know, about -- you know, before Congress because that -- that's a -- you know, that's unforgivable if he lied.

ROVE: Look, he lied. Let's be -- look, there's only two explanations. He lied or he has one of the worst memories in history! You know, there are only two explanations. Now, they tried to -- they tried to get this very sophisticated one that, No, he was really answering narrowly...

VAN SUSTEREN: Karl...

ROVE: ... a question of Hank Johnson. Look, it just didn't -- it doesn't fly!

VAN SUSTEREN: There's no -- it's just impossible. It's impossible not to remember going after the records of the media. It's just impossible for the attorney general because...

ROVE: It is impossible.

VAN SUSTEREN: That's impossible.

ROVE: I agree with you.

VAN SUSTEREN: Anyway, Karl, stick around...

ROVE: Look, I had the same reaction...

VAN SUSTEREN: Go ahead.

ROVE: All right? I had the same reaction when he talked about Fast and Furious and said he was unaware of it. And we know a lot of paper crossed the desk of his chief of staff, and he claimed never to know about it. So either he was lying then, or he was incompetent in not running it.

But you're right, in this one, I just cannot believe that having signed off on that subpoena and having supposedly lived up to the guidelines of the Justice Department in doing so that he would not be able to recall it.

VAN SUSTEREN: Karl, stick around. We have much more to talk about, including why did the former IRS commissioner visit the White House more than 150 times? And did the targeting investigation ever come up in any of those trips? That's next.

========================================================

VAN SUSTEREN: Listen to this. Former IRS commissioner Doug Shulman says he never spoke to anyone in the Obama administration about the targeting investigation at the IRS, but he did visit the White House at least 157 times. That is way, way more than any other cabinet member. Of course, he would need to meet with White House staffers about rolling out ObamaCare, but even HHS Secretary Kathleen Sebelius visited the White House only 48 times. That's a third less as often as former IRS chief Shulman.

Karl Rove is still with us. Karl, you know, I know that the IRS chief had to go talk about the implementation of ObamaCare through the IRS. But 157 times, when Sebelius was only 47, Hillary Clinton only 43 as secretary of state. Your thoughts about these numbers? Should we be suspicious or curious?

ROVE: We should be very curious and very suspicious, curious because that's an odd number of times. I mean, I was in the White House for nearly seven years. Mark Everson, who used to be over at Treasury, I believe -- once he went to the IRS, I never saw him again. I mean, he -- he's on record as saying he came to the White House one time when he was IRS commissioner.

So the idea that the IRS commissioner is coming to the White House every nine days ought to raise warning flags right off the bat because this is an independent agency that needs to have distance from the White House. It shouldn't be, you know, part of the White House, it should be distant from the White House.

We know what's happened before when the White House has gotten too close to the IRS, and the IRS has gotten too close to the White House.

We should also be very dubious about this because of the way that Mr. Shulman handled it. When he was asked in front of Congress, at that time, they knew about 118 appearances at the White House, he responded -- and they said, Well, what were you doing there, and his first response was, Well, I was at an Easter egg roll. Well, that's not a very, you know, good way to explain why you were there every nine days during your tenure as IRS commissioner for nearly four years.

So I'm very dubious about this. I think this is a question that we need to have a lot of answers to. And the answers are going to be difficult because this is the point at which the White House is going to assert executive privilege and say he was inside the White House, giving policy advice as part of the White House policy process, and that's none of your business, Congress, what that was.

But in this instance, I think the perception of White House involvement in this is going to require the White House going a little bit further than they might otherwise be inclined to go.

VAN SUSTEREN: All right, before we move on to one another issue, I just want to say that I would not be very, very curious if it weren't for the fact that -- if Secretary of Health -- of Health and -- HHS, Secretary Sebelius, had about the same number of visits, I would not be suspicious because they have a need to work together on this implementation. But the disparity has got me.

All right, now we go to Dick Durbin. Dick Durbin investigated your -- sent a letter to the IRS about your group, and you've written an article -- or a column in The Wall Street Journal.

What's your beef with Dick Durbin?

ROVE: Well, my beef with Dick Durbin is he went on "FOX News Sunday," and Chris Wallace asked him a question, which was in October -- on October 10th, 2010, you wrote a letter, Dick Durbin, to the IRS demanding that they investigate Crossroads GPS, a conservative policy group.

And Dick Durbin, to his credit, didn't back down like a lot of these other Democrats are. He said, yes, I admit I asked the IRS to target a conservative group. And Chris followed up and said, Well, why didn't you mention any liberal groups? And -- and -- and then Durbin had this laughable response, which was to say, Well, I thought that if the IRS investigated Crossroads GPS, all the rest of them would get the -- would get the message.

And look, that's simply not true. If the IRS was investigating Crossroads GPS, a conservative 501(c)4 group that I helped organized -- I encouraged its organization and I help raise money for it as a volunteer -- the only people laughing would be liberal 501(c)4s, who have been operating for decades in exactly the way that Crossroads GPS did.

And Durbin was -- you know, Durbin had no real good defense. I mean, it was clear he wanted to target Crossroads because we're a conservative group and he's a liberal. He said, Well, you know, they -- they raised -- they're boasting about raising a lot of money. Well, that's simply not true.

The only thing in the fall of 2010 that came close to me boasting about the money that GPS was raising was on October 13th, after he sent his letter to the IRS, two days after he sent the letter to the IRS, I wrote in a Wall Street Journal column that after President Obama had attacked our group as a threat to democracy, American Crossroads and Crossroads GPS has raised $14 million in two weeks.

And that -- you know, look, the Democrats have had these kind of groups around forever. And they did not complain when the NAACP voter fund or 501(c)4s connected with the Sierra Club, Planned Parenthood, National Abortion Rights Action League, League of Conservation Voters, Wilderness Society, Defenders of Wilderness, VoteVets...

VAN SUSTEREN: And...

ROVE: ... on and on and on -- they never criticized them! And in fact, one of these groups was formed by an aide to Durbin, Bill Burton, who founded Priorities USA, and I never heard Durbin complain about that in the 2012 election!

VAN SUSTEREN: Well, the big question is whether he sent a letter of the IRS about -- about that group, Priorities. But anyway, thank you, Karl. We got to go.

ROVE: No, he didn't! You bet. Thank you.