Updated

This is a rush transcript from "On the Record," May 23, 2013. This copy may not be in its final form and may be updated.

GRETA VAN SUSTEREN, FOX NEWS HOST: News tonight: Lois Lerner placed on paid administrative leave.

Former Speaker of the House Newt Gingrich joins us. Nice to see you, Mr. Speaker.

NEWT GINGRICH, FORMER SPEAKER OF THE HOUSE: Good to be here.

VAN SUSTEREN: OK, how does this happen, that the IRS targets different groups? What's your theory of why this happened?

GINGRICH: Well, there was a survey done back in 1991 in which three out of four IRS agents told the people doing the survey they thought they had the right to lie to Congress.

What you have is a gigantic government largely populated by people who are ideologically liberal -- look at the donation pattern of the IRS employees -- who believe they are superior to the American people, that we are subjects and they're in charge. You have a president who is a very weak administrator, great on oratory, terrible at managing.

And you have a system run amok, and it's run amok across the board. It's run amok at EPA. It's run amok at Fish and Wildlife. It's run amok at the Department of the Interior. It's run amok in the Justice Department.

But you're a normal American. You go to work. You pay your taxes. First of all, wouldn't you love the idea you can put on your return 5th Amendment. I'm not going to give you any details about my income or my taxes. I'm pleading the 5th.

And second, that if you do do something wrong, you could call the IRS and say, I'd like to apologize now. But look what's happened to Lois Lerner. She's put on administrative leave with pay. Now, why is this a terrible punishment?

VAN SUSTEREN: Well, according to Senator Grassley tonight -- and one of my colleagues tweeted, Chad Pergram -- is that the acting commissioner asked her to resign and she did not, so she was put on administrative...

(CROSSTALK)

VAN SUSTEREN: I suspect that's a civil service -- what -- I suspect that's the process. I don't know, that we don't...

GINGRICH: It is the process.

VAN SUSTEREN: Can you -- can you fire her?

GINGRICH: It's the process, which is why you have a Congress which ought to fundamentally overhaul the whole system.

VAN SUSTEREN: Well, yes, but...

GINGRICH: It's not just her. How many different people now have their hands on having fundamentally violated the legitimacy of our tax system? Every one of those people should be fired. And the idea that the union representative who, by the way, met with President Obama -- the union representative for IRS workers plays a major role in dealing with disciplinary problems. This is just the opposite of responsible self-government!

VAN SUSTEREN: Is this new?

GINGRICH: No.

VAN SUSTEREN: I mean, so -- so though -- so do we drop this in the lap of President Obama and say, as -- you say he's a weak manager? So can he be blamed for this?

GINGRICH: No. And in fact, I hope we don't blame him. This is -- this is not an Obama scandal. And it's a point that was made very brilliantly in a column in The Washington Post by a foreign policy expert, who said what's really frightening about the Benghazi mess is watching the bureaucracy, as we get all these e-mails, we get all these conversations. What you see is a level of incompetence in our national security system. When you look at Fast and Furious in the Justice Department, you see a level of incompetence.

VAN SUSTEREN: Well, let me bring that over to Congress, then. You talk about Fast and Furious. You talk about this, is that all these areas, the IRS, both Fast and Furious, there is congressional supposedly oversight.

GINGRICH: Right.

VAN SUSTEREN: The only time we hear about Congress in it is when there is some huge event, and then they come in and they investigate. Congress really does not do -- I would not give them an A in oversight because we'd catch these things a lot sooner.

GINGRICH: Look, it's also a failure of, I think, imagination. The civil service model we currently have dates back to the 1880s. It is 130 years old! And it's broken. It's broken in every department of the federal government.

Now -- and that includes the Pentagon. Now, what somebody should introduce is, you know, a smaller, manageable government act that would start with eliminating all of this massive security. These people are breaking the law, willfully breaking the law, arrogantly -- I mean, if you watched the attorney general the other day testifying to the Congress, he had the arrogance you'd expect, you know, from a duke or an earl in the aristocratic era. He did not have the attitude of a public servant.

VAN SUSTEREN: The president says that he didn't learn about this until about last week, with newspaper reports. Is that -- is that a problem? Does that show that he -- I mean, is that a -- was -- was the White House counsel and the aides and everybody else who apparently did know about it - - were they right in not telling him?

GINGRICH: Well, first of all, let's go back just for a second to Benghazi, where I saw one reference this week -- and you're an attorney and I'm not, but that the White House counsel apparently in October said, We had better slow down what we're saying because of the legal implications. You just had somebody go to the Congress and say, My lawyer told me I better plead the 5th because of the legal implications.

I think these folks are in so much trouble on so many fronts. And as you know, what always happens in Washington is the cover-up is worse than the event. And they have been lying to the Congress -- on every single one of these fronts, they've been lying to the Congress.

VAN SUSTEREN: But there are no consequences. I mean, if, indeed, you're right that they are lying -- and I have to tell you, there are some funny things said and we -- and as Speaker Boehner said, "Drip, drip, drip, drip, drip." I don't know why, when you ask for emails last November and they claim national security, you don't get them for six months, and when you finally get them, they certainly are not national security. They shouldn't have been classified. I mean, that happens (INAUDIBLE) Washington, drip, drip, drip, drip, drip, and everything gets hidden by national security classifications.

GINGRICH: Well, what the country should demand is very dramatic overhaul. Don't focus on one or two people. Don't focus on President Obama. Understand this is a system of big government run amok. It is very dangerous to your life.

I mean, the idea that the IRS is going to manage your health care, that they're going to have the ability to have insight into all of your records...

VAN SUSTEREN: Do you believe -- I mean, that -- that brings another issue because one of the women whose name has been brought up in this targeting of conservative groups and Tea Party groups is a woman who's going -- or who's at the IRS who's going to be handling the "Obama care."

GINGRICH: Look, I think they should move Congressman Tom Price's bill that would strip everything involving ObamaCare out of the IRS. I suspect it would pass the House in a huge margin, and I suspect it would pass the Senate because there are 14 Democrats from states Mitt Romney carried, and they can't vote to defend the IRS and go on and get reelected.

VAN SUSTEREN: And I know you weren't there, it's not your fault, but I mean, to put anything in the IRS like ObamaCare if you're trying to impress the American people, to put it in one of the most hated organizations, long before this scandal -- that was just stupid to begin with.

GINGRICH: Well, I think -- I think if you are a true blue liberal, you believe in government.

(CROSSTALK)

VAN SUSTEREN: ... Nobody -- I mean, look, I don't care liberal or Democrat, nobody likes that -- that -- you know, the notice from the IRS that, We're auditing you.

GINGRICH: I'm just reporting I think there's -- and remember, their defense of ObamaCare was that it was tax policy. So it's a little tricky for them to say to the Supreme Court, This is really tax policy, but it's not going to be in the IRS. Part of their defense was that it was in the IRS, and that was part of Chief Justice Roberts's decision that it was constitutional, was that it was tax policy.

VAN SUSTEREN: All right, what should -- what should Chairman Issa do? Should he recall Lois Lerner back?

GINGRICH: I would recall her and I would make her sit there and I would ask her every question and I would allow her to say 600 times.

VAN SUSTEREN: Why?

GINGRICH: Because the country needs to watch a senior civil servant arrogantly say to us, I believe I may be so criminally vulnerable that I'm not going to tell you a thing.

VAN SUSTEREN: And then bring the other ones in from Cincinnati and...

GINGRICH: And do the same. And bring in the woman who's currently at the ObamaCare section who's really part of the original gang and bring in the people who apparently in Washington are coordinating.

Again, this all keeps coming out. The more you learn, the bigger it's going to get, the more people there are that ought to be called. I would also bring in every group that was cheated and have every single group testify. I'd go through days of people saying, We did this, and this is what happened to us.

VAN SUSTEREN: And then what's at the end of that?

GINGRICH: Oh, I think -- I think they should strip ObamaCare out of the IRS. I think they should dramatically shrink the IRS. And I think we need to, frankly, fundamentally repeal the civil service laws and go to a manageable government where when somebody is as bad as these people, you can fire them.

VAN SUSTEREN: Let me ask you about Benghazi. Is that a problem for the president? Beyond the fact that it's a distraction, is it a more serious problem?

GINGRICH: It's a serious problem on a couple of levels. One is, what did -- what was the conversation at 10:00 o'clock at night between he and Secretary Clinton. What did they discuss? What did he know? Get back to Howard Baker's old question, what did he know and when did he know it?

VAN SUSTEREN: Deliberative process is the -- is -- he doesn't have to tell you.

GINGRICH: Yes. The second thing is when the spokesperson for the State Department says in one of the conversations, The leadership of my building is very unhappy with this analysis, it leads you to two great Clintonian questions. What did the term "my building" mean? I think it meant State Department. And what did leadership mean? I think that meant Secretary Clinton.

So Secretary Clinton needs to be called back in to say, You got -- we are told by your own person that you were unhappy with this. What is it you were unhappy with? And why did you not tell us that, in fact, you put pressure on the intelligence community to change what they're doing?

VAN SUSTEREN: And so the end point of this?

GINGRICH: Well, first of all, the end point is, I think, as David Ignatius said in his column last week, to realize how really badly bureaucratic and incompetent our national security apparatus now is. And again overhaul it. I mean, cut through all this stuff. It's not about chasing this week's stupidity. It's about trying to learn what do we have to do to get to a smaller, leaner, more effective, more accountable government?

VAN SUSTEREN: That's not going to happen.

GINGRICH: See, I'm much more of an optimist than you are.

VAN SUSTEREN: Well, I -- no, I mean...

GINGRICH: Americans have enormous capacity to change things.

VAN SUSTEREN: I mean -- I mean, you look at Washington, I mean, it's, like, you know -- you know, you sort of blend into all the huge waste and everything else going on...

(CROSSTALK)

VAN SUSTEREN: It's just a nightmare!

GINGRICH: I look at guys like Scott Walker and John Kasich and Bobby Jindal and Rick Perry. There are lot of state capitals where you're seeing real reform and real change, and that gives me hope that some day, we could do it here in Washington.

VAN SUSTEREN: All right. Let me turn to something a little bit closer to home. Our James Rosen had his emails -- you shake your head.

GINGRICH: Well, look, every once in a while, Roger Ailes gets lucky on a grand scale. I mean, Ailes had the president of the United States target FOX, attack FOX, design a system to discredit FOX, not allow people to show up -- all these different things because he had decided that this was a conservative channel and he was going to make sure it didn't matter.

And now you find out, at a level that is grotesque, that the Justice Department was targeting a reporter at FOX. I mean, when even The New York Times writes an editorial defending FOX News, you know that you're in one of those amazing moments in history. And I think that everybody who's involved ought to feel that FOX has been pretty solidly vindicated. I mean, these guys apparently were wiretapping his mother and father? Now, how do you justify this?

VAN SUSTEREN: Phone records, I think, just the call records.

GINGRICH: (INAUDIBLE) call records. It was mother and father. Now, my only point is, first of all, going after reporters -- even the president today said, Gee, we don't want to make it criminal for reporters to do their job. Well, he then ought to fire the attorney general because the attorney general sure as heck has been consistently going after reporters.

And I think that here what you have is a remarkable vindication of FOX News because you had somebody doing their job, apparently doing it well enough to rattle the national security apparatus, and somebody who was literally doing exactly what a good reporter should do on a nonpartisan basis.

This was not a political story. This was a national security story. And I think that Mr. Rosen ought to take some real comfort in the fact that he becomes a symbol of what American journalism's all about.

VAN SUSTEREN: You know, it's interesting. I pulled a couple -- I went back to October of 2009, CNN Howard Kurtz, October 11th. And Anita Dunn was on and she was talking about FOX and she said, The reality is that FOX News often operates almost as either the research arm or the communications arm of the Republican Party, which -- you know, look, if you go back, you find the Obama administration has constantly been unhappy with FOX News, and you know, even excluding us from a State Department conference call on Benghazi, excluding us from a CIA briefing on Benghazi. So this is -- this is not something...

GINGRICH: No, I mean...

(CROSSTALK)

VAN SUSTEREN: Not a huge surprise.

GINGRICH: And I think, frankly, everybody at FOX ought to feel pretty vindicated that you've now seen the level a hostility which is unconstitutional and I suspect illegal, and that there should be a certain pride in having Rosen as one of the team here because he did exactly what he should have done. And he will become a symbol of freedom of the press in this process.

VAN SUSTEREN: It's fascinating to see the other news organizations, you know -- you know, coming out on this, I mean, talking about...

GINGRICH: Well, I think everybody across the system on every single network and in every major newspaper realizes that when you start having criminal potential proceedings and you start having secret gathering-up of information against reporters, you have really crossed the line of the 1st Amendment, and you threaten every single news-gathering organization in America.

And I'm frankly delighted that people who may be deep ideological opponents of FOX have come to their defense and I would hope that the entire team from Ailes on down would feel a real sense of pride that this became a 1st Amendment fight and that nobody allowed FOX to become isolated.

VAN SUSTEREN: Mr. Speaker, thank you, sir.

GINGRICH: Good to be with you.