Updated

This is a rush transcript from "On the Record," July 3, 2014. This copy may not be in its final form and may be updated.

ANDREA TANTAROS, FOX NEWS HOST: Right now, the news President Obama is trying to forget this July Fourth weekend. The latest Quinnipiac poll showing that 45 percent of voters say America would be better off if Mitt Romney had won the election. Why do they say that?

Karl Rove joins us now. Karl, welcome.

Karl, you have been in the White House. You get a poll like this right before the holiday weekend, where is the first place your mind went when you saw this poll?

KARL ROVE, FOX POLITICAL ANALYST/FORMER BUSH SENIOR ADVISER: Well, if I were them, I would leave the poll on my desk and go home and try to have a nice weekend.

But, look, this is buyer's remorse. In one sense, it's understandable. President Obama did not win re-election of the basis of some compelling and optimistic forward-looking vision. He won by eradiating Mitt Romney and making him unacceptable to a large number of America. That's why President Obama is the only president in the history of the country to get elected with a smaller number of votes than he got the first time around. Never happened before. Every other president that's gotten reelected gets more votes the second time around. In fact, George W. Bush got 25 percent more votes in 2004 than he got in 2000. President Obama got 97 percent in 2012 of what he got in 2008.

TANTAROS: Karl, if you dig deeper in this poll, it shows he has lost across the board, even with Independents, with women. What I thought was interesting was liberals, as well, he went down seven points. What's prompting that?

ROVE: Liberals are getting conflicted because they had such aspirational goals for him. They wanted him to be a certain kind of leader that he has turned out to be.

Look, the president's ability to be a strong and effective leader has been diminished in large part by his own decisions and his own conduct. We saw it this week when he sort of looked like he was in a high school locker room slapping his towel around when he gave those remarks at the Key Bridge in Washington, D.C., in which he sort of taunted Speaker Boehner, "Sue me," and called the Republicans unpatriotic, said it was time for them to get some economic patriotism, and where he defined his proposal for highway funding by saying it's not socialist, you know, it's not unconstitutional. I mean, the president's tone is simply wrong. And it's not only hurt him among conservatives and moderates, but also causing some liberals to say, is this the best we can do.

TANTAROS: Karl, how much do you think the scandals are affecting these numbers and hurting his polling?

ROVE: Well, I wrote about this today in my "Wall Street Journal" column. Because this is mystifying to me. Just take the IRS scandal. When it first broke, the president's rhetoric was improper, inappropriate. He said, this is inappropriate, it shouldn't happen, the IRS needs to be completely unbiased and neutral, and we need to get to the bottom of this. And now he is dismissing it as a phony scandal. In the meantime, he had blown it off by saying, well, it was a couple of bureaucrats in Cincinnati trying to enforce the law and I don't see the big deal about it. He started out in the right place. He is in a wrong place today because, like in the recent FOX poll, three quarters, 76 percent of Americans believe that Lois Lerner's emails were deliberately destroyed. So the president is sitting there saying, don't worry about it, nothing to it, nothing at all, when three quarters of the American people think there might have been a criminal act involved. This is not helping him. And these scandals have eroded, I think, the public confidence. His job approval now is about, I think, 40 percent in the latest Gallup. And that's a pretty big drop from where he was not too long ago.

TANTAROS: It seems like, Karl, it was the IRS scandal. It really started to be the impetus for those poll numbers back in the sprint to start to nose dive. Now we see it with other issues. Foreign policy, he's getting incredibly low marks, domestic policy.

But I want to ask you this. A lot of people are encouraged when they hear this buyer's remorse for Mitt Romney. How much does that translate though into the next election, because I thought, OK, great, you didn't vote for him? How much does this help Republicans the next time around, the party that put up a candidate like Mitt Romney?

ROVE: Well, it really doesn't. These things are not transferable from candidate to candidate. But it is an object lesson. It says Republicans better come up with a candidate who can shape an optimistic and forward-looking vision that allows him to say, we can do better than the Obama years, we can do better than what the Democrats are offering, we are going to have a positive and optimistic growth agenda that is going to include every American. This was like, in 1979 and 1980, it was critical not only to be able to criticize the mistakes of Jimmy Carter. It was really vital that Ronald Reagan offered positive and optimistic agenda for the future, largely built around a strong national defense and the Kemp- Roth (ph) tax cuts to get the American economy moving again. There is an object lesson in here for the Republicans but don't count on the dissatisfaction with President Obama naturally leading to the election of a Republican. It does create a fertile opportunity for it but it doesn't guarantee it.

TANTAROS: And real quick -- I only have about 15 seconds, Karl -- but what about the midterms? How should Republicans use these polls?

ROVE: Well, the Republicans have to make a closing argument this fall that the American people have a choice, do you want to give President Obama a vote of confidence by electing Democrats or do you want to provide a check and balance on him during his last two years in office and send him a strong message it to change course? That's what's going to have to come at the end. Between now and the fall, candidates will have to lay out their vision. They will have to contract themselves with the Democrats in office. At the end of the day, they have got to make an argument that you get a choice, you can either tell President Obama you think he has been doing a great job or you can send him a message and tell him to change.

TANTAROS: All right, encouraging.

Karl, thanks.

ROVE: Have a great Fourth.

TANTAROS: You, too. Happy Fourth.