Updated

This is a rush transcript from "Your World," June 7, 2013. This copy may not be in its final form and may be updated.

NEIL CAVUTO, HOST: Now former GOP presidential candidate Mitt Romney, no doubt watching all of this very, very closely.

He joins me from Park City, Utah, where he's been hosting a rather unusual summit -- among the participants at this event, Chris Christie, the governor of New Jersey, Rand Paul, Paul Ryan, Meg Whitman, of course of Hewlett-Packard fame, ran for California governor, David Axelrod, the man who masterminded the campaign that, well, I guess kept Mr. Romney a former governor and not a president.

Governor, good to have you. Thanks for coming.

MITT ROMNEY, R-FORMER PRESIDENTIAL CANDIDATE: Thanks, Neil. Good to be with you today.

CAVUTO: By the way, that cast of characters reads like the barroom scene at "Star Wars." How did you get them all...

(LAUGHTER)

CAVUTO: How did you get them all at one place at one time?

ROMNEY: Well, I wanted to bring together people actually from both sides of the aisle and people within each party that had different views about the priorities that they think the nation ought to be pursuing, hear from them, so that we can, as a group, listen to them and say, what are the highest priorities, and then ask ourselves, are we actually addressing the highest priorities or not?

And the answer turned out to be, not a surprise, the priorities Republicans were concerned about, the priorities Democrats were concerned about are not being addressed. And, instead, Washington is being pulled in a lot of different directions, but not doing what has to be done to get Americans to work in good jobs with better pay and to secure a brighter future for our kids.

CAVUTO: How did you get David Axelrod to appear?

ROMNEY: Well, I asked David Axelrod, the governor of Colorado, John Hickenlooper, a Democrat as well, the mayor of Los Angeles, Democrat, mayor of Stockton, California, Democrat, asked them to come.

And they were gracious and willing to come and express their views, and they were warmly received, even though we disagreed pretty vehemently on policy.

(LAUGHTER)

ROMNEY: The funny thing is, we're all Americans. We all care about the country. We get along on a personal basis. I think it's helpful to have the kind of dialogue that looks for common ground and looks for common vision for the future.

CAVUTO: Did David Axelrod or you or any of the others there mention all of these percolating scandals, Governor? Because now it appears out of control.

ROMNEY: Well, we didn't spend a lot of time talking about what Washington is feeling right now, all of these scandals.

And there are some that are very, very disconcerting and where we know there's been wrongdoing. The IRS scandal is certainly one of those. There are other issues that have been brought to the fore, such as the NSA monitoring of various data exchanges and so forth, e-mails, phone calls and so forth, but -- that raise questions, but are not necessarily wrongdoing.

And we have to separate those and look into them. But, clearly, the president's ability to manage the government of the United States and the trust that Americans have for our government has been very seriously eroded by the kinds of revelations that have come forward about the IRS.

CAVUTO: Now, the president, you might have heard today, sir, had said that these privacy tradeoffs are relatively modest -- those are his words - - this is not Big Brother.

What do you think of that?

ROMNEY: Well, Congress will take a look. The investigating committees will see what's been done. The president is saying, look, no one is reading e-mails, no one is listening in on phone calls. If that's the case, why, then -- then he's going to be pretty much given a pass.

On the other hand, if that's not the case, people will say, look, that's a violation of constitutional principles. But we don't know exactly where that is.

But, look, I have to tell you this. I'm not going to jump on the bandwagon -- bandwagon of attacking the president on everything that is going on in Washington. He has a responsibility to care for the safety and security of this country and our people.

And the practice of gathering information, looking for trends, finding things that might suggest someone is going attack this country, is an important part of that presidential role. At the same time, he has the responsibility to abide by the Constitution. And I hope he has been doing that.

This is an area that will be looked into, like many others. But areas we do know that there have been major problems of bad decisions being made relate to the IRS and, of course, to Benghazi. And, frankly, I think the subpoena of media and reporter records and phone calls, I think that is an area that is deserving of some real concern.

CAVUTO: All right, a lot of these tapping data from a lot of Internet companies and those like Google and Facebook and Apple, to say nothing of asking Verizon -- actually ordering Verizon -- to hand over phone records, came after mid-April, April 16, I believe, Governor. And that was soon after the Boston terror attacks.

If you were president at the time of those Boston attacks, would you have advocated trying to get your hands on this type of information?

ROMNEY: Well, I really can't respond to that without having all the information about what precisely was sought and what the purpose of it was and how it could be used, and also looking at what the Constitution requirements are.

But, in a modern age, where people are planning attacks using the communications vehicles that we have, including the Internet, to look for communication that suggests impending attacks or that is taking credit for things that have happened in the past, so we can find the bad guys before they attack us again, that is certainly a responsibility which a president and a government -- and a governor -- or a government, rather -- has to assume.

So, I'm not going just attack the president on this. I know a lot of people in my party and other parties are going to be unhappy that I'm not willing to attack him on this, but I'm going say, look, the safety of this country comes first. The Constitution has to be followed. I don't know that it wasn't followed in this case. And before I get real hot under the collar about it, I'm going to want to learn more about what was done and how it was used.

CAVUTO: All right, whether this does represent invasions of privacy, Governor, Dianne Feinstein had said it doesn't really bother her one bit, using her words. Harry Reid has said, calm down, when the media were jumping about whether this was overreach.

Do you think their response is justified, that this wasn't a big deal; everyone just take a chill pill?

ROMNEY: Well, I think, when people hear that their personal communications are potentially being watched by government, they're concerned, and particularly when they have just come on the heels of the revelations that the IRS has been using people's most private information and used it for political purposes.

And there's a loss of trust in our government. There's been a breach of a really significant magnitude. And people have a right to be very, very concerned. And so I don't think you just dismiss this. At the same time, you don't claim that someone is guilty or that there's been a breach of the constitutional principles until you have actually seen what has been done and how it's been done.

And that's something which that investigation will occur. Let's not get the -- if you will, the prosecution ahead of the facts. Let's get the facts and see if, in fact, there's been some kind of inappropriate gathering of information.

CAVUTO: All right, because I mentioned Harry Reid, sir, and I'm just reminded of the time when he stepped on the floor of the Senate to say that he was very well aware from a well-placed source that you had not paid taxes in 10 years.

Now, it was later proven that wasn't true at all, but it did raise the possibility that your tax return information or part of it was leaked to him. What do you think of that?

ROMNEY: Well, if he got it leaked from the IRS, he obviously got the wrong information, because, as was pointed out when we released our returns and actually laid out also what I had paid in taxes over a decade, I had paid millions and millions of dollars in taxes.

So -- and there was never a year where I didn't pay a lot of money in taxes. So he was just wrong. I don't know what his source of data was, but it was a...

CAVUTO: But, obviously, others on your campaign, sir, were leaked.

I mean, we talked to Frank VanderSloot, of course, a major donor to your campaign. He was subjected to no fewer than three agency audits. He was vindicated on each and all, but he had to pay $80,000, $85,000, I think he was telling me when he was here, in various attorney and accountant fees just to protect his good name.

ROMNEY: Well, this is a very dangerous and troubling development, the idea that an agency that has our most personal information -- I mean, they know how much money we make, what we make it in, what kind of assets we have, where all of our properties might be. They know everything about us financially. They have all that information, and they have used that information to pursue a political agenda.

That should be a very frightening thing to people on both sides of the aisle. I know it is. It is a negligent management that has allowed that to occur. It may be worse than negligence. We don't know who all knew about it and who directed it. But it is a violation that the American people will not soon forget. It has to end. And, in my view, we have to put in place a special prosecutor to understand who knew what and how that information was used. And we also need to put in place a structure which says that, in the future, these audits and these -- these inquiries by the IRS or other agencies that carry out various kinds of audits, these things have to be viewed on a nonpartisan, nonpolitical basis, and assure that these institutions aren't pursuing a personal agenda of one kind of or another.

CAVUTO: Do you think that this goes beyond some rogue agents? I mean, it's 88 now, by last count, who were involved in targeting these conservative groups and individuals. Do you think it gets bigger than that?

ROMNEY: Well, you know, if you don't have any process in place that is overseeing what agents are doing, you're opening the door for people to misuse data for their own personal political vendettas or for other vendettas.

And this is what people across this country are afraid of, big government becoming so large and so intrusive that no one can stop it from interfering in our lives and punishing people, without being required to go out into the public and demand that information on a legal basis.

And, so, you know, how many people knew about this and how far up the ladder it goes, we don't know. That, I think we have to find out. That's why I think we need to have a special counsel, special prosecutor appointed to look at this to find out just who knew. And if it goes...

CAVUTO: But do we know now, Governor -- I'm sorry, sir, but do we know now, after the fact...

ROMNEY: Yes. Go ahead.

CAVUTO: ... that it was stymieing at least conservative groups' efforts and those of groups that were arguing on your behalf, some even said even affecting turnout as a result of these groups being stymied. Do you agree with that?

ROMNEY: It's infuriating.

Look, I got to tell you, it's not fun to lose an election, but to hear that the government of the United States, and particularly an agency that has all the information about the American people, was using that power to help defeat me in one way or another, is really quite upsetting, as you can imagine.

CAVUTO: Do you think it tilted the election?

ROMNEY: And it's...

CAVUTO: Do you think it tilted the election?

ROMNEY: Oh, I don't think there's evidence of enough wrongdoing at this stage to suggest a different outcome would have occurred, but I do believe it's...

CAVUTO: No, I'm sorry, sir. I wasn't being clear.

If these groups were not targeted to the degree they were, did it affect their ability to sort of energize the base, or were they so preoccupied -- that was the argument that many conservatives have since argued -- just defending themselves and getting this -- you know, the IRS off their back, that -- that it -- it did affect their ability to get voters out to the polls?

ROMNEY: Oh, there's no question but that, if some organization was being run through the mill, that they were less able to do the job they put themselves in place to carry out, and therefore it had a chilling effect on their capacity to turn out voters or to pursue the -- the agenda that they might have. Of course, that was the result.

CAVUTO: Right.

ROMNEY: But I can tell you that I don't believe it was such a widespread and effective program on the part of the IRS or other agencies that it would have resulted in a different outcome in the election.

But that is not what is at stake here. What is at stake is, do the American people believe that the information they have is not being misused by the government and that the government is not trampling on our rights as citizens of this country and pursuing an agenda, a political agenda, at the expense of freedom in America?

CAVUTO: Among the attendees at your event today, New Jersey Governor Chris Christie.

Now, I know you have addressed the fact that bygones are bygones, and you certainly took no offense to what many in the media described as the governor's bromance with President Obama right after Hurricane Sandy. I know that is your view, but were any in your family -- because I certainly talked to many in your campaign at the time, Governor -- angry at Chris Christie?

ROMNEY: You know, I can't speak for all the members of my family. I have got 21 grandkids, not to mention my 10 kids and in-laws.

(LAUGHTER)

CAVUTO: I'm talking the old -- I'm talking the older ones. I'm talking maybe your wife or your sons.

ROMNEY: Yes. Yes.

Well, I take responsibility for myself. And I have been a governor. And I know what happens when there's been a natural disaster. And that is, you look for help from the federal government. You want the president and you want other agencies of the federal government to step in and provide help. And Governor Christie did what he thought was best for the people of his state, and I don't blame him for that at all.

He was helpful to me in my campaign in every way possible. I consider him a good friend. He has a great future. He has been a terrific governor. He's doing a good job in New Jersey. I have got no ill will.

CAVUTO: All right, so you don't agree, as some others in your campaign did at the time, that he got a little too chummy with Barack Obama at just the wrong time?

(LAUGHTER)

ROMNEY: Well, I can tell you the hurricane didn't come at the right time. That's not because of Chris Christie. That's because one of the advantages of incumbency is that, when there is an event like that, you get to see the president in a fatherly role and showing his sympathy for people who are harmed, who have been victims of a storm. And, obviously, that gives a little boost to the president's efforts. And that's just -- that's just the nature of how our system works and the nature of politics.

CAVUTO: Yes.

ROMNEY: And, look, I'm not going to -- I'm not going to go back and try and revisit that.

The right thing for me to do is to look forward and to say, how can the things I care about, which are helping put people back to work, getting America on track to a balanced budget, fixing our schools, improving our health care system and getting the costs down, how can I do those things from where I sit today?

CAVUTO: All right, well, you're a big man about that. But I will still be the tacky journalist and follow up with this line of questioning then.

(LAUGHTER)

CAVUTO: Do you think that, as others look back at that incident, that Hurricane Sandy, for whatever severity it had, clearly, it did arrest what had been the president's slide in the polls?

There are others who look back at your post-first-debate performance, when you knocked it out of the park, and they felt that you or your campaign had sort of sat on that and then just dropped the ball yourself, that, in other words, you didn't stay on offense and you just didn't keep going for the jugular.

What do you think?

ROMNEY: Well, we went after that first debate as hard and heavy as we could, and kept on promoting that as well as we thought we could.

I'm sure people come up with ideas of things we might have done differently. That's always the nature of Monday-morning quarterbacking. But when you're in the middle of the game fighting hard yard by yard, you're doing your very best.

And I believe that my team was a superb team. I spoke with a leading Democrat. I won't mention his name, but one of the top leaders of the Democratic Party. He said, look, I thought you had won one week out.

(LAUGHTER)

ROMNEY: This was a very close race. But, you know, events occurred. The unemployment rate dropped below eight percent for the first time just weeks before the election. That changed the national mood. The media celebrated that. Had it stayed above eight percent, why, that would have made a difference.

But, all told, you're not going to spend your time going back and saying, what kind of events could have happened differently? I made plenty of mistakes in the campaign. The president made mistakes in his campaign. No one runs a perfect campaign.

The net result of all of it was, he won, I lost. You get over that. You learn from the experience, and you move on, and you say, how do we fight for what's right?

CAVUTO: But, nevertheless, with all the scandals that have ensued in the last few weeks, Governor, between Benghazi, and the Justice Department, and overreach on reporters, and the IRS, and the large number of groups and individuals who are roped in, and now the Verizon files and the Internet files, and all of this stuff occurring as the Health and Human Services Department is hitting up companies to donate to push for health care, if all of this has come up last fall, different results?

ROMNEY: You know, there's no way I can make that calculation.

But I could tell you, it would have made a difference, of course. And that's, again, one of the advantages of incumbency...

CAVUTO: Yes. Yes.

(LAUGHTER)

ROMNEY: ... is you have some control about when things come out.

But, for me, I know a lot of people are really focused on these scandals. And they are. This IRS thing is really troubling, and Benghazi.

CAVUTO: OK.

ROMNEY: For me, the big issue on Benghazi is, why was there not a rescue mission?

CAVUTO: All right.

ROMNEY: And you might say, well, it wouldn't have succeeded. Why wasn't it tried? I mean, these are -- these are real, real issues and real concerns.

CAVUTO: All right.

ROMNEY: But, at the same time, one of my biggest concerns is, what is the president's agenda?

Content and Programming Copyright 2013 Fox News Network, LLC. ALL RIGHTS RESERVED. Copyright 2013 CQ-Roll Call, Inc. All materials herein are protected by United States copyright law and may not be reproduced, distributed, transmitted, displayed, published or broadcast without the prior written permission of CQ-Roll Call. You may not alter or remove any trademark, copyright or other notice from copies of the content.