Updated

This is a rush transcript from "Your World," June 4, 2014. This copy may not be in its final form and may be updated.

NEIL CAVUTO, HOST: Curses, you said it. We said there was a closed-door meeting. Right now, they curiously show open doors. We're on to you, though, because we know, once you do meet in half-an-hour, it's going to be behind closed doors. And people can't get back and forth, even though you are debating that prison swap, how it went down, why it went down, who was informed, who was it.

South Carolina Republican Senator Lindsey Graham says those doors ought to be open, just like they are right now.

But they won't be for long, Senator. What do you make of that?

SEN. LINDSEY GRAHAM, R-S.C.: Well, we will have to push them open later on. We're going to have a closed-door meeting, where we will learn almost nothing, as we usually do.

We will learn more from Fox News than we learn from this meeting. But we're going to air this out in public in a responsible way. They're not going to get away with this. We're going to have public hearings about this prisoner deal.

CAVUTO: Do you know -- or I have heard from so many of your colleagues, sir, who were not informed about this, even Dianne Feinstein saying that no one gave her a heads-up. Republican leaders have been saying much the same thing.

GRAHAM: Right.

CAVUTO: Now, who is the administration is talking about when it says they were informed?

GRAHAM: They didn't inform anybody until after the fact.

We have been talking about this issue for a couple of years. And every time they came to the Congress, we said, no, that's not a good idea to let these five killers go for Bergdahl. Gates said no. Panetta said no. Clapper originally said no. Feinstein said no. All of us said no.

This time around, they didn't ask us. They told us after the fact. And they used the excuse that Bergdahl was about to die, they didn't have time to notify us. So, I'm very curious to see if that withstands scrutiny.

CAVUTO: If they were to do this again, the White House were to do this again...

(LAUGHTER)

CAVUTO: ... I think you were quoting as saying, sir, that that's an impeachable offense, not running this by Congress, doing this on your own.

GRAHAM: Right.

CAVUTO: Well, he's done it. He's done that.

GRAHAM: Well...

(CROSSTALK)

CAVUTO: So is that impeachable now?

GRAHAM: Well, the question was, what if the president just let everybody go tomorrow?

We have got a lot of restrictions on transferring detainees to Yemen and other very dangerous places. I said if he just arbitrarily emptied the jail, there would be Republicans calling for impeachment. It would be a constitutional crisis.

I'm not saying the lack of notification is an impeachable offense. There's a legitimate argument by the commander in chief that this is a prisoner situation where he had to act. If he just unilaterally emptied the jail and let everybody go, you would have a constitutional crisis, because we have restrictions on that.

I would advise the president not to go down that road. I'm not saying what he did is an impeachable offense. I am saying it was a very bad deal for us. Americans are more likely to be kidnapped in the future because of this deal.

And as surely as night follows day, these five are going to go back to the battlefield and create havoc all over the world, particularly in Afghanistan.

CAVUTO: Let's say that he did just release the five most dangerous guys at Gitmo. That would, by definition, mean the ones who are remaining there are not as dangerous, and that it might compel the administration's argument, just shut down Gitmo because the worst of the worst is already gone.

GRAHAM: Yes.

CAVUTO: What do you think?

GRAHAM: Well, the worst of the worst are still there.

Anybody there now, Neil, is there because they are so bad. These five represent the military and political leadership of the Taliban. Khalid Sheikh Mohammed is still there. Anybody left in Gitmo has been held under the law of war.

And this absurd idea that the day we withdraw from Afghanistan is the end of hostilities and we have no legal authority to hold these people at Gitmo is just flat wrong. The hostilities we're engaged in is not against the Afghan government or people. It's against terrorist organizations and those who provide material support.

CAVUTO: All right.

GRAHAM: We can hold these guys as long as they're dangerous to us long after we lead Afghanistan, because the war is with Al Qaeda, their affiliates and people that help Al Qaeda, not with the Afghan people.

CAVUTO: Senator, thank you very much. Good luck at that meeting.

GRAHAM: Thank you.

Content and Programming Copyright 2014 Fox News Network, LLC. ALL RIGHTS RESERVED. Copyright 2014 CQ-Roll Call, Inc. All materials herein are protected by United States copyright law and may not be reproduced, distributed, transmitted, displayed, published or broadcast without the prior written permission of CQ-Roll Call. You may not alter or remove any trademark, copyright or other notice from copies of the content.