Updated

President Obama releasing five Taliban detainees from the U.S. prison in Guantanamo Bay, Cuba, without notifying Congress has congressional Republicans arguing the president took the first step in his larger plan to side-step Capitol Hill to close the facility.

South Carolina GOP Sen. Lindsey Graham is among those most critical of Obama and is trying to thwart his effort.

Graham said this week that his goal is “to keep these prisoners in jail when appropriate to protect the United States” while Obama and fellow administration officials are “trying to clean out the jail.”

His remarks were followed within hours by fellow Republican Sen. Mark Kirk, Illinois, handing out a flier with mug shots of the five detainees that read: "Obama's Release Threatens Americans' Safety Around the World and is a Backdoor Effort to Close Guantanamo."

Earlier in the week, Oklahoma Sen. Jim Inhofe, the top Republican on the Senate Armed Services Committee, said: “The whole motivation here is the president wants to shut that [prison] down. … It fits right in with what the president has been trying to do on Gitmo since before he was president.”

Graham’s effort to stop the closure in part focuses on using the fiscal 2015 National Defense Authorization Act to achieve his goal. He recently got an amendment attached to the legislation that would require a congressional vote -- not just a 60-day notice -- on any administration plan to close the prison by sending detainees to the United States.

However, such a plan will likely have no immediate impact considering the Senate is not expected to vote on it until after the November elections.

Obama has repeatedly stated his intentions to close the facility, dating back to his 2008 White House campaign, and restated his case as recently as last month.

“What makes us exceptional is not our ability to flout international norms and the rule of law; it is our willingness to affirm them through our actions,” he said during a May 28 foreign policy speech at the U.S. Military Academy-West Point, N.Y. “And that’s why I will continue to push to close Gitmo -- because American values and legal traditions do not permit the indefinite detention of people beyond our borders.”

However, the prisoner swap for Taliban-captured Army Sgt. Bowe Bergdahl could make closing Guantanamo harder for Obama, because some lawmakers might now push for tougher restrictions.

"Congress has passed a law and [Obama] has signed a law that he will not close Guantanamo,” California GOP Rep. Buck McKeon, chairman of the House Armed Services Committee, told CNN on Sunday. “Whatever his desires are, that does not conform to the law and he is the chief administrator of the law of the nation and should uphold his constitutional obligation to follow the law."

National Security Adviser Susan Rice recently suggested an increased focus on Obama accomplishing some of his major political goals as he nears the end of his second term -- which would likely include closing Guantanamo.

"It is a very important part of my job, particularly at this stage in the president's tenure, to not just be fighting fires and the headlines, which to some extent is unavoidable, but to make sure that the things he cares about -- his legacy interests, issues, accomplishments -- are nurtured and carried into the end zone," Rice recently told The Associated Press. "We are trying to put points on the board."

Obama has argued that the prison, built on foreign soil during the Bush administration with congressional approval in the aftermath of 9-11 terror attacks, is expensive and has outlived its purpose because the U.S. cannot continue to hold terror suspects without a trial, a more pressing situation as the last of American troops leave Afghanistan in the war on terror.

When Congress approved the fiscal 2014 National Defense Authorization Act, which is the bill Graham is trying to amend, members relaxed some restrictions on detainee transfers. However, the legislation still includes a 30-notification to Congress, which Obama sidestepped last week in releasing the Taliban detainees.

Officials stated publicly after the swap that they had previously told Congress about such a potential deal and that they had to act swiftly because Bergdahl’s health was in imminent danger. And congressional staffers said privately late last week that administration officials told them in a close-door meeting they were compelled to secrecy because Taliban negotiators vowed to kill Bergdahl if the deal was made public.

Obama has also hinted that he might use his powers as commander in chief to take action on Guantanamo.

Upon signing the defense bill in December 2013, the president called it “a positive step” toward lifting restricts on prisoner transferring he considers “unwarranted and burdensome.” However, he also suggested some of the remaining limitations still violate “constitutional separation of powers principles.”