Former National Security Adviser Sandy Berger (search) announced Tuesday that he would no longer serve as an informal adviser to John Kerry (search) in his race for the White House, after reports revealed Berger is the subject of a criminal investigation over the disappearance of terrorism documents.

"Mr. Berger does not want any issue surrounding the 9/11 commission to be used for partisan purposes. With that in mind he has decided to step aside as an informal adviser to the Kerry campaign until this matter is resolved," said Lanny Breuer, Berger's attorney.

Kerry issued a statement after Berger stepped down.

"Sandy Berger is my friend, and he has tirelessly served his nation with honor and distinction. I respect his decision to step aside as an adviser to this campaign until this matter is resolved."

People close to the Kerry campaign confirmed to FOX News that it was never notified by Berger that he was under investigation by the FBI.

"People are pretty pissed," one source said, then said: "Well, I wouldn't say 'pissed.' I'd say 'shocked.'" The source added: "It's weird" that Berger was not more forthcoming with the campaign prior to the leak.

Reports about Berger allegedly taking highly classified terrorism documents that should have been turned over to the independent commission probing the Sept. 11, 2001, terror attacks had already started to impact the Massachusetts senator's presidential pursuit.

Although Capitol Hill lawmakers said they didn't want to make a judgment call on Berger's fate until all the facts were known, several told FOX News that the situation didn't look good for Berger, or even for Kerry.

"I think the problem here is Sandy Berger is a close adviser of Sen. John Kerry and there's a certain discipline you have to have in this business … We're all political animals ... we're all trying to get a leg up on the other guy" in an election, Rep. Duncan Hunter, R-Calif., told FOX News. "Obviously, he's violated that discipline."

"There's an ethic here — that is of strict discipline, of not letting the fact you're working on a political campaign start to color your actions when it comes to national security," added Hunter, who chairs the House Armed Services Committee.

Sen. Joe Lieberman, D-Conn., called the news "surprising" and said that "unless we learn otherwise, I have to assume that what Sandy said was right — that any removal of documents was inadvertent. But it is serious."

Docs in the Pants

Sen. Saxby Chambliss, R-Ga., said the Kerry campaign "needs to come forward" about any information that could have been passed on to them from Berger.

"I don't know what happened to these documents after they were put in Mr. Berger's pants, but it's been reported in the press that these documents related to homeland security and our airports and seaports and it's very interesting to note that those are two areas where Sen. Kerry has been critical of the Homeland Security Department," Chambliss said. "I would hope, No. 1, that the Kerry administration would disavow any connections with Berger, that they would come forward with any documents ... and that we can bring this matter to a close very quickly."

Democratic strategist Richard Goodstein, however, said Kerry should "absolutely not" drop Berger from his team.

"The documents that Sandy supposedly took were copies. There are copies elsewhere throughout the [National] Archives and elsewhere in Washington, so it's not like he was trying to cover something up," Goodstein said.

But Ben Ginsburg, national counsel for the Bush-Cheney '04 campaign (search), said the big question is whether Kerry got his hands on any of the information in question.

"That's an essential question that needs to be answered from the Kerry campaign — did they benefit from documents that they should not have had?"

Sen. Trent Lott, R-Miss., said more information is needed before a judgment is made on Berger's actions, but "obviously, the timing of it is not good" for Kerry.

"From now on, until the election, everything like this will have a spotlight put on it, examined very carefully," Lott said.

Sen. Kit Bond, R-Mo., told FOX News that it's not yet known if there was any damage to national security as a result of Berger's alleged actions, but said the case shouldn't yet be put in a bright spotlight.

"I really don't see that this is going to be a major issue ... while it indicates some sloppiness, I don't think this is a big deal that would be  — should be  — the focus of national attention," Bond said.

"Unfortunately, there are a lot more serious [security] breaches every day … I wish we could get a better handle on the leaks that come from the members of Congress themselves because they have been harmful and they happen every day."

Kerry Campaign Response

The Kerry campaign maintains that most of the Clinton-era information conveyed to the Kerry campaign was delivered by former U.S. ambassador to the United Nations Richard Holbrooke and former Deputy Secretary of State Jamie Rubin, a paid Kerry staffer.

Berger has been an informal, unpaid adviser; campaign officials said his participation in the campaign has been limited to conference calls and policy discussions inside the campaign and conversations with the press.

Within the Kerry campaign, speculation has long been made that Berger has kept himself out of Kerry's inner circle to remain available for a top posting in a potential Kerry administration. There was even talk that Berger was on Kerry's short list for director of central intelligence.

Privately, insiders say Berger had his share of critics in the Kerry campaign, largely as a function of the competition for the candidate's ear. Inside sources claim that when Berger didn't get a prominent paid official position, he backed away from the Kerry campaign.

Officials with the Bush-Cheney campaign point out that Berger gave a surprise background briefing to reporters on Feb. 27 on behalf of the Kerry campaign, in which he outlined airline security issues apparently drawn from the now-missing classified memos Berger is accused of removing from the National Archives (search).

But Kerry's spokesman Phil Singer released a statement to FOX News, questioning the timing of the report.

"This appears to be a partisan attempt to divert attention away from the 9/11 commission report. Instead of using the report's recommendations to learn how we can improve our homeland security, Republicans are playing politics with a criminal investigation. That's wrong, and in November voters will have a choice on the ballot between a candidate they can trust and a president that continues to mislead the nation," he said.

No matter what the outcome of the investigation, other lawmakers and observers said it's no coincidence the information's coming out now at the height of an election year.

"There's more politics around this town than you can cut with a knife — it's unbelievable," Sen. Bill Nelson, D-Fla., told FOX News. "I've never seen Washington so partisanly charged. Frankly, I'm getting sick and tired of it and I suspect the American people are getting tired of it as well."

Sen. Gordon Smith, R-Ore., said there's a "curious connection" between the documents' removal and a Kerry press conference attacking Bush on port security issues.

"I will admit to you, when you stuff classified legal documents into your briefs, it looks sloppy. But it may be worse than that," Smith added.

"In fairness to the president of the United States, it's important that this be followed and pursued so the American people can know that the predicate of many of the charges made against George W. Bush are based on lies and deception."

"Someone leaked it obviously to do damage to Mr. Berger — that is unfortunate," said Senate Minority Leader Tom Daschle, D-S.D. "Sandy Berger has done a remarkable job for his country and certainly deserves the benefits of doubt until the investigation is complete."

Others say the "shocking" actions of Berger — if true — don't look that "sloppy" or "inadvertent."

"I think it's gravely, gravely serious what he did ... if he did it," said House Majority Leader Tom DeLay, R-Texas. "It could be a national security crisis ... we don't know what was in those documents or what other documents may have been compromised and the questions that this scandal raises are absolutely shocking."

FOX News' Carl Cameron, James Rosen and Brian Wilson contributed to this report.