The nominating process for the head of the National Counterterrorism Center, the hub for threat assessment within the U.S. government, is typically a routine affair.

But Matthew Olsen, President Obama's pick to lead the agency, faced a challenge to his nomination Tuesday amid accusations that he misled a member of Congress when he was in charge of the administration's Guantanamo task force to close the military detention camp in 2009.

Earlier this month, Rep. Frank Wolf, R-Va.,who represents the district where the center is located, wrote to the Senate Select Committee on Intelligence, objecting to the nomination. Wolf said he believes Olsen, general counsel for the National Security Agency, under White House pressure downgraded the threat level of some potentially dangerous detainees to allow for their expeditious release to unstable countries like Yemen, Somalia and Afghanistan as part of the administration's failed effort to shut the facility.

Wolf also wrote that Olsen misled him about the status of detainee transfers to the U.S. during an April 2009 meeting. At that meeting, Olsen chose only to discuss the process and not the decisions that he had already made, Wolf contended.

"I do not question Mr. Olsen's professional qualifications for this position, but from my observations of his recent leadership positions, I believe that he lacks the judgment to lead the NCTC," he wrote.

Against that backdrop, a Senate hearing on Tuesday turned contentious.

Republican Sen. Saxby Chambliss laid out the concerns, including accusations that the White House intended to transfer detainees to the U.S. in a secretive way.

"It was to be a stealth move," he said, contending that the detainees were going to show up in the U.S. before it was announced in a "swift, secretive operation that was designed by the administration to preempt any political outcry that could prevent the transfer."

In response, Olsen admitted that a decision on the transfer of two detainees had been made April 14, 2009, and that when he met with Wolf eight days later, he was not told.

Olsen claimed he did not have the authority to disclose the information.

"I do understand his frustration," Olsen said. "I did not mislead. I was not in a position to decide myself, at that time, that I was going to lay out exactly where that decision-making process was. We had met before that briefing and talked about what we were going to say and that we were going to be able to talk about in terms of the review process."

A vote on Olsen is likely to come this week but the accusations may not derail his nomination after Chambliss said he would weigh the strong recommendations for Olsen.

If confirmed, Olsen would replace Michael Leiter, the former Navy pilot who served in the director's job under Obama and President George W. Bush.

The NCTC was established in the wake of the Sept. 11, 2001, terrorist attacks and is responsible for analyzing and integrating information gathered across the intelligence community and then offering assessments to the president and senior officials.

Fox News' Catherine Herridge contributed to this report.