Republican Sen. Judd Gregg withdrew his nomination to be President Obama's commerce secretary on Thursday, citing "irresolvable conflicts" over issues like the economic stimulus package and the census. 

"It has become apparent during this process that this will not work for me as I have found that on issues such as the stimulus package and the census there are irresolvable conflicts for me," Gregg of New Hampshire said in a written statement. 

Obama called Gregg's withdrawal "something of a surprise," and the White House issued a statement saying it regrets Gregg's "change of heart." Other Cabinet nominees have dropped out of the process, but Gregg was the first to do so in protest.

White House Chief of Staff Rahm Emanuel defended the administration's record and acknowledged that the latest setback might lead some to question the administration's early competence.

"Some may call it amateur hour," he said. "Having been in two separate White Houses, I'd more than -- and within our third week given this set of accomplishments -- measure them up."

Republicans have been largely unified in their opposition to an economic stimulus bill that now totals $789 billion. They say it is full of government waste and won't create enough jobs to turn the economy around.

And in the past week, the GOP has stoked controversy over Obama's plan to share oversight of next year's census, which is carried out by the Census Bureau, part of the Commerce Department -- though Gregg said that was only a "slight" factor in his decision to withdraw.

Gregg, who was first elected to the Senate in 1999, says he doesn't intend to run again in 2010. He didn't give specifics on why he didn't plan to seek another term, saying to reporters Thursday, "Sometimes, there's other things to do in life."

White House press secretary Robert Gibbs said it was Gregg who first "reached out to the president" and offered to be commerce secretary. Gibbs said in a written statement that Gregg made it "very clear throughout the interviewing process" that he could "support, embrace and move forward" with Obama's agenda, despite "past disagreements about policies."

But it became clear after Gregg was nominated, Gibbs said, that Gregg could not support some of Obama's "key economic priorities."

"I said yes. That was my mistake," Gregg told reporters Thursday on Capitol Hill. He said he'd always been a strong fiscal conservative. "It really wasn't a good pick."

Gregg said he has followed his own path for 30 years but felt, after joining the Obama team, that he couldn't be himself. After agreeing to become the commerce chief, Gregg said he began to feel uncomfortable with his role and feared this would hinder his success. He did not go into specifics.  

A senior Obama administration official confirmed to FOX News that Gregg met with Obama at the White House Wednesday to notify him of his decision to withdraw.

Asked why they met, the official said, "It's kind of tacky to break up with your girlfriend over the phone."

New Mexico Gov. Bill Richardson withdrew his name for the same position of commerce secretary in January amid an ethics investigation in his state. And Tom Daschle withdrew his name for health and human services secretary over criticism about his failure to pay taxes on unreported income. 

Gregg said his and Obama's policy views are too different. 

"We are functioning from a different set of views on many critical items of policy," he said in his written statement. "Obviously the president requires a team that is fully supportive of all his initiatives." 

In referencing the stimulus, Gregg made clear his distaste for the package, echoing his fellow Republicans.. 

Only three Republican senators have supported the spending and tax-cut plan. They were the lone members of their party who pushed a compromise bill expected to cost $789 billion and be on the president's desk in a matter of days. Gregg didn't vote on the package in the Senate.

As for the Census Bureau, Republicans have accused the Obama administration of trying to politicize the once-in-a-decade event by having the bureau's director report directly to White House senior staff members, as well as the Commerce Department.

The outcome of the census has deep political implications, since congressional districts are drawn based on population. Many federal funds are distributed on the basis of population, as well. 

But Gregg said at the news conference that the census was only a "slight catalyzing issue. It was not a major issue."

Senate Republican leader Mitch McConnell of Kentucky said Gregg "made a principled decision to return and we're glad to have him. He is among the smartest, most effective legislators to serve in the Senate -- Democrat or Republican -- and a key adviser to me and to the Republican Conference. It's great to have him back." 

A day after Gregg's nomination was announced, The Associated Press reported that a former staff member was under criminal investigation for allegedly taking baseball and hockey tickets from a lobbyist in exchange for legislative favors while working for Gregg. 

The former staff member, Kevin Koonce, has been identified in court papers only as "Staffer F" in the sprawling corruption probe stemming from disgraced lobbyist Jack Abramoff. 

Gregg said at the time that he had been told he was neither a subject nor target of the investigation, and would cooperate fully.

The Associated Press contributed to this report.