Attorney General Eric Holder’s resignation announcement has set off a flurry of speculation over who might replace him – and, perhaps more importantly, who could survive the bruising confirmation process on Capitol Hill.

“I have a hard time coming up with anyone the president could trust who would have an easy road,” said Holder critic Hans von Spakovsky, who published the book, “Obama’s Enforcer: Eric Holder’s Justice Department,” in 2013.

The president did not offer any hints when he formally announced Holder’s decision to step down after six years on the job. He said Holder would stay on until a successor is named.

Several prominent names could be in the mix, though, and already are starting to get some buzz. Among them are Massachusetts Gov. Deval Patrick, Solicitor General Donald Verrilli and California Attorney General Kamala Harris.

Within hours, a few of the possible prospects were busy knocking down the speculation.

Harris released a written statement saying: "I am honored to even be mentioned, but intend to continue my work for the people of California as Attorney General. I am focused on key public safety issues including transnational gangs, truancy and recidivism."

Patrick, 58, a former Justice Department official in the Clinton administration, has been floated as a potential successor since as far back as 2012. He is serving his second and last term as governor.

Tom Whalen, presidential historian and associate professor of social science at Boston University, suggested Patrick could meet an important test.

“Attorney generals tend to be consigliere, people who defend the president’s interests” first, he said. “That is why the president usually picks someone who is a close friend who is beyond doubt loyal to him or her. Deval Patrick fits that to a T.”

Patrick and Obama are close friends and political allies, having campaigned for one another over the years. Earlier this year, Obama said his friend would make “a good president or vice president,” stoking speculation that Patrick had his sights more or less on the White House. But on Thursday, the media seized upon news that the governor had plans to arrive in Washington that day.

Patrick, though, also quelled some of the speculation when he spoke to reporters at a morning event.

“First of all I want to say of Eric Holder that he has distinguished himself and the role of the attorney general, as attorney general, and I thank him for his service to the administration and his service to the nation,” Patrick said, according to the full statement released by his press secretary. “That’s an enormously important job but it’s not one for me right now.”

Jesse Rhodes, associate political science professor at the University of Massachusetts at Amherst, still said Patrick’s commitment to enforcing civil rights law was “clear” and that in this way he was “substantively quite similar” to Holder.

Von Spakovsky, a judicial and legal expert at the Heritage Foundation, predicted that the hearings will be tough on any nominee, considering the unfinished business over the Fast and Furious scandal and the ongoing IRS scandal. He said Patrick's own record heading the Justice Department’s Civil Rights Division, which von Spakovsky called spotty and “extreme,” would be under the microscope if he were nominated.

Others under consideration could include: Preet Bharara, U.S Attorney in Manhattan; Deputy Attorney General James Cole; and former White House Counsel Kathryn Ruemmler. Sen. Sheldon Whitehouse, a former Rhode Island attorney general, and other Democratic senators also have appeared in some reports as possible prospects.

Aside from Patrick, speculation on Thursday centered fairly heavily on Verrilli, 57, who is the administration's top representative to the Supreme Court.  He successfully defended the Affordable Care Act -- but other landmark cases including one on voting rights and the Hobby Lobby challenge over ObamaCare’s contraceptive coverage did not break in the administration’s favor. Critics say his record is mixed as a defender of the Obama administration.

Meanwhile, Harris, despite her statement on Thursday, is a staunch ally of Obama and had also been mentioned in the top tier of potential nominees. Having taken on the banking industry, Harris is popular among progressives and would be only the second female AG – after Janet Reno.

On the other side of the country, Bharara has emerged as one of the most colorful and ambitious officials in New York politics -- making Wall Street corruption, cybercrime and terrorism his signature issues as U.S. Attorney for New York’s Southern District. More recently, he has entered a battle with Democratic Gov. Andrew Cuomo over the governor’s decision to shut down a corruption panel. 

Whalen said the call to “top cop” would be a tough request to resist. On one hand, “you make enemies everywhere, not just on the opposite political aisle but in your own party because you prosecute people.

“But if the president calls and says I need you in this post, it would be difficult to turn down this office.”