Updated

Call them the survivors.

While a stream of top Obama advisers have peeled away – for academia, for a return to private life – over the last four years, several of his closest allies in the administration have toughed it out.

Some of them, like Attorney General Eric Holder, survived seemingly against the odds, with droves of Republicans demanding his resignation by the end. Others, like Secretary of State Hillary Clinton, had a to-do list that was simply too long to abandon a couple years in.

If President Obama wins a second term, it's doubtful the entire cast will commit to another four years. But as Election Day, and then Inauguration Day, approaches, here's a look back at the figures who helped define Obama's entire -- and controversial -- term.

Attorney General Eric Holder

More than a hundred Republican lawmakers called on Holder to resign as details emerged about Operation Fast and Furious, a failed program in which some of the 2,000 guns put into the black market were linked to crimes along the Mexico border, including the fatal shooting of U.S. Border Patrol Agent Brian Terry.

Holder survived after a U.S. Inspector General investigation found no evidence he was aware of the operation or involved in a cover-up, and the president expressed "absolute confidence" in Holder at the time.

Holder in fact contemplated resigning amid depression brought on by his mother's death and criticism for arguing the Sept.11 terror suspects should be tried in civilian court,  according to the book "Kill or Capture: The War on Terror and the Soul of the Obama Presidency."

Health and Human Services Secretary Kathleen Sebelius

Health Secretary Kathleen Sebelius has largely been the face of Obama's controversial health care law, making her the target of conservatives and one of Obama's most loyal Cabinet members.

In February, during an official administration event in North Carolina, Sebelius told attendees to vote for Obama to ensure he "continues to be president for another four years."

Sebelius was found to be in violation of the Hatch Act, which forbids public officials from campaigning in an official capacity, but will probably not be punished.

The White House has said her remarks were extemporaneous, the Health and Human Services Department has since reclassified the event to meet the correct standard, the Treasury Department has been reimbursed and Sebelius has met with ethics experts "to ensure this never happens again."

Vice President Joe Biden

Biden has a long list of gaffes -- obvious and at other times perhaps calculated.

The biggest might have been when Biden, during a TV interview in May, appeared to jump the gun on the president's evolving personal views on same-sex marriage.

Biden said he was "absolutely comfortable" with gay marriage. The White House downplayed the remarks. But sure enough, the president three days later said he needed to personally affirm that he thinks "same-sex couples should be able to get married."

More recently, Biden at campaign rally in Virginia with many black supporters in attendance, said a Romney-Ryan administration would "unchain Wall Street."

"They're going to put y'all back in chains," he added. And who could forget Biden's "Big F---ing deal" comment during the signing of the health care law.

Still, Obama generally seemed to shrug it off, sticking by the avuncular veep through his second presidential run.

Treasury Secretary Tim Geithner

Perhaps nobody in the Obama administration endured a more sustained challenge than Geithner, the perceived brains and triggerman behind the Obama administration's spending of $350 billion under the Trouble Asset Relief Program during the early days of Obama's presidency, as the economy nearly collapsed. The other half was spent during the Bush administration.

Geithner appeared almost countless times before Congress and on national TV to explain and defend administration's strategies, which also included efforts to restructure regulation for the U.S. financial system as well as saving the country's mortgage and auto industries.

U.S. Ambassador to the United Nations Susan Rice

Rice is the most recent administration official to face calls for resignation.

Rice went on five Sunday talk shows after the Sept. 11 attacks on U.S. outposts in Benghazi, Libya that killed four Americans, including an ambassador, to say the attacks were "spontaneous," not pre-planned.

The administration has since said the strikes were terror attacks, which eventually led to Rep. Peter King, R-N.Y., chairman of the House Homeland Security Committee, to say Rice should resign for intentionally misleading the American people.

Secretary of State Hillary Clinton

If anybody among the president's inner circle has emerged as a large success, that would be Clinton.

Despite some recent criticism about her handling of the Libya crisis, Clinton has largely been lauded for being a tactful envoy of U.S. foreign policy and courageous for saying "I take full responsibility" about the Libya crisis.

The 65-year-old Clinton has insisted she will step down should Obama win a second term, but appeared to leave the door open during a recent Wall Street Journal interview which she said, "a lot of people have talked to me about staying."

"Kicked off the Island"

Others in the Obama economic team didn't make it, including Obama's budget director Peter Orszag, National Economic Council Director Larry Summers and Council of Economic Advisers Chairman Christina Romer, whose widely-known disagreements reportedly contributed to their departures.

Orszag in July 2010 cited the need for more time in his personal life but reportedly left in part because he couldn't get Summers and others to focus more on less federal spending and cutting the deficit.

Romer left two months later to return to University of California, Berkley. And Summers returned to Harvard after the 2010 elections.