Where does the buck stop in the United Nations Oil-for-Food (search) scandal?

For some, the answer is that it stops with U.N. Secretary-General Kofi Annan (search). But for others, Annan is seen as a convenient scapegoat for lawmakers who take a dim view of the United Nations as an institution.

In the first report from the investigative panel headed by Paul Volcker (search) released earlier this month, the committee confirmed the now-widespread reports that Oil-for-Food was grossly mismanaged and that the United Nations official running it had a serious conflict of interest.

But Volcker also tabled more issues than he addressed and drew no conclusions about who was ultimately responsible for letting Saddam Hussein (search) turn the humanitarian aid program into his personal multi-billion dollar cash machine.

“This is not the whole story by a long shot,” Volcker said. The full Volcker report on corruption within the United Nations will not be finished until June.

According to a FOX News/Opinion Dynamics poll, 58 percent of the public is aware of the scandal and more than half of them also believe that Annan must go. Among the secretary-general’s critics are more than 60 U.S. lawmakers — mostly Republicans, but also some Democrats — who've called for Annan's resignation.

“I think any other organization in the world whose CEO was in charge when this massive fraud took place would step down,” said Sen. Norm Coleman (search), R-Minn., the chairman of one of five congressional committees investigating Oil-for-Food. By Coleman’s estimate, Saddam stole $21 billion on Annan’s watch.

But Annan's supporters say he's not going anywhere.

“He is committed to staying the course,” said John Ruggie (search), a former U.N. assistant secretary-general of strategic planning and a close Annan advisor. Annan would not appear on FOX to discuss Oil-for-Food or his role in the program.

Ruggie said it was important to remember Annan’s accomplishments. “Don't forget he had a phenomenally successful first term,” Ruggie said, noting that Annan “was awarded the Nobel Peace Prize in 2001 at the end of his first term for bringing new life to the United Nations.”

And Ruggie said he faulted Coleman for discounting the fact that Oil-for-Food provided Iraq with $30 billion worth of food, medicine and other goods over the years. Ruggie also accused Coleman of grossly exaggerating the amount of Saddam’s take.

“He claims, in his public statements, that, ‘Saddam Hussein skimmed $21 billion, from the Oil-for-Food program.’ That's just not true. It is mathematically impossible,” Ruggie said.

Coleman offered his own challenge, disputing the other point-of-view, that Saddam skimmed only about $7 billion or $8 billion from the program.

“If I sat in front of you and said there was a $7 billion, $6.7 billion theft under Kofi’s watch, that should, that would be enough [to demand his resignation]. But somehow that’s discounted,” Coleman said.

Even though the estimates differ, the United Nation’s own report concedes Saddam's theft was into the billions. The Oil-for-Food scandal is not just about how much money Saddam stole from suffering Iraqis but how he used that money to buy power, weapons and influence inside the United Nations.

“The scandal is horrible in terms of public perception,” Ruggie said. “All the disagreements in the past, all the political hullabaloos, throughout the Cold War, all the rest of it — there is nothing quite like this."

In December, as the calls for Annan to resign began to grow, some of the secretary-general’s closest advisors quietly called a meeting with him at the home of Richard Holbrooke (search), a former U.S. ambassador to the United Nations during the Clinton administration.

“The purpose of that meeting was to say, look, here are five Democrats, all of, all of whom love you, and all of whom have high regard for the institution,” Ruggie said, adding that the message they gave to Annan was “you need to take this seriously and, and things need to ... happen.”

Annan didn’t speak during the meeting but Ruggie said he listened carefully to what was being said about him and about the United Nations. The consensus of those at the meeting was that Annan would survive the scandal, Ruggie said.

Coleman said he can’t believe that people who say they love and value the United Nations are closing ranks around Annan.

“It is somewhat perplexing that there is this massive defense when in fact folks know the fraud took place ... took place under Kofi's watch,” Coleman said. “If he didn't have the strength to clean it up, he needs to go.”

Coleman previously said that despite Oil-for-Food, he believed the United Nations was a needed institution. He’s not so sure anymore.

“I have greater concerns today about the ability of … the U.N. to … to clean up its own mess, to effectively deal with some of the critical things it needs to deal with,” Coleman said. “It needs a boost of credibility that it's sorely lacking today.”

On that point, Annan’s supporters have come to agree.

“Whatever the area of responsibility is for the secretariat in the Oil-for-Food program, that has to be fixed,” Ruggie said. “Nothing like it can ever happen again. Because if it does, the U.N. won't survive.”

FOX News' David Asman Brian Gaffney, George Russell, Grace Cutler and Betsy Petrick contributed to this report.