The U.S. watchdog on Iraq reconstruction is disputing a report quoting him suggesting that $6.6 billion in Iraqi oil money entrusted to U.S. hands may have been stolen.

The charge, if true, would make the theft of funds the largest in U.S. history, and has already angered Iraqis reportedly debating whether to sue over the missing funds.

The Pentagon refuses to endorse the charge that the disappearing dollars were stolen -- either by greedy U.S. contractors or others involved in its movement from U.S. holdings to Iraq. And now, Special Inspector General for Iraq Reconstruction Stuart Bowen said he never said that $6.6 billion in missing money was swiped.

"What we concluded in our previous audits is that it's been virtually impossible to account for what happened to that money," Bowen told Fox News in a telephone interview Monday, adding that criminal cases have led to the convictions of people who have stolen money from a special fund set up by the U.N. Security Council.

But Bowen said he did not mean to imply anything more when he answered a Los Angeles Times reporter's question about whether it would be serious if billions of dollars was stolen from the Development Fund for Iraq.

"I said, yes, it would be a very significant serious crime," he said. "So yes, the reporter was correct that some of it, and perhaps a lot of it, has been stolen. But we don't have a factual basis to reach that conclusion. What we said over and over again is that the lack of controls created vulnerabilities to fraud, waste and abuse."

Bowen did not say where the Los Angeles Times derived the $6.6 billion figure in question, a figure that he didn't use.

After the U.S.-led invasion of Iraq in 2003, the Bush administration airlifted to Baghdad a total of $12 billion that was carried by tractor-trailer trucks from the Federal Reserve currency repository in New Jersey to Andrews Air Force Base in Maryland for reconstruction of the country.

Bowen said most of the money was deposited in Iraq's central bank for distribution to Iraqi ministries and contractors, but he doesn't know how much.

"Iraq could have it all," he said. "They probably do have most of it."

The Pentagon has been unable to properly account for the $2.8 billion that it controlled under the Commander's Emergency Response Program, which allowed military commanders to spend money for  reconstruction projects. Bowen said the Pentagon's comptroller asked him last year to help audit the flow of money.

"More important, we're trying to work with the Iraq government to find out what happened to the rest of the money," Bowen said.

Bowen emphasized that the missing money is not U.S. taxpayer funds. The revenues in the DFI come from among other sources -- Iraq's oil and gas exports, as well as frozen Iraqi assets and surplus funds from the now-defunct, Saddam Hussein-era oil-for-food program.

But previous audits have shown the Defense Department didn't do a good job of tracking the money and Iraqi officials told The Los Angeles Times that under a 2004 legal agreement, Washington is responsible for the missing funds.

Iraq's chief auditor and president of the Iraqi Board of Supreme Audit has warned Washington that Baghdad will sue if necessary to recoup the money.

"Clearly, Iraq has an interest in looking after its assets and protecting them," the newspaper quoted Iraqi Ambassador to the United States Samir Sumaidaie saying.

Bowen said the U.S. needs Iraq's assistance to obtain the bank data regarding the disbursal of the funds so that his office can close the books on jurisdiction over DFI by the end of the summer since it's been years since the U.S. has had control over it.

"We're going to account for it as best we can," he said.

Fox News' Justin Fishel contributed to this report.