The latest tough question facing U.N. Secretary General Kofi Annan (search) is:  Who inspects the inspector?

Annan spoke Thursday for the first time about the scandal surrounding Under-Secretary-General Dileep Nair (search), the head of the U.N. Office of Internal Oversight Services.

Nair, a senior official in the office that's supposed to expose wrongdoing or mistakes at the organization, is now himself accused of sexual misdeeds, abusing his power and corruption.

An anonymous letter sent to numerous offices in the United Nations (search) contained allegations Nair promoted women in exchange for sexual favors, accepted money for jobs and was biased against employees of non-Indian descent.

The allegations were so serious the staff union asked for an investigation and Annan asked Nair to respond to the charges.

"I saw the letter you are referring to... distributed quite widely...," Annan told reporters Thursday. "Mr. Nair indicated there was not much truth in what was contained in that document."

Later Thursday, Nair met the press but would only speak off-camera, telling reporters his office isn't in crisis but is certainly under pressure and he says he will fight to clear his name.

"I categorically deny all the allegations made in the staff resolution.  As for the poison pen letter, I'm not going to respond to those allegations," Nair said.

But he eventually responded about the letter, telling reporters it was false and that the husband of one of the women named in the letter called the whole thing a joke.

Annan has had his hands full trying to figure out what to do next even as he deals with the ongoing oil-for-food investigation.

Nair, a former banker and civil servant from Singapore, was picked by Annan in 2000 to run the U.N.'s anti-corruption office. He serves a five-year, non-renewable term.

Fox News' Rick Leventhal contributed to this report.