U.S. military officials are pushing back on an article that claimed a top Army commander in Afghanistan directed a specialized unit to manipulate visiting U.S. dignitaries, as Gen. David Petraeus orders an investigation into the allegations.

Both sides are starting to speak out after Rolling Stone reported that the command of Lt. Gen. William Caldwell, who is in charge of training Afghan troops, tried to tap members of the military's "information operations" unit to use their skills on visiting senators and congressmen, among others. The goal, according to the article, was to convince officials to provide more troops and money.

The accuser, Lt. Col. Michael Holmes, stood by his allegations in an interview with Fox News. He said the top brass ordered his unit to gather information in a way that was "over the line," and provided Fox News with documents detailing his concerns. He said he was "badgered" for months to get information on senators and other visitors.

"It was, 'What are these people wanting from us? What is it that we can tell them that will get them to give us more resources, more people, more money ... make them vote our way in Congress' essentially?" Holmes told Fox News. He said he got a written "directive" from Caldwell's office to find out what dignitaries "did for us" and what "we need to do next time in order to make things better."

Petraeus, the lead U.S. commander in Afghanistan, would not comment on the allegations after ordering an investigation. But other military officials either denied the claim or said Holmes had an ax to grind.

More On This...

"The NATO Training Mission -- Afghanistan categorically denies the assertion that the command used an Information Operations Cell to influence distinguished visitors," said Caldwell spokesman Lt. Col. Shawn Stroud.

Further, another senior U.S. military official who worked in Afghanistan for the training mission told Fox News that Holmes was disgruntled because his original assignment to influence Afghans was no longer needed when Caldwell took up the training command. Holmes apparently thought he was supposed to use "psychological operations" to find the Taliban and "turn them," according to the officer. But that wasn't what Caldwell's command wanted of him.

"The instructions given to him were to do homework on the visitors and provide prep material for the general," one military official said. "By his own statements, that is not the mission he wanted to do and he became disgruntled. Here is an analogy: I am a combat engineer trained to deal with mines and demolition. I like to do that, but for the past few years I am assigned to do staff work which is boring and not what I think I should be doing."

He never had contact with Caldwell, according to aides.

Holmes reportedly spent much of his time on Facebook setting up a strategic communications firm called SyzygyLogos with the woman with whom he was accused of having an improper relationship in Afghanistan, Maj. Laural Levine. Holmes denies that charge.

The U.S. military officer described the Rolling Stone article and subsequent news items as "a blatant twisting of the facts." Now that the commanders are muzzled because Petraeus has ordered an investigation, they are concerned they can't defend themselves.

"This was a guy who contributed little to the command and then is trying to salvage his reputation by making these allegations," the officer said of Holmes.

One aide who was with Caldwell almost continuously for the past year said, "At no point did Holmes ever provide a product to Gen. Caldwell."

The aide described the questions Caldwell asked of visiting dignitaries as innocuous. "What are their interests? Are they interested in the police, in the Army? In the budget? Let's show them what they are interested in," the aide said. "That's what the strategic communications directorate was tasked with."

The official said Caldwell was given 1,200 people to train the Afghan military in January 2010, though his requirement today would have been 7,000 trainers -- so he was understaffed and reportedly reassigned the "information operations" unit led by Holmes to do other things.

Before this, Holmes had been given a lot of freedom to freelance and conduct so called psy ops in downtown Kabul, which was devoid of the Taliban at the time, according to the official.

The assessment of those who worked with Holmes was that he had an inflated sense of self, was difficult and reluctant to perform his duties, and didn't understand what Caldwell was trying to do.

Holmes was asked to depart the theater early because he "added no value," according to one U.S. military officer.

He reportedly was rarely in uniform and spent much of his time setting up the communications firm with Levine for when they returned to the United States.

One senior military officer expressed concern that the facts had been misrepresented with regard to what Holmes was asked to do.

"Allegations become truth. They whip across the Internet. Judgments are made. That's the travesty," the officer said.

Holmes was not trained in psychological operations per se. According to U.S. military officials he never graduated from psy ops training at Fort Bragg, though he was in "information operations" which can include a psychological component. The task he was carrying out in Kabul before being reassigned included bringing music concerts to Afghans to "influence" their impression of the U.S. military.

But Holmes stood by his claims. He told Fox News the military has since retaliated against him by putting out "spurious information."

Responding to accusations by military officials that he had an "ax to grind," Holmes said:

"Do I have an ax to grind? Yeah. But the ax is this. If they can do this to a lieutenant colonel, what are they doing to the sergeants out there? I have a lot of education and training. ... I knew where to go and what the rules were and weren't."

After first broaching his concerns with higher-ups, Holmes became the subject of an investigation, which cited him for, among other things, going off base in civilian clothes and drinking alcohol.

In a "rebuttal" Holmes sent in June 2010, he claimed he was "tried and convicted by a 'Kangaroo Court.'" The memo, obtained by Fox News, suggested the investigation was biased and that the command was angry with him for resisting attempts to "focus my team on illegally providing themes and messages to influence the people and leadership of the United States."

U.S. lawmakers have played down these alleged operations, suggesting they were not influenced to do anything they didn't already want to do.

Fox News' Jennifer Griffin and Justin Fishel contributed to this report.