The White House said Tuesday the U.S. was "in touch" with Iraqi officials over the American contractors reported kidnapped from an apartment in Baghdad over the weekend.

White House Press Secretary Josh Earnest wouldn't elaborate. State Department spokesman John Kirby added, "without getting into details, I can tell you, the picture is becoming a little bit more clear in terms of what might have happened here. And we're working, again, very hard to try to resolve this."

An Iranian-backed militia is suspected of kidnapping the three men, a U.S. official tells Fox News.

The State Department and FBI are leading the investigation. Speaking to The Washington Post, a police major general described the building as a brothel, but other officials denied it.

Witnesses said men in uniform carried out the kidnapping in broad daylight Saturday, 100 yards from a police station.

"Gunmen in military uniforms came in five or six SUVs, they entered the building and then left almost immediately," said Mohammad Jabar, 35, who runs a shop down the street from the three-story apartment building where the Americans had been invited by their Iraqi interpreter.

"A few hours later we heard that three foreigners had been kidnapped by these gunmen," Jaber said.

The three were abducted in Dora, a mixed neighborhood that is home to both Shiites and Sunnis. However, they were then taken to Sadr City, a vast and densely populated Shiite district to the east, and there "all communication ceased," an Iraqi intelligence official told The Associated Press. The official spoke on condition of anonymity because he was not authorized to speak to reporters.

A similar scene unfolded in September, when masked men in military uniforms abducted 18 Turkish workers from a construction site in a Shiite neighborhood. A hostage video later showed the men standing before a banner that read "Death Squads" and "Oh, Hussein," a Shiite religious slogan. The workers were released later that month.

In December, gunmen driving SUVs raided a remote camp for falconry hunting in Iraq's overwhelmingly Shiite south, kidnapping 26 Qataris, who are still being held. Iraq's Interior Ministry said at the time that the abduction was "to achieve political and media goals," without providing further details.

Baghdad authorities said in a statement that the three Americans were kidnapped from a "suspicious apartment" without elaborating, and have provided no other details.

The kidnapping of the Americans comes at a time of deteriorating security in and around the Iraqi capital after months of relative calm. Last week two Iraqi journalists were killed within sight of a police checkpoint in Diyala province north of Baghdad.

The scale and sophistication of the recent kidnappings of foreigners suggest those responsible are operating with some degree of impunity, said Nathaniel Rabkin, managing editor of Inside Iraqi Politics, a political risk assessment newsletter.

"You kidnap 26 Qataris out in the desert, that's not like four or five yahoos out in the south. ... That's a pretty well-run operation. It must be some relatively established group that did it," he said.

The only groups operating in Iraq with those capabilities, Rabkin said, are the country's powerful Shiite militias.

Shiite militias have played a key role in battling the Islamic State group, filling a vacuum left by the collapse of the Iraqi security forces in the summer of 2014 and proving to be some of the most effective anti-IS forces on the ground in Iraq.

The government-allied militias are now officially sanctioned and known as the Popular Mobilization Committees. But many trace their roots to the armed groups that battled U.S. troops after the 2003 invasion and kidnapped and killed Sunnis at the height of Iraq's sectarian bloodletting in 2006 and 2007. Rights groups have accused them of kidnapping and in some cases killing Sunni civilians since they rearmed in 2014, charges denied by militia leaders.

Although the militias are fighting on the same side as the U.S.-led coalition against ISIS, many remain staunchly anti-American. When the Pentagon announced an increase in the number of U.S. special forces in Iraq last month, the spokesman for one militia vowed to attack them.

"Any such American force will become a primary target for our group. We fought them before and we are ready to resume fighting," said Jafar Hussaini, spokesman for the Iraqi Hezbollah Brigades, one of the most powerful Shiite militias.

Fox News' Lucas Tomlinson and The Associated Press contributed to this report.