More than a dozen Islamic State fighters from Iraq and Syria -- some with direct ties to the group's leader Abu Bakr al-Baghdadi -- are now in Libya working on behalf of the terror group, according to U.S. and European sources.

And the U.S. has no authority to take them out.

The fighters are in addition to dozens of ISIS sympathizers in the country. The situation is raising fresh security concerns, as the U.S. and its allies struggle to keep tabs on the many arms of the ever-expanding ISIS network.

One source, who is not authorized to speak on the record given the sensitivity of the data, said they would not be surprised "if the next 9/11 came out of Libya."

An ISIS presence has been flagged in Benghazi, Derna and Sirte, in addition to training camps providing a steady flow of recruits. But a State Department spokeswoman insisted U.S. policy is not to blame.

More On This...

"Are we happy with the situation in Libya from a governance and security perspective? No. Absolutely not," spokeswoman Marie Harf told reporters."At the end of the day, though, this is not just a line, it's true. This is ultimately up to the Libyan people, leaders in Libya to take the security of their country into their own hands and try to move it in a better direction with our help, the U.N. and others' help as well."

The nation's most senior intelligence officer, Director of National Intelligence James Clapper, recently testified before the Senate Armed Services Committee that Libya is not only a safe haven for ISIS but at least six other terror groups, including Ansar al-Sharia, which paraded police vehicles through Benghazi last month.

This kind of freedom to train, plot and fundraise helped the growth of Al Qaeda and the Taliban in Afghanistan before 9/11.

"There are, in addition to ISIL, probably six or eight other terrorist groups that have gathered in Libya. So it's a magnet because, essentially, it's ungoverned," Clapper said.

Clapper also said the U.S. should send its own aerial surveillance to Libya or partner with France, which already has assets in the region to identify ISIS members.

"From an intelligence perspective, I think clearly we need to step up our game,” Clapper said.

After Libyan dictator Muammar Qaddafi was pushed out with the support of NATO and the U.S., Libya descended into civil war between rival Islamist factions. A central player is Abdelhakim Belhadj, who was once courted by the Obama administration and members of Congress, but now leads a political faction that supports the establishment of sharia law.

"The lesson from Belhadj is that the United States can't rely on the third parties to achieve its objective in the region," retired general and Fox News contributor Robert Scales said.

While administration critics say the establishment of Islamist groups in Libya reflects one of the single greatest policy failures of the Arab Spring, some analysts believe the influence of ISIS is overstated, and the group will have profound difficulty navigating the north African nation's complex dynamic of tribes, militias and political groups.

Catherine Herridge is an award-winning Chief Intelligence correspondent for FOX News Channel (FNC) based in Washington, D.C. She covers intelligence, the Justice Department and the Department of Homeland Security. Herridge joined FNC in 1996 as a London-based correspondent.