If you didn't know where to look, you'd probably never find Islamberg, a private Muslim community in the woods of the western Catskills, 150 miles northwest of New York City.

The town, sitting on a quiet dirt road past a gate marked with No Trespassing signs, is home to an estimated 100 residents. There are small houses and other buildings visible from the outside, but it is what can't be seen from beyond the gate that has some watchers worried.

Click here for video.

Islamberg was founded in 1980 by Sheikh Syed Mubarik Ali Shah Gilani, a Pakistani cleric who purchased a 70-acre plot and invited followers, mostly Muslim converts living in New York City, to settle there.

The town has its own mosque, grocery store and schoolhouse. It also reportedly has a firing range where residents take regular target practice. Gilani established similar rural enclaves across the country — at least six, including the Red House community in southern Virginia — though some believe there are dozens of them, all operating under the umbrella of the "Muslims of the Americas" group founded by Gilani.

Federal authorities say Gilani was also one of the founders of Jamaat al-Fuqra, a terrorist organization believed responsible for dozens of bombings and murders across the U.S. and abroad. The group was linked to the planning of the 1993 World Trade Center bombing, and 10 years earlier a member was arrested and later convicted for bombing a hotel in Portland, Ore.

Shoe bomber Richard Reid has been linked to the group, along with convicted D.C. sniper John Allen Muhammad. But it is Sheikh Gilani who creates the most controversy and concern.

Gilani has told his followers that "Zionist plotters" plan to rule the world, and he encourates them to leave America's cities and avoid the "decadence of a godless society." Gilani is the man American reporter Daniel Pearl was trying to interview in Pakistan when he was kidnapped and beheaded. The Sheikh was taken into custody and later released by Pakistani authorities; he denies any involvement in Pearl's murder.

Gilani also denies any connection to Jamaat al-Fuqra, as do residents of the MOA compounds, who say the "terrorist" group doesn't exist and was created by enemies of Islam hoping to destroy their communities. Members also deny sending a portion of their earnings to the Sheikh, but a former resident told FOX News that 10 to 30 percent of their income is regularly delivered to Gilani in the form of cash donations.

FOX News attempted to visit Islamberg after earlier efforts to set up on-camera interviews were rebuffed. A spokesman said by phone that residents typically shy away from interviews since they worry their words will be manipulated and turned against them. He accused FOX News of misrepresenting the group and suggested covering an Islamic festival in Binghamton later in the month to celebrate the birthday of the prophet Muhammad. Then he hung up.

Some residents, collecting their mail outside the compound or stopping for a short spell at its edge, spoke briefly of life on the inside. One woman told FOX News she was happy to be "away from the city and away from the drugs and crap that's going on," raising her family safely in Islamberg.

Another resident drove out, dressed in a cap and robe, video camera in hand, and moved so close to a female producer that he actually made contact. The resident kept his camera rolling for the next 15 minutes without saying a word, but he finally began to answer questions.

The man said residents get along with their neighbors and don't trust the media, which he said paints their town as a guarded compound "where no one can enter and exit."

"It's a village," he insisted, "with people of all backgrounds, cultures and races."

He said he was a 20-year veteran of the Air Force, still active in the reserves, who grew up in Brooklyn and converted to Islam while in the service.

"I got awards for 'bridging the gap' between American and Saudi soldiers," he said, noting that he had helped process the bodies when 19 U.S. servicemen were killed in a 1996 terror attack in Saudi Arabia. He said there were many veterans in Islamberg, including one with a Purple Heart citation from Vietnam.

The man denied the existence of Jamat al-Fuqra, and at first denied giving a share of his earnings to Sheik Gilani, but he later admitted that "all churches have tithes."

During the interview, two state police cars roared up, as one of the residents had called the cops, a common occurrence when reporters show up at the gate.

Local police told FOX News there has been plenty of rumor and innuendo over the years but very little trouble. The FBI's Albany Division said the agency has an open discourse with the residents of Islamberg. They've visited the compound but won't discuss whether there are any ongoing investigations.

That has not dispelled the worries of some watchdogs. Robert Spencer, director of Jihad Watch, says the group is making a concerted public relations effort to present a benign face and hide its violent past.

"I think we need to be very much on guard about every member of these compounds," he said. Though Spencer admits there is nothing inherently wrong with living in isolation, he stressed that "they're not at all open to visitors, they're not at all open to scrutiny and there's an abundance of evidence of sinister goings-on."

Spencer offered no evidence to back his misgivings, but suggested political correctness may be hampering investigations. He says the group's connection to Sheikh Gilani is reason enough to be concerned that they're planning for "something on a larger scale and longer term," to "further the causes of the global Islamic Jihad", something MOA has repeatedly denied and scoffed at in the past.

Residents call it a peaceful place to raise a family away from the pressures of the city, and maintain that the group is woefully misunderstood.

"I used to work customs when I was overseas and I come here and get harassed at the grocery store and the bank. A clerk claimed my military ID was invalid because I'm Muslim," the resident told FOX News.

Critics, lacking an eye into the cloistered community, still wonder whether it's something more.