A small-time money manager who did business out of a shabby storefront in Brooklyn was charged Tuesday with running a $40 million Ponzi scheme that secretly diverted client money to unauthorized ventures, including a mail-order pornography business.

Federal prosecutors and the Securities Exchange Commission said Philip Barry, 52, ran the scam without detection for 31 years until the economic downturn bankrupted him last summer.

Working from a small office in the Brooklyn neighborhood of Bay Ridge, far from the city's financial center, Barry claimed to be investing in stock options and guaranteed his neighborhood clients solid returns.

But in reality, investigators said, Barry was using much of the money to speculate on real estate. He bought an office building in Brooklyn and big tracts of undeveloped land upstate.

Authorities said he hid the scheme by feeding his customers financial statements boasting of hefty profits that didn't exist.

The SEC said he also used client money to cover personal expenses and operate a side business that sold "pornographic materials" through the mail.

Investigators said they learned of the scheme when Barry turned up at the U.S. Attorney's office in Manhattan in August of 2008 and asked to speak to a prosecutor. They said Barry acknowledged that, for years, he had been paying off his guaranteed profits by taking money from some customers to cover withdrawals made by others.

"The defendant's investment fund was nothing more than a classic Ponzi scheme," said U.S. Attorney Benton Campbell.

Since 1978, nearly 800 people have invested with Barry, prosecutors said. Some walked away with more money than they put in, but others lost hundreds of thousands of dollars.

A federal magistrate ordered Barry held without bond Tuesday at his arraignment in Brooklyn. His court-appointed lawyer did not immediately return a phone call.

Barry's story was reported in newspapers earlier this year after he was sued by dozens of disgruntled investors, some of whom have been ruined.

He told a New York Daily News reporter in August that he was no thief and just needed more time for his investments to pay off.

"It's all in real estate," Barry said. "I'm going to keep on working to make sure everyone gets the profit they are entitled to."