Updated

The trustee trying to unravel Bernard Madoff's massive Ponzi scheme is threatening legal action to recover $735 million from investors who unwittingly made money off the swindle.

For decades, Madoff paid out steady profits to his clients, telling them the money had been earned in the stock market. The gains, though, were fictitious, and Madoff pleaded guilty last month to stealing funds from some investors to pay bogus profits to others.

Click here for photos of what Madoff's money bought.

Click here for photos of the Madoff fraud scandal.

Click here to read the complaint.

In recent days, court-appointed trustee Irving Picard has sent letters to 223 investors, ordering them to return money they withdrew from their accounts at Bernard L. Madoff Investment Securities in the six years before the scheme collapsed.

"These amounts were paid to you at the expense of other customers while BLMIS was insolvent," said one letter, sent to an investor who gradually withdrew $975,000 between 2003 and 2008. "The Trustee demands that you immediately return such amounts to the Trustee for the benefit of all defrauded creditors."

Investors who don't return the money could face legal action, the letters said.

Madoff victims have been fearing these so-called clawbacks since the scandal broke, in part because some people who withdrew significant amounts have long since spent the money, then lost the rest of their savings in the fraud.

Lawyers representing some of those investors expressed dismay with the letters and said they would challenge the legality of the demand.

"The effect of these letters is to scare the living daylights out of people who have already been victimized once by Bernard Madoff," said Jonathan Landers, a lawyer representing a large group of victims.

Picard has said the clawback effort is an attempt to return as much money as possible to its rightful owners. His staff has so far found around $1 billion in customer assets — just a fraction of the billions that Madoff is believed to have collected.

Any money returned by investors would be pooled and divided among the victims, based on how much each lost in the scam.

Picard has pledged not to try and recover money from "net losers" in the scam, meaning anyone who gave more money to Madoff than they ever received in false gains.