Authorities have begun sweeping up the leftovers of disgraced financier Bernard Madoff's posh Palm Beach lifestyle — including a mansion and vintage yacht — but it won't go far repaying investors bilked out of billions.

Barry Golden, a spokesman for the U.S. Marshals Service, said about five marshals arrived at the 8,753-square-foot, five-bedroom mansion late Wednesday afternoon, hours after the boats were seized.

Authorities spent about three hours securing the mansion, changing the locks and conducting an inventory of the property, which Palm Beach County records show had a taxable value of $9.3 million last year.

Click here for photos of what Madoff's money bought.

Click here for photos of the Madoff fraud scandal.

Golden said marshals filmed and photographed items in the house that might be removed at some point. "It's not an April Fools' joke," he said.

The mansion was unoccupied when federal authorities arrived, and the inspection took longer than expected because so many locks needed to be changed. They left around 9 p.m. after setting the alarm and posting a "no trespassing" sign on a window.

Palm Beach County property records show the Madoff mansion was purchased in 1994 under his wife Ruth's name for $3.8 million. Golden said the estate would be "monitored and maintained" and is no longer considered Madoff's property.

Earlier in the day, Golden said Madoff's 55-foot yacht named "Bull" and a 24-foot motor boat were taken from marinas on Florida's east coast. The yacht, a 1969 Rybovich, is worth $2.2 million.

"A lot of money was put into maintaining this boat," Golden said. "This boat was extremely well kept, extremely clean. Engine compartment was spotless. It looked like somebody took a bottle of 409 and scrubbed it every day."

Madoff, 70, is in jail in New York awaiting sentencing after he pleaded guilty to swindling billions from investors in what could be the biggest scam in Wall Street history. He faces up to 150 years behind bars.

Prosecutors are seizing as much as they can of Madoff's personal fortune, and have begun demanding millions from his relatives. Roughly 6,700 people have filed claims for a share of whatever is recovered. Thousands more are expected to come forward.

Last month, the feds went after jewelry belonging to Madoff's wife Ruth, along with other family assets. They also have set their sights on all four of the Madoff homes.

One of those mansions is a $1 million chateau in Antibes, France. Lawyers for a court-appointed trustee who tracked down more than $1 billion of Madoff's assets predicted that "the French will grab" the Madoff chateau, Reuters reported.

Court documents filed by Madoff's attorneys indicate Madoff and his wife had up to $826 million in assets — including the boats — at the end of last year.

If prosecutors get their way, Madoff and his wife, who has not been charged, will have to give up all their assets, including a $7 million Manhattan penthouse bought in 1984, the Florida home, a $1 million home in Cap d' Antibes, France, and a $3 million luxury home on New York's Long Island. The government also wants Madoff and his wife to forfeit $10 million in furnishings for all the homes and luxury cars, among other items.

Defense attorneys have indicated they may try to keep the Manhattan apartment, as well as about $62 million in securities, for his wife.

"We have no objection to the seizure or to the assets being sold," lawyer Ira Sorkin said in brief remarks Wednesday. "The proceeds of the sale will be put aside for discussion at a later date."

A small group of aggrieved investors — fearing they might be shortchanged by the forfeiture process — took steps Wednesday to try to force Madoff into personal bankruptcy so they can directly pursue his assets.

A court-appointed trustee already has begun liquidating Madoff's business assets. Court papers filed in Manhattan asked a judge to lift an order that currently prevents involuntary bankruptcy for Madoff himself.

Also Wednesday, Massachusetts' top securities regulator accused a major feeder fund for Madoff's investment scheme of misrepresenting its lack of knowledge about Madoff's operations.

Secretary of State William Galvin accused Fairfield Greenwich Group of Connecticut of civil fraud charges, saying company officials were coached by Madoff on how to answer questions about his investment practices and misrepresented how much they really knew.

The administrative complaint seeks restitution for Massachusetts investors for losses from Fairfield Greenwich.

Fairfield Greenwich said in a statement the allegations are false and misleading and it intends to "vigorously" contest them. "FGG is appalled by the Madoff losses suffered by its investors, including its employees and the three investors who reside in Massachusetts," the statement said.

Click here for more on this story from FOXBusiness.com.

The Associated press contributed to this report.