The Federal Emergency Management Agency (FEMA) has asked hundreds of victims of 2012's Superstorm Sandy to return nearly $6 million paid as compensation, and thousands more could be asked to do the same. 

The Associated Press reports that as of early September, FEMA had asked around 850 aid recipients to return a collective $5.8 million. Approximately 3,600 other cases were still under review. 

FEMA's campaign to recover overpayments, called "recoupment" in agency lingo, typically involves inadvertent violations of eligibility rules, bureaucratic mistakes or missing documentation, rather than outright fraud.

Many people asked to return money were deemed ineligible because their damaged properties were vacation houses or rental properties, not their primary residences. Others had double dipped into the aid pool, with more than one household member getting payments. Some received FEMA money for things later covered by insurance.

As of July 30, the average demanded refund was $6,987, a sum that could be difficult for many, given the modest annual incomes of most aid applicants. Roughly half of the households under scrutiny reported an annual gross income of $30,000 or less.

The larger pool of cases still under review as of that date involved $53 million in aid payments — or about 3.7 percent of the total given out by FEMA through its individuals and households program — though any potential refunds would likely involve only a portion of that money.

"For most people, the money is long gone and long ago spent on storm recovery," said Ann Dibble, director of the New York Legal Assistance Group's storm response unit, which has been helping about a dozen families fight a FEMA clawback.

Twenty five cases under review involved households reporting a gross annual income of $500,000 or more.

The list of people asked to return cash includes Gary Silberman of Lindenhurst, New York, who got a letter last November demanding just under $17,000. The agency said he was ineligible partly because he and his elderly father, Albert, had both applied for disaster funds even though they were living together. That's a violation of FEMA double-dipping rules.

The Silbermans also were barred from getting some types of aid because they had failed to buy flood insurance after getting $25,000 in FEMA aid for flood damage during Hurricane Irene a year earlier. Federal rules require homeowners who accept disaster aid to buy insurance to protect them from future storms.

Silberman says he should still qualify for the money because he was a rent-paying tenant in his father's house, not a dependent, but FEMA has so far rejected his appeals.

"I lost my home. I lost everything. I don't have $17,000 to give back," Silberman said.

Sandy was among the costliest hurricanes in U.S. history. More than 280 people died in the U.S. and the Caribbean. When the storm struck the New York and New Jersey coastlines, the surging ocean poured into densely populated seaside neighborhoods, flooding hospitals and subway tunnels and turning entire communities into soggy, moldy wrecks.

About 179,000 households in New York and New Jersey received FEMA payments for temporary housing or damaged property. Most of the recoupment actions have also been aimed at residents of those states, but the agency is also reviewing payments to households in Connecticut, Maryland and Rhode Island.

FEMA mobilized for Sandy hoping to avoid problems that plagued the aid distribution process following Hurricane Katrina's strike on the Gulf Coast in 2005. That destructive storm forced the overwhelmed agency to relax internal controls to speed up relief efforts, which led to huge numbers of people getting money they shouldn't have received.

FEMA's attempts to recover hundreds of millions of dollars, often from people who couldn't afford to pay, led to a court fight and a procedural overhaul. By 2011, the agency had mailed out letters to at least 90,000 households asking for aid refunds. Congress authorized the agency to waive much of that debt.

The agency says it has since gotten better at making sure aid only goes to the right people, and in proper amounts.

"They have a lot more controls in place," said John Kelly, the Department of Homeland Security's assistant inspector general for emergency management oversight.

In most cases, aid recipients won't know they're under scrutiny before a notice arrives in the mail demanding repayment. FEMA says it has reviewed thousands of cases and decided the payments were warranted.

FEMA isn't the only government agency trying to recover Sandy-related funding.

Last spring, the U.S. Department of Housing and Urban Development urged officials in New Jersey and New York to step up their scrutiny of hundreds of millions of dollars in other federal rebuilding grants.

New Jersey's Department of Consumer Affairs says it has asked 51 homeowners to return a combined $940,000 — a number that will probably will rise as the state looks at more cases.

Ted Friedli got a letter from the state in July giving him five days to return a $26,101 grant he'd received to repair his flooded house in Long Branch, New Jersey.

The letter explained that auditors had pegged the cost of fixing the house and raising it up on a higher foundation at less than the family had already received from their insurance policies and a low-interest federal disaster loan. That meant that the state grant was pure surplus and had to be returned.

"I was stunned," Friedli said of his reaction when he got the notice, which he said came with no warning and no mention of any possibility of appeal.

New York officials were unable to say how much money they had recouped so far from grant recipients who had been overpaid or belatedly declared ineligible.

During one audit earlier this year, HUD officials warned that New York wasn't doing enough to ensure that rebuilding grants weren't "unduly enriching homeowners beyond their actual expenses."

It cited several examples in which checks had been sent improperly. One homeowner returned a check for $119,138 after realizing that it included payment for damage covered by insurance. A month later, she got another letter saying she had been approved for an even larger overpayment of $121,731.

Office of Storm Recovery spokeswoman Barbara Brancaccio said New York state has since improved its record-keeping procedures.

Federal prosecutors have, so far, brought far fewer fraud cases against aid recipients than they did after Katrina, though it is unclear whether that is because less theft has occurred or because investigators haven't been as aggressive.

By Katrina's second anniversary, more than 768 people had been charged with hurricane fraud-related crimes in 41 federal jurisdictions. By comparison, only seven people so far have faced federal fraud charges related to disaster aid payments in the states where FEMA distributed aid after Sandy, according to an AP survey of prosecutors' offices.

Kelly, the assistant inspector general, said the modest tally of Sandy-related fraud cases likely reflects a shift in law-enforcement priorities. The Justice Department enforced a "zero tolerance" policy for Katrina fraud, he said. But he believes it has since given prosecutors more freedom to turn down cases involving relatively small sums of money.

Justice Department spokesman Peter Carr declined to comment on the number of Sandy-related prosecutions, but he noted that a federal task force on disaster fraud has received more than 30,000 complaints from the public about possible Katrina-related fraud compared to 1,758 complaints related to Sandy.

The Associated Press contributed to this report.