A congressional inquiry has found New York failed to follow instructions from Congress on the spending of $44 million in Sept. 11 aid (search) and should give the money back or get lawmakers to pass a law allowing the expenditure.

A New York official countered Tuesday night that if the federal government persists with the effort, it would have to take the money back directly from Sept. 11 victims.

"The federal government authorized these funds to go to 9/11 victims and their families in the first place, and given that they originally approved it, it would be wrong for them years later to take the money back from victims," said Jon Sullivan, a spokesman for the state workers' compensation board.

The bad news for New York comes just as the House is considering whether to take back another $125 million in Sept. 11 workers' compensation aid because the state has yet to spend it nearly four years after the 2001 terror attack.

The Government Accountability Office (search), the investigative arm of Congress, said in a 10-page memo Tuesday that the Labor Department "should seek recovery of $44 million improperly transferred" or get specific congressional approval for New York's use of the money.

The GAO found the state was not entitled to distribute the $44 million through other agencies. The state used the other agencies because officials said it would be a quicker method of getting aid to victims.

Labor Department spokesman David James said the agency is "reviewing the GAO legal memorandum to determine the course of action we will take."

The House will consider later this week whether to take back an additional $125 million in unspent workers' compensation aid. The White House Office of Management and Budget (search) has sought to retrieve the money because it has not been spent.

New York lawmakers vowed to hold onto the workers comp funding for future Sept. 11-related claims.

The GAO found the federal law that provided the $44 million did not allow it to be used that way.

In such instances, GAO lawyer Anthony Gamboa wrote, the funds "must be recovered ... even when those expenditures have been incurred innocently."

Jon Sullivan, a spokesman for the state workers' compensation board, defended the state's actions as "fully appropriate" and with clear permission from the Labor Department.

The federal government granted more than $20 billion to New York to help recover and rebuild after the attacks.

Rep. Carolyn Maloney, D-N.Y., had requested the original GAO investigation into the workers' compensation program run by the state, but on Tuesday said it made no sense for the government to start taking back even more money.