The most serious impasse in years between the owner of ground zero and the developer building office towers on half of the site is headed for government intervention.

Larry Silverstein and the Port Authority of New York and New Jersey started talks about seven months ago to rewrite Silverstein's lease to build three of five office towers to replace the destroyed World Trade Center.

Silverstein, unable to find private financing for all the towers in a poor real estate market, wants the port to back at least two of them.

The Port Authority said more than a month ago that it would back one tower with about $800 million, provided Silverstein invests more money in building the infrastructure his towers will share with a huge transit hub and Sept. 11 memorial.

Top politicians from two states are meeting Thursday at Mayor Michael Bloomberg's mansion to try to pressure both sides into an agreement and avoid yet more delays in the site's tortuous rebuilding history.

The departure of commercial tenants in downtown Manhattan — at a time when 10 million square feet of office space is planned at the site — is the key to any question about timing and financing at the site, a business leader said.

"I don't think it serves anyone well to have a battle on the World Trade Center site. It's in our shared interest and the community's interest, clearly and in the city's interest, to move forward as quickly as possible," said Kathryn Wylde, CEO of the Partnership for New York, a downtown business group. "But you can't ignore the market, and right now there isn't one."

The Port Authority has said publicly for weeks that the uncertain market would make it foolish to build all five planned office towers as planned, saying at least three should be put off until the market rebounds.

On Tuesday several civic groups wrote Bloomberg and the governors of New York and New Jersey, saying the agency would be better off using its money on transit in the region than on a "speculative office building on the World Trade Center site."

But a complex 2006 lease — an amended version of the lease Silverstein originally signed six weeks before the Sept. 11, 2001, terrorist attack — gives Silverstein the right to build three towers or take the agency to court for failing to let him.

Silverstein officials argue they have turned over $2 billion — in monthly rental payments for the undeveloped space and a share of Sept. 11 insurance money — to the Port Authority since 2001, and said the agency has failed to rebuild on time anything it is responsible for.

An elaborate, $3.2 billion transit hub sitting in the middle of Silverstein's towers has been under construction for four years, and is five years behind its original schedule. Ground was broken in 2004 for the tallest skyscraper planned at the site, but it was redesigned and moved; it may open in four more years.

"We are not asking the Port Authority for their money. But in order for us to justify letting the port keep the $2 billion they have collected from us ... we are asking for their assistance," said Janno Lieber, who heads trade center construction for Silverstein.

The Port Authority has estimated Silverstein's proposal to back his towers would cost the agency well over $4 billion, while Silverstein's estimate is closer to $2 billion, three officials familiar with the negotiations told The Associated Press. The officials spoke on condition of anonymity because of confidentiality agreements governing the talks.

While the agency and several public officials have backed financing just one Silverstein tower, Assembly Speaker Sheldon Silver suggested that he would support backing two towers. He called for Thursday's meeting over a week ago, which will include Gov. David Paterson, New Jersey Gov. Jon Corzine and Bloomberg. The city doesn't own the trade center site but has agreed to lease nearly half the office space in the Silverstein tower under construction.

Bloomberg last week suggested that the agency would have difficulty negotiating with Silverstein, who "doesn't have any skin in the game" because the towers are backed with bonds and Sept. 11 insurance money.

On Wednesday, he said he hoped only to restart talks between two sides that have been stalled for over a month.

"We'll have a few ideas to throw out — nothing magical about our ideas — but anything we can do to get them going," the mayor said. "It's in the interest of this city and this country to get development going at the World Trade Center site."