"To pass this bill in the next couple of weeks would materially increase the risk that the economy would stall out and we would have a double dip," he told reporters at the White House.
Summers added that he thinks Congress will pass the deal President Obama struck with Republicans, an agreement that has come under withering criticism from liberal Democrats over the two-year extension of tax cuts for the rich as well as the middle class.
Summers said that he doesn't think that Congress would take a step that "materially increases the risk that this economy will stall out."
Positive reviews coming in from economists, Summers said, show the deal is the right thing to do.
Sen. Jon Kyl, the No. 2 Republican in the Senate, told reporters that a tax cut bill could be on the Senate floor as early as Thursday.
But, in a surprise move, Senate Democrats, meeting behind closed doors to discuss the tax deal, have asked the administration for "changes" to the bill, according to Sen Dick Durbin, D-Ill., the No. 2 Democrat in the chamber. Durbin would not detail what those changes are, deferring to Senate Majority Leader Harry Reid, D-Nev.
Kyl said an official analysis and price tag, referred to as a "score," from the Joint Committee on Taxation (JCT) are not yet ready, but that the Senate could move to the bill anyway.
Vice President Biden traveled to the Capitol Wednesday for a private meeting with House Democrats who are resisting the deal, saying it gives away too much to the wealthy.
Some Democrats are unhappy that Obama agreed to extend expiring tax cuts not just for the middle class but also for higher earners, and that he agreed to impose a lower estate tax on wealthy heirs. Both provisions are seen by many Democrats as giveaways to the rich that will do little to help the economy.
In return, Democrats would get extended jobless benefits for people who have been unemployed for long stretches. Workers would also see their share of Social Security payroll taxes cut by nearly a third for the coming year, a boost in take-home pay meant to encourage spending and aid the economy.
Liberals are going as far as trying to prevent outgoing House Speaker Nancy Pelosi from bringing the tax cut compromise to the floor.
Reps. Peter DeFazio of Oregon, and Jim McDermott and Jay Inslee of Washington state are crafting a letter to share with the House Democratic Caucus that would try to block the bill from seeing the light of day. They're hoping to get 60 signatures on their letter and then force a vote in the caucus.
"There does not seem to be a majority of the Democrats who support the deal negotiated by Vice President Biden," he told Fox News. "So we want to have a record vote in the caucus on a resolution that says this resolution should not go to the floor without a majority of Democratic votes."
DeFazio said House Democrats aren't going to keep rubber-stamping deals they don't like.
"All through this administration, the White House has been a spear carrier for the president," he said. "They sent something over, the speaker has jammed it through and then Obama has negotiated with the Senate. And they're trying to get us to jam it through again. And a number of us are just saying that's enough."
Obama rejected claims that he betrayed Democrats by cutting a deal with Republicans. Speaking to reporters at a joint appearance with Polish President Bronislaw Komorowski, Obama said economists predict higher job growth in 2011 and 2012 if Congress passes the agreement and urged lawmakers to examine the details of the deal and "get this done."
But House Democrats are not listening.
"So far, the response has not been very good," Pelosi said. She called the estate tax provision "a bridge too far."
Fox News' Trish Turner, Chad Pergram contributed to this report.