Updated

The so-called "Gang of 12" lawmakers shepherding the latest immigration bill through the Senate on Thursday said they believe their proposal can weather further attacks from both Democrats and Republicans while still keeping the most important provisions in the law.

"We had an avalanche of objections to the bill before there was a bill. ... And now that we're moving ahead, I think we're in about the right position. We see essentially no enormous road blocks or no poison pills or no killer amendments ahead that we can't deal with," Sen. Arlen Specter, R-Pa., said during a news conference — even as the measure's supporters narrowly defeated potentially fatal challenges.

Specter is one of the dozen lawmakers doing their best to keep the bill afloat among heavy opposition in some camps throughout Capitol Hill. The bill under consideration would tighten the borders and toughen standards for businesses hiring immigrants, in an effort to make sure the employees are legal residents.

The bill also would create a legal path to citizenship for some of the 12 million illegal immigrants already in the country, as well as offer a merit-based system for future immigrants, under which officials would assign more points to those with higher skills, and prioritize employment over family ties.

"I think we've made progress this week. We're under no illusion that there aren't some very difficult and challenging times ahead. But I think the American people have spoken this week when they said: We want the Senate to act. We want it to act responsibly. We want it to act now," said Sen. Edward Kennedy, D-Mass., the chief Democrat on the negotiating team of 12 Democrats and Republicans.

The Senate on Thursday rejected on a 49-48 vote a proposal by Sen. Byron Dorgan, D-N.D., to end the temporary worker program after five years. The Senate also beat a proposal by Sen. Norm Coleman, R-Minn., that would have allowed authorities to greater power to question suspected illegal immigrants.

The victory on the Dorgan amendment vote apparently came after Kennedy leaned on Sen. Daniel Akaka, D-Hawaii. When asked later what he said to Akaka, Kennedy grinned sheepishly and downplayed any political leverage he might have exercised to a simple, "Hi. How ya doing? Long time, haven't talked to you," and a simple, "Can you help us out?"

It was "an exercise in pragmatic politics," said Sen. Dianne Feinstein, D-Calif., who voted for this sunset provision last year. "It's a central tenet of the bill for the Republicans, for the business community, holding the bill together," Feinstein said.

When lawmakers return from the Memorial Day break, another of the chief hurdles the bill will have to overcome is the fear by conservatives who say the bill would grant amnesty to illegal immigrants. Both the senators and President Bush emphatically argued that the bill would not grant amnesty, and would instead be a fair, but tough system for immigrants to earn citizenship rights.

Bush said, "This bill does not grant amnesty. Amnesty is forgiveness without penalty." Rather, he said, the bill requires illegal immigrants to acknowledge they committed a crime, pay a fine, undergo background checks, stay employed and maintain a clean record. He said it also is important to have strong document verification to make sure illegal immigrants are not being hired.

Bush also pointing to the underground trade of money for passage into the United States as one of the biggest problems the temporary worker program would solve.

"Good, hard-working, decent people pay pretty good-sized money to be smuggled into the United States of America," Bush said, responding to a reporter's question.

"There is a document forgery industry in America. There are people who are willing to stuff people into temporary shelter in order to evade the law. I don't think this is American. I think the whole industry that exploits a human being is not in our nation's interest," Bush said.

"The best way to deal with this problem is to say, if you're going to come and do jobs Americans aren't doing, here is a opportunity to do so, on a temporary basis. I would much rather have people crossing the border with a legitimate card, coming to work on a temporary basis, than being stuffed into the back of an 18-wheeler," Bush said.

Bush has argued for comprehensive immigration reform that would give the estimated 12 million illegal immigrants living in the country a path toward citizenship, alongside a number of enforcement reforms including beefing up border security and employment verification.

"Immigration reform is a complex issue. It's a difficult piece of legislation, and those who are looking to find fault with this bill will always be able to find something, but if you're serious about securing our border and bringing millions of illegal immigrants in our country out of the shadows, this bipartisan bill is the best opportunity to move forward," Bush said.

The bill already has undergone a number of changes since the lawmakers struck the initial deal, including a reduction of the number of unskilled workers that would be allowed to visit annually, from 600,000 down to 200,000.

Another amendment agreed to Wednesday sought to toughen border security and workplace enforcement triggers, which under the change would have to be in place before the temporary worker program or the legalization of unlawful immigrants could go into effect.

A preliminary study of the immigration bill by the Congressional Budget Office, released late Wednesday, shows that the overall cost of the bill might not be a problem for supporters, but individual portions of the bill might overstep federal rules on costs to states, local governments and businesses.

While the bill would increase federal spending by as much as $17 billion through 2012, and as much as $38 billion through 2017, the federal revenues would nearly double the expected amount of costs — primarily through Social Security payroll taxes. The CBO predicts an increase in revenues by as much as $19 billion through 2012, and as much as $75 billion through 2017.

The report notes that the bill could run into problems related to employment eligibility requirements, and federal law governing how much cost the federal government can force state and local governments as businesses.

CBO says the bill would likely overrun federal thresholds in the Unfunded Mandates Reform Act for the private sector, which at 2007 levels would be $131 million.

"The most costly mandate would require all employers and other entities to verify the employment eligibility of workers. Based on the large number of individuals that employers and other entities would be required to verify under the bill, CBO expects .... costs of the mandates would exceed the annual threshold for private-sector mandates," the report says.

CBO says the law also would require no more than $66 million in federally imposed mandates on state and local governments to verify the work eligibility, but it's too soon to say how much the bill's requirements would cost because the regulations that would govern the program haven't been developed yet by the Department of Homeland Security.

CBO also said that a formal cost estimate was still in the works, and it would include projections for changes to the immigrant population, and more specific cost estimates an a number of federal programs.

While the bill's success is largely riding on whether the 12 leading senators are able to keep enough lawmakers happy — they continue negotiations on amendments that pose the biggest threats to the compromise — others lawmakers have expressed reservations about the group.

Sen. Jeff Sessions, R-Ala., derisively refers to them as "the masters of the universe."

Sen. Jim DeMint, R-S.C., is seeking far-reaching changes to the immigration measure and wants to remove the legalization program for unlawful immigrants.

"A lot of us don't feel like they're speaking for us, that this idea that we can't offer an amendment or it's going to blow up the deal is a bunch of nonsense," DeMint said.

"This is something that every member of the Senate should be participating in — not a small group," DeMint said. "There's never been a more emotional issue for people back home. They feel betrayed and violated. They don't trust our Congress."

The approach, however, may be the only way to ensure the bill makes it through the Senate and has a chance of being signed by the president.

FOX News' Trish Turner and Major Garrett and The Associated Press contributed to this report.