Updated

Rep. Charles Rangel, the 80-year-old longtime congressman from Harlem, may scoff at the ethics violations he’s facing, but his Republican opponents are seizing on them as a rallying cry to regain control of the House in an anti-incumbent year that has even the safest of lawmakers worrying about their re-election.

Rangel's Democratic primary challengers says the ethics violations are another reason why he should call it quits after 40 years in office.

It was disclosed Thursday that Rangel is being charged with multiple ethics violations. The ethics committee won't reveal the specific charges until next Thursday at a public meeting.

Rangel says he looks forward to fighting the ethics charges.

"I am pleased that, at long last, sunshine will pierce the cloud of serious allegations that have been raised against me in the media," Rangel said in a statement posted on his website. "I will be glad to respond to the allegations at such time as the Ethics Committee makes them public."

More On This...

But when he looks behind him, Rangel may not find many Democrats at his side. Even House Speaker Nancy Pelosi, who once was his most vocal supporter as the allegations first surfaced, has been stepping further away from Rangel.

In a statement that conspicuously lacked any hint of support, Pelosi spokesman Brendan Daly said the action taken against Rangel "would indicate that the independent, bipartisan ethics committee process is moving forward."

House Majority Leader Steny Hoyer, D-Md., dodged questions on Friday about how pushing the Rangel hearing back into late July and August could adversely affect Democrats in November's elections.

"The ethics process is working," Hoyer allowed.

The silence by Democrats is due to the timing of the ethics trial of the New York congressman and former Ways and Means Committee chairman. It coincides with campaign season, and Democrats will have to defend their party's conduct to voters. If enough of them lose, the party could cede control of the House.

Republicans are already going negative, reminding voters that Pelosi promised to "drain the swamp" of ethical misdeeds in Congress.

But Pelosi and other Democrats had helped fend off Republican efforts over two years to strip Rangel of his gavel on the Ways and Means Committee. Rangel finally stepped down in March after a closed-door meeting with Pelosi in the wake of the ethics committee's conclusion that the lawmaker violated House rules on two trips to Caribbean conferences. The committee said Rangel should have known that corporate money paid for the trips.

Now Republicans are poised to use Rangel's ethics issues as a weapon this fall leading up to the congressional elections and they're targeting all the money he has raised for scores of Democratic candidates before his ethics problems surfaced.

In Pennsylvania, Republican Senate candidate Pat Toomey called on his Democratic opponent, Rep. Joe Sestak, to return $12,000 in contributions from Rangel's campaign committee and leadership political action committee.

"Throughout the campaign, Congressman Sestak has spoken about accountability and putting principle over politics, but it is now becoming clear that his pledges and lofty promises are just hollow words from another Washington insider," Toomey campaign communications director Nachama Soloveichik said in a written statement.

In New Hampshire, the state Republican committee called on Rep. Paul Dodes to return $17,000 and Sen. Jeanne Shaheen to give back $10,000 of "tainted campaign contributions they received from disgraced" Rangel.

"For months, Paul Hodes and Jeanne Shaheen have refused to take a strong stand against Mr. Rangel's unethical behavior and return his tainted contributions," the committee spokesman Ryan Williams said in a written statement. "It's time for Paul Hodes and Jeanne Shaheen to finally do the right thing and donate Congressman Rangel's dirty money to charity. Any attempt by these Democrats to cling to Rangel's filthy donations will serve as an implicit endorsement of his unethical behavior."

A number of congressional Democrats have already returned donations to Rangel.

In New York, Republican Senate candidates David Malpass and Gary Berntsen called on Sen. Kirsten Gillibrand to donate $29,000 of Rangel contributions to charity and Sen. Charles Schumer to cancel a Rangel fundraiser scheduled for Aug. 11.

"Senator Gillibrand likes to talk a good game when it comes to ethics, but now is her opportunity to put her money where her mouth is and finally take a position on the corruption issues facing Congressman Rangel," Malpass said in a written statement.

"On Aug. 11, Senator Schumer is set to host a fundraiser for the very same Charlie Rangel that received a House Ethics violation charge today," Berntsen said in a written statement. "What do you think Chuck will do with this one?"

While it's unclear whether these lawmakers will return the money, many Democrats now wouldn't dare touch a contribution from Rangel's leadership fund and might pressure him to accept the charges or even get out of his re-election race.

"I'm sad that we've gotten to this point," said New York banker Vince Morgan, one of Rangel's Democratic primary challengers who once served as Rangel's campaign director. "I hope he uses this opportunity to do the right thing for this district. I hope he ends up taking his 40 years of institutional knowledge and investing it in the next generation of leadership."

Morgan said that while a lot of voters still support Rangel, many of them are disappointed too.

"Any time you have been in office for so long you confuse that you've been elected to represent the people with your own personal agenda...it shows that you're out of touch with the reality of the situation," he said.

Another primary challenger, New York Assemblyman Adam Clayton Powell IV, whose father Rangel beat 40 years ago to gain the seat, said the ethics violations doesn't change the fact that "people are ready to turn the page."

Powell, who lost badly to Rangel in the 1994 primary but is waging a competitive campaign this year, said he may not have launched a bid if Rangel hadn't lost his gavel. But since he lost his power and is 80 years old, Powell said there's no need for Rangel to have another two years in office that would amount to simply a "symbolic term."

"We have too many great needs for affordable housing, better education, health care, jobs," he said. "We cannot afford just a symbolic term."

Rangel had a choice.

His lawyer had been negotiating with the House ethics committee to settle his case. But to end it, Rangel would have had to accept the allegations. Rangel had been willing to accept some, but that didn't satisfy the committee, according to a person familiar with the talks but not authorized to be quoted by name.

Several persons familiar with the allegations, who were not authorized to discuss them publicly, said some of the charges against Rangel, who has spent 40 years in Congress, were related to:

--Rangel's use of official stationery to raise money for the Charles B. Rangel Center for Public Service at City College of New York.

--His use of four rent-subsidized apartment units in New York City. The city's rent stabilization program is supposed to apply to one's primary residence. One had been used as a campaign office, raising a separate question of whether the rent break was an improper gift.

--Rangel's failure to report income as required on his annual financial disclosure forms. The committee had investigated his failure to report income from the lawmaker's rental unit at the Punta Cana Yacht Club in the Dominican Republic. Rangel also belatedly disclosed hundreds of thousands of dollars in investment assets.

Sanctions can range from a damaging committee report to censure by the House and even expulsion, a punishment reserved for only the most egregious violations.

FoxNews.com's Stephen Clark, Fox News' Chad Pergram and The Associated Press contributed to this report.