With House Republicans settling on new committee assignments, most of the chatter on Capitol Hill has centered on a few key changes among the chairmanships and whether the Republican leadership is trying to bring its more independent members to heel.

"If you don't toe the line you're gone — consolidation of absolute power," one Democratic congressional aide, who asked not to be identified, told FOXNews.com.

Democrats call the developments "a purge" of more independent-minded chairs that will make the next Congress even more difficult for the minority party.

"They are moving to quash any differing opinions within their own party," Democratic Minority Whip Steny Hoyer, D-Md., said in a statement.

Headlines about two GOP replacements have fueled speculation about whether the outgoing chairmen are being punished or whether the leadership just wants to remind Republicans in the new 109th Congress that they are all on the same team.

"You sort of have to remind folks that you are only as strong as you are perceived to be out there," said Greg Crist, spokesman for House Republican Conference Committee Chairwoman Deborah Pryce, R-Ohio.

This means, in part, showing a unified stance on limited spending. Republicans were criticized by fiscal conservatives, as well as deficit-hawk Democrats, for wild spending in the last Congress.

Sources say Rep. Chris Smith (search), R-N.J., was the first casualty of the new "unity." As chairman of the Veterans Affairs Committee, Smith pressed for increased spending on the Veterans Administration (search) budget, putting him on a different page from party leadership, with whom he has tangled before.

Smith is being replaced by Rep. Steve Buyer (search), R-Ind., who is considered more fiscally conservative, and has shown a proclivity toward cutting back spending on VA health care, according to veterans' advocates.

"We felt strongly and still feel strongly about Chris Smith," said Rick Weidman, director of government affairs for Vietnam Veterans of America (search). "That he's being punished for being too pro-veteran in the Congress while we are at war is breathtaking."

House Speaker Dennis Hastert, R-Ill., had no comment on Smith, who was also taken off the committee, but did post on his Web site a statement about Buyer, a Persian Gulf War veteran.

"I'm confident under Steve's leadership, those who faithfully fought to defend our freedoms will continue to have access to quality health care," Hastert said.

Weidman is not so sure, saying that the leadership thinks too much money is already going into VA health care. "Mr. Buyer is exactly the opposite [of Smith] — he's very partisan. Based on his past, this [committee] is likely to be more of a partisan entity in the next Congress."

Even staunch conservatives say they were taken aback by Smith's removal. "Smith is derided by the leadership as a 'liberal' who is in organized labor's pocket, but his voting record is moderately conservative," wrote columnist Robert Novak, who called Smith a "tireless supporter of veterans."

Laura Zuckerman, spokeswoman for Buyer, said her boss would not comment on Smith, but would say that he and Smith have genuine "disagreements" over handling the VA system.

"He and Congressman Smith have a lot of different views about the VA, on where the Congress sees the VA in the future," she said.

A Republican committee aide who did not want to be named said the leadership had done no wrong. "Our opponents were using Smith to advocate against our policies," the aide said. "No one has discounted veterans, or questioned his loyalty or desire to help veterans, we just had a fundamental disagreement on how to approach that."

Meanwhile, it looks like Rep. Joel Hefley (search), R-Colo., will not be asked to stay on in his post as chairman of the House Standards of Official Conduct Committee (search), sources say. Leadership aides have maintained that per House rules, Hefley’s three-term limit as chairman had expired.

But critics say he is being punished for leading three admonishments last fall against House Majority Leader Tom DeLay (search), R-Texas, who was found by the ethics committee to have crossed the line of appropriate conduct. His political action committee, Texans for a Republican Majority, is currently the target of a criminal probe in Texas.

"Such a mean-spirited and vindictive act by House leaders can only further diminish public confidence in Congress by painting it as hopelessly politicized and self-dealing," said a recent public statement by eight government watchdog groups calling themselves the Congressional Ethics Coalition (search).

They and others say the speaker should waive the rules so that Hefley can stay on and oversee the remaining investigations, or else the power of the committee will be "decimated." Hefley has said he would accept the waiver.

Jonathan Grella, spokesman for DeLay, said Democrats had manufactured the ethics flap and that the GOP was determined to proceed with its agenda "undeterred."

Crist said he found it "mind-boggling" that Democrats would ask them to bend the rules. "I find it extremely hypocritical."

The leadership is also replacing term-limited Appropriations Chairman Bill Young, R-Fla., with Rep. Jerry Lewis, R-Calif. He beat out two contenders, including Rep. Ralph Regula, R-Ohio, who was next in line to Young, according to seniority.

"[Lewis] is a good guy, highly thought of, but it was clear from the selection process that whoever was selected for the Appropriations chairmanship was going to be expected to toe the line," said the Democratic congressional aide. Calls to Republicans on the Appropriations Committee were not returned.

Larry Sabato, director of the Center for Politics at the University of Virginia, said it's the leadership's right to whip members into shape. "They are doing what leaders do, they make sure [committee chairmen] serve them well, for their interests and for the interests of the party," he said. "For those who think [chairmen] are there to serve only in the interests of the American people — get real."