Updated

The political handicappers don't think that Democrats have much of a chance to win control of the House of Representatives this year.

Still, there's a twisted, silver lining in that for House Minority Leader Nancy Pelosi (D-CA). If Republicans hold the House, lawmakers will return the gavel to House Speaker John Boehner (R-OH). But it's likely Pelosi will have fewer Democratic defectors as she runs against Boehner for House Speaker when everyone votes on the floor come January.

Why?

Wither the Blue Dogs.

"A Blue Dog was a yellow dog Democrat squeezed from the right and the left and so he turned blue," said Rep. Heath Shuler (D-NC), a retiring member of the centrist, Democratic Blue Dog Caucus.

And because moderate to conservative Democrats like Shuler are stepping down or face challenging re-election prospects, there will be fewer renegades to challenge Pelosi in the 113th Congress.

In fact, Shuler took on Pelosi for Democratic leader after the 2010 midterm bloodletting which propelled the party into the minority. As a result, 19 Democrats objected to Pelosi's leadership, casting their ballots for others. Shuler received 11 votes, including his own. Other Blue Dog defectors included Reps. John Barrow (D-GA), Jason Altmire (D-PA), Sanford Bishop (D-GA), Dan Boren (D-OK), Jim Costa (D-CA), Jim Cooper (D-TN), Joe Donnelly (D-IN), Tim Holden (D-PA), Larry Kissell (D-NC), Mike McIntyre (D-NC), Jim Matheson (D-UT), Michael Michaud (D-ME), Mike Ross (D-AR) and Kurt Schrader (D-OR).

Shuler, Boren and Ross are retiring. Donnelly is retiring from the House to run for the Senate. Barrow, Kissell, McIntyre and Matheson face tough re-election bids. Altmire and Holden both lost their primaries. And this is to say nothing of Blue Dogs who voted for Pelosi who are trying to hang around. Rep. Leonard Boswell (D-IA) is locked in a pitched, member-versus-member contest with Rep. Tom Latham (R-IA) after redistricting. Rep. Ben Chandler (D-KY) won re-election by about 600 votes last time and is in a rematch with GOP challenger Andy Barr. Republicans view Chandler as a seat they "left on the field" after the 2010 shellacking.

In addition, former Blue Dog Reps. Gabrielle Giffords (D-AZ) and Dennis Cardoza (D-CA) retired before the conclusion of this Congress.

In 2010, 23 Blue Dog Democrats lost and five retired. The Blue Dogs had as many as 54 seats two years ago. That figure has dwindled to just 26 and it's expected to shrink next year.

And without Blue Dogs, the House Democratic Caucus will be a lot more liberal.

Blue Dogs like Dan Boren say part of the Democrats' agenda made it tough on conservatives and moderates like himself.

"I voted against health care. I voted against cap and trade. I think if we're going to move forward, we have got to look at doing things that are incrementally bipartisan," Boren said. "I'll say the same thing to the GOP. If they push things way to the right, they're not going to stay in the majority."

Boren touches on a fascinating phenomenon which now dominates House politics. Attrition in the Democratic Caucus has made it more liberal. By the same token, Republicans have grown more conservative. This polarization comes as the country lands somewhere in the middle.

"It's demoralizing to look at what's going on the Hill," said former Rep. Bud Cramer (D-AL). He helped found the Blue Dogs in the early 1990s and retired in early 2009.

"Party bases are rattling cages, looking after members, expecting them to be more ideological. Yet you have a growing middle in the country that doesn't want Congress to conduct business that way," Cramer said.

"I knew it was a big day when left and right-leaning organizations were both picketing my office at the same time," laughed Shuler. "Now, to maintain your seat, you have to be extreme left or right so (you) don't have a primary."

With redistricting, more than 300 of the House's 435 Congressional districts are now designed to tilt heavily in one direction or another. Former House Speaker Dennis Hastert (R-IL) says that contributes to erosion of districts where Blue Dogs can survive.

"Basically Blue Dogs were traditional Democrats that represented areas that were Republican," said Hastert. "There is no middle ground. We're pushing to the extremes in deciding who gets elected."

That's why redistricting has been so onerous for Blue Dogs. Utah gained a Congressional seat after the 2010 Census. The new district boundaries are drawn in such a way that Jim Matheson finds himself in a scrap with Republican challenger Mia Love. She's attempting to become the first African American Republican woman ever elected to the House. Pennsylvania lost one seat after the census. So redistricting forced Altmire to unsuccessfully duke it out with Rep. Mark Critz (D-PA) in a primary. Newly-minted lines resulted in Blue Dog Tim Holden discovering he was essentially drawn out of his district. Republicans controlled the redistricting process in the Keystone State. They recast the moderate Holden into a much more liberal district. Challenger Matt Cartwright defeated Holden in the primary. Redistricting was especially cruel to Blue Dogs in the Tar Heel State. Shuler retires as Blue Dogs Mike McIntyre and Larry Kissell face competitive contests.

Both the Democratic or Republican party labels remain particularly toxic in certain districts. So when it comes to Blue Dogs like McIntyre, Kissell, Ben Chandler or John Barrow, Republicans try to tie them to President Obama or Pelosi as much as they can.

"When I was a Member, I would go home and some would say ‘You're not really a Democrat. I vote for you. But you're not really a Democrat,'" said Cramer. "And Republicans would say ‘For a Democrat, you're not bad.'"

Cramer believes things have changed. Today, just the Democratic party label or a voting record that aligns too closely with the president or Pelosi can contribute to a Blue Dog getting upside down in their district very quickly.

But for Heath Shuler, it's not an issue of just getting elected. He believes the essence of a Blue Dog is thoughtfully weighing the merits and demerits of legislation and perhaps being the determining factor whether a bill passes or fails.

"Without our vote, nothing could ever pass," said Shuler. "That's the best thing that could happen to Congress."

But there simply won't be many Blue Dogs around to serve as "go to" lawmakers on tough votes which require lawmakers who work the center aisle.

At this point, Blue Dogs are fighting for survival. That's partly why so many Blue Dogs voted against Nancy Pelosi for Speaker at the beginning of this Congress. Republicans demonized Pelosi and were ready to catalogue roll call votes where lawmakers supported the California Democrat. The leadership votes opposing Pelosi were a kind of onside kick - a way for Blue Dogs to protest some policy gambits they disagreed with while simultaneously fighting for survival.

"She (Pelosi) was in 2010 a big issue," said Boren.

Shuler may have challenged Pelosi for a leadership slot. But he harbors no ill will.

"She represents a different constituency in San Francisco and we understand she works hard. She is misunderstood in a lot of different ways," said Shuler, a former NFL and collegiate quarterback. "I've been in the same situation before in athletics. A leader is expected to be successful and when you're not it's time for a change."

In a few weeks, Congress will reconvene. Lawmakers will have a little bit of maneuvering room because of the election as they begin to wrestle with the so-called fiscal cliff: expiring tax cuts, an obligatory, indiscriminate set of spending cuts known as the sequester and a pending debt ceiling increase. Bud Cramer says a few years ago, courting the votes of Blue Dog Democrats would be central to solving a crisis like this. That's because there were more Blue Dogs to work with and so many of them were somewhere in the middle.

"I hope part of the message when members come back is look, I can wear a party label but sometimes I need to cross the aisle," said Cramer. "I hope we don't have to fall off the cliff to have members see we can't work the way we've been working lately."

Lawmakers will certainly exhibit those party labels when they return to Washington. They'll even portray themselves as "progressive" or "Christian conservative" or "tea party." But there's one political species that's endangered these days: the Blue Dogs.

- Fox's Mike Emanuel contributed to this report.