House Republicans have picked Nov. 5 as the date to override President Barack Obama’s veto ofthe annual defense budget bill, but Democrats think they can holdthe line.

With the original House vote clocking in at 270-156, Republicansonly need 20 more lawmakers to defect in order to successfullyfight Obama’s veto, The Hill reports. This isthe fifth time Obama has actually used his veto power, instead ofjust threatening to use it.

The problem with that assumption is that in the face of a veto,Democrats who first supported the defense budget may jump ship toavoid embarrassing the president.

Rep. Michael Turner led the charge from the Republican side,saying in a letter on behalf of 102 House Republicans thatthey would not accept any proposal to decrease basefunding for the Pentagon below $561 billion.

According to a report conducted by the Congressional ResearchService, Congress does not have a successful track record infighting off defense budget vetoes. In the last four times theWhite House has sent the legislation back, Congress caved, gave inand changed the offending provisions. (RELATED: Report: Congress Has Removed Defense Bill Provisions FourTimes To Avoid Veto)

In this case, the White House has complained about a provisionthat adds additional restrictions on transferringprisoners out of Guantanamo Bay. The president has spent the lastyear desperately trying to make good on his campaign promise in2008 to shutter the detention facility, despite congressionalopposition. That opposition is especially fierce because of thepossibility that hardened detainees might end up on U.S. soil. Inother words, the Obama administration doesn’tbelieve that all detainees should be released. Some need to staylocked up indefinitely. But if Gitmo closes, they need a place togo.

As reports emerge of the Pentagon sending out scouting teams todetermine the suitability of facilities for dangerous detainees,congressmen in those districts continue to come out hard againstthe proposals. Some of the facilities under question include FortLeavenworth in Kansas and the Naval Brig in Charleston, S.C., aswell as the Colorado State Penitentiary by Canon City.(RELATED: GOP Reps Promise To Do Everything In Their Power To BlockGitmo Transfers To Colorado)

But GOP Sen. John McCain , who has sided with the Obamaadministration on the status of Guantanamo Bay, blamed the WhiteHouse for refusing to submit a draft closure plan for considerationto Congress.  (RELATED:McCain Blames Obama Administration For Not Submitting Gitmo ClosurePlan In Time)

The White House has also complained about the use of OverseasContingency Operations, an emergency war fund, to evade budgetcaps.

The deadline for Congress and the White House to come to abudget deal is Dec. 11. The options are: come to a compromise,allow a short-term funding measure, or neither, which would resultin a government shutdown.

Follow JonahBennett on Twitter