America's first family is heading off for a 10-day vacation on the exclusive island of Martha's Vineyard, but while the White House says the president will "recharge his batteries" during his time off, it is not lost on those in the administration that a "vacation" is not fun and games for any president - especially given the history of news breaking during down time.

"The president -- any presidential vacation -- whenever you talk about a presidential vacation you ought to put the word "vacation" in quotes because you can bet that there will still be work that he's doing every day," Deputy White House Spokesman Bill Burton told reporters on Air Force One Wednesday. "He'll continue to get his daily intelligence brief from John Brennan [Assistant to the President for Homeland Security and Counterterrorism] who will be there. He'll be getting constant updates on what's happening in the economy and other issues."

In addition to Brennan, Burton said National Security Chief of Staff Denis McDonough, senior advisers Valerie Jarrett and Peter Rouse will also head up to New England for some part of the Obama's vacation. If the staff situation looks beefed up, it's not an illusion. President Obama will be facing what almost all of his predecessors have: how to balance vacation with the daily rigors of running the country.

"The responsibility of the office follows the president wherever he may be, whether in Texas or Maine, or a vacation spot of his choosing," Scott Stanzel, former Deputy White House Spokesman for President George W. Bush told Fox News.

And on Christmas Day of 2009, the responsibility of the office followed President Obama and his family to a two week Hawaii vacation. On December 25, Umar Farouk Abdulmutallab, attempted to light a bomb in his underwear on fire, a situation that resulted in a complete overhaul of the TSA and security systems in the United States. At the time, Brennan was in Washington and briefed the president by phone about the investigation. McDonough, who was in Hawaii, ended up spending a fair amount of time briefing the traveling White House press corps.

While the White House is hoping for a quiet time in Massachusetts, those who know say nothing is ever really quiet. "President's and first ladies need to be able to take a little bit of time off. But, it's never really fully a vacation, especially for the president," Former National Security Spokesman Gordon Johndroe told Fox News. "For the President there's lots of staff, military aides, communications aides...the press is always there. And of course, plenty of secret service. So, it's a long day for presidents anyway I think, wherever they are."

And presidential historian Doug Wead agrees. "The work of the president is making decisions and that work is on-going. It doesn't matter whether they're on the golf course." Wead also adds, it's not a good idea to take stock of a president based on how many days off they take. "Jimmy Carter had the lowest number of days off and the highest inflation."