Card, who served in the White House under former President George W. Bush during some of the administration's toughest moments including the decision to go to war in Iraq, said in a statement that the decision is in the best interest of his family. His decision was announced just hours after Democratic Rep. Edward Markey said he also won't jump into the race to succeed Kennedy.
Card was the highest-profile Massachusetts Republican to consider jumping into the special election. A second potential GOP candidate, state Sen. Scott Brown, had said he'd drop his campaign if Card jumped in.
"Now is not the right time for me to enter a political race," Card said in a statement. "In these critical times, I know that Massachusetts would be well served to have Senator Scott Brown as the Commonwealth's next United States Senator. He has my full support."
Brown scheduled a news conference for Saturday afternoon.
Card, 62, is perhaps best remembered to most Americans as the White House aide who whispered in Bush's ear on Sept. 11, 2001 that a second plane had struck the World Trade Center in New York and that America was under attack.
Although he's tried to distance himself somewhat from Bush, a Card candidacy could have introduced the former president's legacy into the Senate campaign, including the decision to invade Iraq.
Kennedy derided that decision as a "fraud made up in Texas" and repeatedly called his vote against the war the proudest moment of his Senate career.
Card had even conceded that his time in the White House would both help and hurt him with Massachusetts voters.
The Holbrook, Mass., native served as state representative from 1975 to 1982 and in 1980, he helped run the campaign of Bush's father, George H.W. Bush, in Massachusetts for the Republican presidential nomination. He has maintained close ties with the family since.
After launching an unsuccessful bid for governor in 1982, Card went to work in the Reagan White House as a liaison to the nation's governors and eventually worked as deputy chief of staff and Transportation Secretary under the first President Bush.
After the elder Bush lost his re-election bid, Card left for the private sector, landing a job as director of government affairs for General Motors, before returning to the White House during the second Bush administration.
Card said he always intended to run again for office in Massachusetts, but had a hard time turning down presidential job offers.
"I happen to believe that if a president asks you to help, you should try to find a way to say yes, and I did," he said.
Card, who stepped down as chief of staff in March, 2006, would have had to work quickly to raise the money needed to launch a viable campaign. Although he's not personally wealthy, Card could have tapped into national networks of Republican donors.
He estimated that he might need to raise between $4 million and $6 million. "I'm quite confident that if I were to run, I would be able to raise enough money to be competitive," he said.
Card and his wife, an ordained Methodist minister, live in Virginia. The couple also owns a house in Holbrook, and Card said he maintains deep ties to Massachusetts.
Attorney General Martha Coakley was the first Democrat to declare her candidacy. U.S. Reps.
Michael Capuano, Stephen Lynch and John Tierney also have said they're considering running.
On the Republican side, Bob Burr, a selectman from suburban Canton, also says he will seek his party's nomination.
Kennedy died last month of brain cancer at age 77.