Chief Justice William Rehnquist (search) returned to work Friday after defiantly quashing retirement speculation with a pledge to stay as long as his health allows.

The 80-year-old chief justice, who is battling thyroid cancer, looked pale but confident as he left his house in a wheelchair for the trip to his office. He spent two nights in the hospital with a fever earlier in the week, and was discharged on Thursday.

Rehnquist on Thursday emphatically announced his intention not to step down.

Hours after being released from the hospital, Rehnquist, who suffers from thyroid cancer, released a statement saying he plans to stay on the court for as long as he can serve.

"I want to put to rest the speculation and unfounded rumors of my imminent retirement," the statement said. "I am not about to announce my retirement. I will continue to perform my duties as chief justice as long as my health permits."

The White House had not been informed that Rehnquist would be making the announcement prior to the statement. However, White House Deputy Press Secretary Dana Perino said the administration supports his decision.

"The chief justice is doing an outstanding job, and we are pleased he will continue his great service to the Nation."

Rehnquist's announcement is the second surprise to hit the high court in less than two weeks. Justice Sandra Day O'Connor (search) unexpectedly stepped down earlier this month at a time when the White House, the Senate and outside groups had been preparing for the chief justice to leave the court.

Rumors of his departure intensified after Rehnquist was hospitalized. The White House was proceeding with contingency plans to fill two vacancies, a prospect that might have given Bush the political flexibility to please more than one constituency.

Political analyst Dick Morris told FOX News that Rehnquist's decision could hurt conservatives' ambitions to build a court in their image.

"If he has any sense of responsibility to the Republican Party, he has to give George Bush some wiggle room," Morris said.

On Tuesday, the 80-year-old Rehnquist was taken by ambulance to Virginia Hospital Center in Arlington, Va., to have a fever monitored, Supreme Court spokeswoman Kathy Arberg said.

While dozens of news crews were staked out to watch the chief justice leave the hospital, he tried to bypass reporters at home by entering through the back.

Sen. Arlen Specter (search), R-Pa., who is being treated for Hodgkin's Disease , another form of cancer, said Rehnquist's stay at the hospital was not extraordinary.

"I can tell you that when you have cancer they watch your fever very, very closely. And really we have to await developments, there's nothing beyond the admission to the hospital, but I would not think that out of the ordinary, because they take very extensive precautions with cancer patients who have elevated temperatures," Specter said Wednesday.

Dr. Richard Haber, an endocrinologist, told FOX News that a fever is an indication of an infection.

"Because he has a tracheotomy, that is a hole in his windpipe through which he breathes, he is at risk for respiratory infection, including pneumonia, and something like that must have happened," Haber said, adding that thyroid cancer is among the more survivable types of the disease.

"The vast majority of cases can be successfully treated and have an excellent prognosis. It's a minority of cases which are more aggressive and more frightening," he said.

It was the second time in less than four months that Rehnquist was taken by ambulance to the hospital. In March, he was taken in with breathing problems. He did not stay overnight then.

Rehnquist has said next to nothing about his prognosis, leaving medical experts to wonder what type of thyroid cancer he had — the deadly anaplastic form or a more treatable type.

"The prognosis for [anaplastic thyroid cancer] is so poor. Most patients succumb very quickly, within three to six months," said Dr. Mark Urken, a cancer expert at Beth Israel Hospital in New York.

Dr. Kenneth Burman, a thyroid specialist at Washington Hospital Center, said that after nine months doctors should have a good idea about his prognosis, if he has a less serious type like papillary or lymphoma. People with those types can be treated and live for years without more problems.

Rehnquist's health has been carefully watched by reporters tracking his daily moves from his home to his office at the court. When he did not show up as usual Wednesday morning, and news media observed a court police officer making several trips to the house, Arberg was repeatedly questioned until she made a statement.

Rehnquist has served on the high court for 33 years and whether he will retire is a source of frequent speculation in Washington. The latest predictions from oddsmakers are that the chief justice won't be so quick to step down because getting up and going to work each day helps him battle his disease and stay in a healthy frame of mind.

As for O'Connor, 78 percent of respondents said they want a woman to replace her, according to a USA Today poll released Thursday. Sixty-seven percent of the poll's respondent's prefer a Hispanic and 68 percent said they do not want to see Roe v. Wade (search), the controversial 1973 ruling that permits abortion, overturned.

One lawmaker who will be involved in the confirmation process said the polls should not influence the president.

"I don't think any of those are issues under which the president ought to make a choice. He ought to find the most qualified person who has stellar personal attributes, a good mind, and intellectually superb reasoning and ought to understand and read the Constitution and understand it reigns supreme," Sen. Tom Coburn, R-Okla., told FOX News

FOX News' Megyn Kendall and The Associated Press contributed to this report.