Rear Adm. Ronny Jackson will not return as President Trump's personal physician after withdrawing his nomination to run the Department of Veterans Affairs, Fox News has learned.

It was not immediately clear whether Jackson would stay with the White House Medical Unit. He already had been replaced as Trump's personal physician when the president nominated him to lead the VA last month.

Officially, the White House said Monday there are no "personnel announcements."

“Rear Admiral Ronny Jackson is currently on active duty, assigned to the White House as Deputy Assistant to the President. Despite published reports, there are no personnel announcements at this time,” Raj Shah, White House deputy press secretary, said in a statement.

The 50-year-old Jackson had been dogged by allegations about his prescribing practices and leadership ability, including accusations of drunkenness on the job and that Jackson "got drunk and wrecked a government vehicle" at a Secret Service going-away party, which Jackson denied.

The top Democrat on the Senate Veterans' Affairs Committee, John Tester of Montana, said more than 20 military and retired military personnel who've worked with Jackson brought the allegations to his attention. Still, documents showed then-President Obama strongly praised Jackson for his work, recommending promotions.

Jackson formally withdrew his nomination Thursday. The next day, the White House said internal records showed Jackson was in three minor incidents in government vehicles during the past five years, but none involved the use of alcohol and he was not found to be at fault. In one case, a side-view mirror was clipped by a passing truck. In another incident an enraged driver in Montgomery County, Md., allegedly punched out Jackson's window during a morning drive to Camp David.

The White House medical unit that Jackson ran successfully passed regular controlled substance audits, according to the records for the last three years. The reviews did recommend improvements to the medical unit's handling of controlled substances, but did not find misconduct.

Separately, the Secret Service said it has no evidence to support an allegation that its personnel intervened to prevent Jackson from disturbing former President Barack Obama during a foreign trip in 2015.

Jackson has denied the accusations, calling them "baseless and anonymous attacks" on his character and integrity that are "completely false and fabricated."

On Saturday, Trump called on Tester to resign for making what Trump called "false" allegations about Jackson public.

Trump tweeted early Saturday that the allegations "are proving false. The Secret Service is unable to confirm (in fact they deny) any of the phony Democrat charges which have absolutely devastated the wonderful Jackson family. Tester should resign."

Trump said people in Montana, a state he won by 20 percentage points in 2016, "will not stand for this kind of slander." He called Jackson "the kind of man that those in Montana would most respect and admire, and now, for no reason whatsoever, his reputation has been shattered. Not fair, Tester!"

"What Tester did to [Jackson] is a disgrace," Trump told a rally audience in Michigan Saturday night. Trump later said: "I know things about Tester that I could say, too. And if I said 'em, he'd never be elected again." He didn't elaborate.

Tester has said that not investigating the claims would have been "a dereliction of duty" and said making them public was important for the sake of transparency.

Fox News' John Roberts and Kristin Brown and The Associated Press contributed to this report.