The government watchdog President Obama canned for allegedly being "confused" and "disoriented" fired back sharply Wednesday, saying the White House explanation for removing him was "insufficient," "baseless" and "absolutely wild." 

Gerald Walpin, who until last week was the inspector general for the Corporation for National and Community Service, told FOXNews.com that part of Obama's explanation was a "total lie" and that he feels he's got a target on his back for political reasons. 

"I am now the target of the most powerful man in this country, with an army of aides whose major responsibility today seems to be to attack me and get rid of me," Walpin said.  

Facing bipartisan criticism for the firing, Obama sought to allay congressional concerns with a letter to Senate leaders Tuesday evening explaining his decision. In the letter, White House Special Counsel Norman Eisen wrote that Walpin was "confused" and "disoriented" at a May board meeting, was "unduly disruptive," and exhibited a "lack of candor" in providing information to decision makers. 

"That's a total lie," Walpin said of the latter charge. And he said the accusation that he was dazed and confused at one meeting out of many was not only false, but poor rationale for his ouster. 

"It appears to suggest that I was removed because I was disabled -- based on one occasion out of hundreds," he said, adding that the administration is grasping at "non-existent straws" to explain its actions. 

"I would never say President Obama doesn't have the capacity to continue to serve because of his (statement) that there are 56 states," Walpin said, adding that the same holds for Vice President Biden and his "many express confusions that have been highlighted by the media." Obama mistakenly said once on the campaign trail that he had traveled to 57 states.

White House Press Secretary Robert Gibbs said Wednesday that the administration stands by the letter, and reiterated that board members shared the concerns expressed in it. 

Sen. Claire McCaskill, D-Mo., who had raised questions about the firing Tuesday, released a statement Wednesday in light of the letter saying the president's reasons are "substantial" and the decision to remove Walpin "appears well-founded." She said the letter puts the White House in "full compliance" with the law, which requires the president to provide an explanation before firing an inspector general. 

Walpin, though, concluded that his firing stems from bad blood between him and the board, as well as with Sacramento Mayor Kevin Johnson -- an Obama supporter whom he had investigated for alleged misuse of federal funds. He said his performance at the May meeting drew criticism because he issued two reports critical of the board. In one, he criticized the settlement reached in the Johnson case; in the other, he criticized the use of millions of dollars for a program at the City University of New York. 

"The board at that meeting was clearly angry at my temerity," he said. 

The White House, in its rationale for giving Walpin the boot, also complained that Walpin was "absent" from the corporation's headquarters, "insisting" on working from home in New York over the "objections" of the board. 

Walpin, though, said he reached an agreement with the agency early this year that would allow him to work from home. The former inspector general, who was appointed by George W. Bush, said he originally was going to resign before Obama took office because his wife of 52 years was not happy with their "commuting marriage" -- he was commuting weekly from New York to Washington. He notified Bush of his intention to leave, but said his staff convinced him to reconsider. 

In the end, Walpin said he worked out an agreement with corporation leaders under which he would travel to Washington two or more times a week, and spend the rest of the time working from home in New York. He said some board members had initial reservations, but they were resolved. 

"I never had a single objection" before reading Tuesday's letter from the White House, he said.