Citing a lack of time, Focus on the Family founder James Dobson withdrew Tuesday from the team overseeing counseling for the Rev. Ted Haggard, the evangelical pastor who was fired amid allegations of gay sex and drug use.

"Emotionally and spiritually, I wanted to be of help -- but the reality is I don't have the time to devote to such a critical responsibility," Dobson said.

The other two members of the team, Pastors Jack Hayford of The Church on the Way in Van Nuys, Calif., and Tommy Barnett of First Assembly of God in Phoenix, declined to comment.

Haggard was forced out as senior pastor of the 14,000-member New Life Church on Saturday after a former male escort alleged they had sex repeatedly and that Haggard used methamphetamines.

In a statement read at the church Sunday, Haggard confessed to unspecified "sexual immorality," accepted responsibility for his actions and asked forgiveness.

The counseling process, called restoration, could take years, said H.B. London, vice president for church and clergy at Focus on the Family.