Episcopal Bishop Robert Duncan of Pittsburgh, whose diocese is moving toward splitting from the national church, was ousted from ministry Thursday by his fellow bishops.

The House of Bishops voted 88-35, with four abstentions, to remove Duncan on a charge of "abandonment of the communion of this church."

Duncan, who led the Pittsburgh diocese for 11 years, is a leader in a national network of theological conservatives who are breaking away from the liberal denomination in a dispute over Scripture. The long-simmering debate erupted in 2003, when Episcopalians consecrated the first openly gay bishop, V. Gene Robinson of New Hampshire.

The Pittsburgh diocese said in a statement it will move ahead with the Oct. 4 secession vote despite Duncan's removal. If the diocese decides to split off, it will align with the like-minded Anglican Province of the Southern Cone in South America, which has already taken in Duncan as a bishop. The Episcopal Church is the U.S. branch of the worldwide Anglican Communion, a 77 million-member fellowship of churches that trace their roots to the Church of England.

The Rev. David Wilson, president of the committee that oversees the Pittsburgh diocese, called Duncan's ouster "a very painful moment."

"The leadership of The Episcopal Church has inserted itself in a most violent manner into the affairs and governance of our diocese," Wilson said. "We will stand firm against any further attempts by those outside our boundaries to intimidate us."

But Progressive Episcopalians of Pittsburgh, which represents parishioners fighting to stay with the national denomination, said Duncan "has rejected numerous opportunities and warnings to reconsider and change course."

While the 2.2 million Episcopalians vary on their views of the Bible and gay relationships, the overwhelming majority did not consider Robinson's consecration cause to leave the church.

Still, national Episcopal leaders are facing lengthy and expensive legal battles with conservatives who want to leave and take their property with them.

The first Episcopal diocese to split off was San Joaquin, based in Fresno, Calif., in a 2006 vote that also aligned the diocese with the Anglican Province of the Southern Cone. Soon after, a bishop loyal to the U.S. church was elected to head the diocese, while the national church entered a legal fight for control of the diocese and its multimillion-dollar assets.

A third conservative diocese, in Fort Worth, Texas, is set to vote in November on whether it should secede.