Taliban militants began pulling out of a recently seized district of northwestern Pakistan on Friday, a local official and the insurgents said, after the government warned it would remove them by force.

The withdrawal, if completed, eliminates the most immediate threat to a peace agreement in the militant-held Swat Valley that the U.S. government worries has created a haven for allies of Al Qaeda.

Television pictures showed dozens of militants emerging from a high-walled villa that served as their headquarters in Buner, a rural area just 60 miles from the capital.

The men, most of them masked with black scarves and carrying automatic weapons and rocket-propelled grenades, clambered into several pickup trucks and minibuses before driving away.

Syed Mohammed Javed, the top government administrator in the region, said a hardline cleric who helped mediate the disputed peace deal persuaded the Taliban to return to Swat in a meeting on Friday.

"We told them that we have a deal, we have a peace agreement. We told them not to become a tool in the hands of someone aiming at sabotaging the peace in the region," Javed told The Associated Press by telephone from Buner.

Javed said he and the cleric, Sufi Muhammad, were leading the Taliban convoy back to Mingora, Swat's main town.

The government agreed in February to impose Islamic law in Swat and surrounding areas of the northwest in return for a cease-fire that halted nearly two years of bloody fighting between militants and Pakistani security forces.

But hard-liners have seized on the concession to demand Islamic law, or Sharia, across the country, and the Swat Taliban have used it to justify a push into Buner, putting them within striking distance of the capital and key roads leading to the main northwestern city of Peshawar.