Less than 24 hours before U.S. Marshals swooped in Monday and arrested escaped Arizona inmate Tracy Province on the steps of a Wyoming church, the convicted killer was singing spirituals about God's grace with its small-town congregation.

Authorities nabbed Province -- armed with a gun and a hitchhiker's sign -- as he walked out of Meeteetse, Wyo., a tiny town of about 350 people just 60 miles southeast of Yellowstone National Park.

Province had been seen at the Meeteetse Community Church Sunday, where he sang along with music including a song titled "You're Grace is Enough," parishioner Jay Curtis told the Associated Press.

"Just shook his hand and said `Welcome, welcome to our church,"' said Curtis. "He just smiled and said: 'Thank you."'

A day later federal officials finally had him in cuffs once again, but a massive manhunt continues for a second escaped inmate and that man's alleged accomplice, an incestuous duo who have blazed a deadly path through the western U.S.

Province separated from fellow inmate John McCluskey and Casslyn Welch, McCluskey's cousin and fiancee -- a woman officials say helped to engineer their July 30 jailbreak.

Authorities are scouring areas around Yellowstone for the pair, focusing their efforts in Wyoming. The inmates have been linked to a double homicide in New Mexico, and authorities have been warning all visitors to the park of the dangers posed by the fugitives.

"They consider themselves as Bonnie and Clyde," U.S. Marshal David Gonzales said. "This is very, very serious business."

Province, who was armed with a 9mm handgun, did not resist arrest, and later told authorities he was "relieved" that the hunt was over after being on the run for more than a week, said Gonzales.

His arrest came about after another resident of Meetseetse met Province on the steps of the community church Sunday and chatted with him before heading home, where she saw news accounts of the Arizona prison break and immediately recognized him as one of the fugitives.

The woman called authorities, who tracked Province down and nabbed him at 6:20 a.m. Monday morning.

"He immediately denied that he was Province, but later admitted that it was him and that he was relieved that this manhunt was over for him," Gonzales told reporters Monday.

Province, McCluskey and Daniel Renwick had escaped from the medium-security Arizona State Prison near Kingman after the 44-year-old Welch threw wire cutters over the perimeter fence, authorities say. Renwick was arrested in Colorado on Aug. 1.

The Arizona Department of Corrections says the three men escaped by cutting a hole in the prison's perimeter fence and later kidnapping two semi-truck drivers at gunpoint and using the big rig to flee. The group left the drivers unharmed in the truck at a stop just off Interstate 40 in Flagstaff and then fled.

Days later, the badly burned skeletal remains of Linda and Gary Haas, both 61, of Tecumseh, Okla., were found in a charred camper on a remote ranch in Santa Rosa in eastern New Mexico.

Province was serving a life sentence for murder and robbery out of Pima County. McCluskey was serving a 15-year prison term for attempted second-degree murder, aggravated assault and discharge of a firearm out of Maricopa County. Renwick had been serving a 22-year sentence for second-degree murder.

Authorities believe McCluskey and Welch have left the sprawling Yellowstone, where more than 100 armed park rangers were patrolling amid an estimated 30,000 campers and tourists.

Authorities say the two are driving a 1997 Nissan Sentra with gray paint and the Arizona license plate 620PFV. A $40,000 reward has been offered for the capture of McCluskey, Gonzales said.

The Associated Press contributed to this report.