James Clapper, the top U.S. spy, described on Sunday his secret mission recently to bring two American captives back from North Korea, the isolated, communist country hoping the deal would lead to a diplomatic “breakthrough.”

The Director of National Intelligence said Sunday on CBS’s “Face the Nation” that North Korea feels “under siege” by much of the rest of the world.

Clapper also said that he thought at one point the mission to return Americas Kenneth Bae and Matthew Miller might fail because North Korean officials wanted a diplomatic concession in exchange, and he had none to offer.

"I think they were disappointed," Clapper said, a week after the trip was completed.

Only when Clapper was ushered into a hotel room for an "amnesty-granting ceremony" did he think the releases would proceed as planned.

Clapper described the beginning of the trip as arrived in Pyongyang in the dark, being taken to a guest house and being met by a small party led by the state security minister and a translator.

What followed, he said, was a "terse" and “not exactly pleasant” dinner with the head of the Reconnaissance Guidance Bureau.

He brought a short letter from President Obama characterizing North Korea's willingness to release the pair as a positive gesture. But the North Koreans wanted more.

"I think the major message from them was their disappointment that there wasn't some offer or some big -- again, they term they used was `breakthrough.’ ”

Afterward, Clapper waited hours until he got word that he had 20 minutes to pack his luggage for a drive to a downtown hotel. It was then he knew he would be leaving with Bae and Miller.

At a ceremony, Clapper exchanged handshakes with his North Korean interlocutor, the prisoners changed clothes and they left for the flight back.

The U.S. and North Korea have no formal diplomatic relations and a legacy of mutual hostility. Clapper sensed a "ray of optimism" about the future from his brief encounter with a younger generation -- specifically, an official in his 40s who accompanied him to the airport and "professed interest in more dialogue, asked me if I'd be willing to come back to Pyongyang. Which I would."

Clapper said visiting North Korea has "always been on my professional bucket list."

Bae was detained in 2012 while leading a tour group to a North Korean economic zone. Miller was jailed on espionage charges after he allegedly ripped up his tourist visa at Pyongyang's airport in April and demanded asylum. They were the last two Americans held captive by North Korea.

Clapper also said the trip unfolded more smoothly than his first into North Korean air space, aboard a U.S. helicopter in December 1985.

"They shot at us, and fortunately we made it back to the South," he said.

At the time, Clapper, a retired Air Force lieutenant general, was an intelligence chief for U.S. forces in South Korea. This time, he was a presidential emissary with a deal in the works and permission to land.

The Associated Press contributed to this report.