Retiring House Speaker John Boehner tearfully reflected on his more than two decades of experience in Washington Thursday morning as he bid farewell to his colleagues.

“I leave with no regrets, no burdens,” he said as he prepared to hand over the gavel to his successor. “I leave the way I started – a regular guy humbled to do a big job.”

Boehner, who described his life as a “chase for the American dream,” told colleagues during his roughly 10-minute speech to remain optimistic. 

“Hope always springs eternal – if you do the right things for the right reasons – good things will happen,” he said, wiping away tears.

Earlier, Boehner, reflecting on a nearly quarter-century run on Capitol Hill, defended his decision to push a bipartisan budget deal through the House as his final act before leaving Congress.

"It's a solid agreement and I'm proud of it," Boehner told a group of reporters in his office shortly before the House approved the deal that raises the government's borrowing limit through early 2017 and sets spending levels for two years -- taking both those thorny issues off the table for Speaker-nominee Rep. Paul Ryan.

"It was something that in my view had to be done," Boehner said. "I didn't think leaving this for the next speaker was at all fair."

Boehner said he's proud of the work he's done as a member of Congress from Ohio's 8th District, and as speaker of the U.S. House and second in line to the presidency for nearly five years. He said he's having no second thoughts about his decision to resign under conservative pressure, leaving Congress before the end of his term.

Indeed he said he was starting to get concerned in recent weeks after Majority Leader Kevin McCarthy abruptly dropped out of the race to replace him and some lawmakers started talking about drafting him to remain in office. He said around 50 members were planning to march over to his office to demand that he stay -- and even release a letter saying as much.

"That would've been a real nightmare," a relaxed and jacketless Boehner remarked before his speech, smoking a Camel. One of his lieutenants talked the band down, he added.

But Ryan agreed to run, and Boehner said that the GOP's 2012 vice presidential nominee, endorsed by House Republicans on Wednesday for speaker and set to be elected on the House floor Thursday, will be a solid successor.

"He's an innovative thinker who's focused on giving more Americans more opportunity to achieve the American dream and I think he's got the skill set to do this job," Boehner said.

Boehner said he'd given Ryan lots of advice. Chief among it: "This is the loneliest place in the world," Boehner said. "Almost as lonely as the presidency."

Boehner's biggest regret, he said, was not finalizing a major budget deal with President Obama in 2011 that would put the country on a much sounder fiscal footing.

That failure "still stings," said Boehner. But he demurred on getting drawn into why it was tough to work for Obama, saying he'd save such reflections for a book.

Boehner said he wasn't aware of anything he could have done differently to deal with the House Freedom Caucus, the group of hard-line conservatives who pushed him toward the exits by threatening a floor vote on his speakership after complaining of his penchant for compromise.

Yet he expressed some bemusement as to why the Republican Party has splintered as it has, with some hardliners now contemptuous of those like him with a business-friendly leaning. "Well, I don't know," said Boehner, who started out as a rebel himself when he first came to the House.

The 65-year-old has worked every day of his life since he had a paper route at age 8, and claimed to have no idea what he'll do next. Golf and time with his first grandchild -- who he joked will address him as "Mr. Speaker" -- will play a big part.

"It's the Congress," Boehner said. "We've been America's favorite whipping boy for 200 years. And guess what. Two hundred years from now they're going to be saying the same thing."

The Associated Press contributed to this report.