David Letterman is getting ready to end his 33-year-run as a late-night TV host, but the comedian told The New York Times there was a time when his days on CBS could have been numbered.

After his 2009 sex scandal broke, when it was revealed that the married host had sexual relationships with several of his staffers, he said the network could have let him go.

"I think [CBS] would have had good reason to fire me," he said.

He added that he was "largely ignorant as to what, really, I had done."

“It just seemed like, O.K., well, here’s somebody who had an intimate relationship with somebody he shouldn’t have had an intimate relationship with. And I always said, ‘Well, who hasn’t?’ to myself. But then, when I was able to see from the epicenter, the ripples, I thought, yeah, they could have fired me. But they didn’t. So I owe them that.”

More On This...

Letterman ended up addressing the affair on the air and delivered an apology to his wife and his staff.

Now, this month, the 68-year-old will sign off of “The Late Show with David Letterman.” His last day at the franchise will be May 20 and Stephen Colbert will take over his hosting duties.

Colbert will appear as the “Late Show” host beginning Sept. 8, 2015.