Convicted Ponzi schemer Bernie Madoff is dying of cancer, sources told the New York Post.

Madoff, who is serving 150 years at a North Carolina federal prison after pleading guilty to swindling more than $65 billion, has been telling fellow inmate he doesn't have much longer to live.

"He's been taking about 20 pills a day for his cancer," one inmate told the newspaper. "He talks about it all the time. He's not doing very well."

Speculation has swirled as to why Madoff, 71, took sole responsibility for the scheme and rumors of his suffering from pancreatic cancer surfaced months before he arrived at the Butner, N.C., complex in June.

Madoff's attorney did not return messages yesterday but, when previously asked by the Post, would neither confirm nor deny his client has cancer. A lawyer for his wife, Ruth Madoff, also did not return messages.

Click here to read more on this story from the NYPost.com.