South Carolina Rep. Jim Clybrun said Tuesday that he takes issue with comments by Bill Clinton in which the former president seemed to suggest Clyburn undercut Clinton's reputation with black voters.

The No. 3 Democrat in the House, Clyburn said his heart has been with Barack Obama, but his head had been supportive of Hillary Clinton despite suggestions otherwise.

Clinton told ABC News last week that Clyburn "used to be" an old friend of his, but he "was not Hillary's supporter. Never. Not ever. Not for a day."

When told that Clyburn had said Clinton damaged his own credibility with the black community, Clinton responded, "That may be by the time he got through working on it, that was probably true."

Cilnton "is not correct in his conclusions," Clyburn told FOX News.

As for whether Clinton thinks Clyburn undermined him with black voters, the congressman said, "That's easily to be understood from his comments, and I just beg to differ with that. Because the fact of the matter is all the stuff that I saw reported were reports on things the president said from his own mouth."

Clyburn specifically pointed to Clinton's comparison of Obama's primary win in South Carolina to Jesse Jackson's 20 years earlier. Jackson won the black vote and not much else on the way to losing the Democratic nomination to former Massachusetts Gov. Michael Dukakis.

"Most people thought that the most telling thing back in January was the equation that the president made of Jesse Jackson having won South Carolina caucuses 20 years earlier, and compared that with Obama winning the South Carolina primary. There's a big difference in a caucus and a primary," Clyburn said. " And so a lot of that, irrespective of what the president may have meant by the statement, a lot of people interpreted that as having a racial connotation, and Jim Clyburn didn't speak on that issue at all."

As for whether Clinton did hurt himself among black voters, Clyburn said, "I don't know that I've done any surveys to determine whether or not the president, former president has ever damaged himself or not."

What does his gut tell him? "My gut tells me that some things I ought to keep to myself," he said.

FOX News' James Rosen contributed to this report.