Ben Carson on Sunday defended giving paid speeches while being a 2016 Republican presidential candidate, saying they were prior commitments and that canceling would be unfair to the event organizers.

Carson told ABC’s “This Week” that he continues to give the speeches despite political advisers asking him to cancel and that he has only about four remaining events.

“People have gone through a lot of trouble getting sponsors and selling tables,” said Carson, a retired pediatric neurosurgeon. “You don't just walk out on them. I think that is horrible."

Carson, popular with the GOP’s social conservative and Tea Party wing, rose to political prominence after the 2013 National Prayer Breakfast speech in which he criticized President Obama’s policies while on the same stage.

Giving paid speeches while a candidate and touting one’s platform on the campaign trail is not illegal, but unusual.

Carson’s team told The National Review earlier this spring that the candidate was under contractual obligations for the speeches and that a third party, the Washington Speakers Bureau, is involved.

How much Carson is getting paid for the speeches is unclear.

Carson, a first-time political candidate, entered the 2016 race with little name recognition beyond conservative circles but appears to be rising in polls, amid a GOP field of more than a dozen candidates and potential candidates.

A recent Quinnipiac University poll shows him among the top five contenders, all with 10 percent of the vote. The other four are former Florida Gov. Jeb Bush, former Arkansas Gov. Mike Huckabee, Florida Sen. Marco Rubio and Wisconsin Gov. Scott Walker.