The haters are gonna hate.

But "19 Kids & Counting's" Jim Duggar says he doesn't care what his critics think.

When an online petition that urged TLC to cancel their hit show received more than 180,000 signatures last month, Duggar said he wasn't worried.

"They won't succeed," he said at the Booster Banquet at Hannibal-LaGrange University. "Our show is the No. 1 show on TLC. It's a small group creating this fuss."

In fact, thanks to the attention due to the petition, the Duggar family's Facebook paged gained 50,000 followers, he said.

More On This...

"All it has done is give us more exposure."

The Change.org petition was in response to the reality show's matriarch Michelle Dugger being featured in a robocall in Fayettville, Ark., urging residents to oppose an anti-discrimination ordinance. The ordinance, Michelle Duggar said in her recording, would "allow men to use women's and girls' restrooms, locker rooms, showers, sleeping areas and other areas that are designated for females only."

The Duggar's fans came to their support with a counter-petition asking TLC to save the show.

"Sign our petition today to let TLC know that you support the Duggars' stand for traditional marriage," it stated.

"We love everybody," responded Duggar. "But we don't always agree with them."

WATCH: Four4Four: 'The Interview' terror threat shocker