Updated

Phil Robertson may never return to A&E’s hit reality show “Duck Dynasty,” and the rest of his family is hesitant to carry on without its patriarch, a source tells FOX411. But the family may not be in a position to take an “all of us or none of us” stand.

"It's not as easy as just not returning to the show, since they all have binding agreements. It's the network's option, not theirs," an insider connected to the production told FOX411. "Plus the family makes far too much money selling merchandise because of the show. It would be a bad business decision to leave it."

According to Beverly Hills-based entertainment attorney Julian Chan, each show participant is generally under a separate deal when it comes to these types of contracts. So each family member could be subject to a breach of contract claim regardless of what happens to anyone else on the show, no matter what his or her role.

We're also told that the contracts have morality clauses that give the network the right to get rid of cast members -- but the others still have to keep the show running.

Multiple industry players assured us that even without Phil Robertson, there is just no way A&E will dump "Duck Dynasty" now. With an average of 14 million viewers each week, counting reruns, it’s the highest-rated show on television.

More On This...

"There is almost no way in the bayou that A&E would dump ‘Duck Dynasty.’ The show is a massive hit for them with 11.8 million viewers tuning in for [the] ‘Duck Dynasty’ Season 4 premiere earlier this year, and 8.9 million watching the Robertson clan's December 11 Christmas Special," said Dominic Patten of Deadline Hollywood. "Season 5 is set to debut on January 15 and you can bet it'll be a record breaker for a record-breaking show."

Deadline.com reported in August that the family was able to secure big salary increases, earning more than $200,000 an episode, divvied up among the nine adults and 11 kids on the show. That is roughly four to five times what the Robertsons previously made. And the deal is said to include additional seasons as well as cast options.

Even if another network is willing to take the show with its original cast—a high possibility —we're told that A&E owns the "Duck Dynasty" intellectual property, so it is in the position to pull all the strings.

Last week, the family released a statement saying that while they have had a "successful working relationship with A&E," they "cannot imagine the show going forward without our patriarch at the helm."

Robertson, who says he is a staunchly devoted Christian, came under enormous fire last week from activist groups after he made very graphic, negative comments about homosexual sex. But his suspension also ignited an outcry over freedom of speech and one's constitutional right to voice his faith and beliefs.

And though the prospect of Robertson being welcomed back into the fold appears grim, some industry experts are saying never say never.

"Often a show will suspend a cast member who does something controversial. This gives them the chance to disassociate themselves from the controversy, see how the wind blows before deciding what to do, then either fire or reinstate the person after the media pressure has passed," Chan said. "We have seen in the past where someone has been rehabilitated or forgiven, and other times where they have been moved out."

A&E declined a request for further comment.

Follow @holliesmckay on Twitter