Julie Chen revealed a family secret on "The Talk" that has haunted her for years.

VIDEO: Top Celeb Scandals of Summer

"My grandfather was a polygamist," Chen said on Tuesday's show. "He had nine wives, six of who bore him children. He had 11 children that we know of and countless mistresses."

This secret is one that was even kept from Chen for a time, as her mom feared that it could have negative effects on Chen's career and reputation.

Chen's uncle was also a polygamist, who had 20 children by four wives.

More On This...

"Had he not died in a freak car accident when he was 42, he probably would have had more wives and more than 20 children," said Chen.

VIDEO: Julie Chen Takes You Inside the 'Big Brother' House

Chen never got to meet her grandfather, as he died before she was born. Still, his life choices seemed to have lasting effects on the family.

"As you can imagine this was very hard on my grandmother," said Chen, who revealed that her grandfather's wives lived in separate residences and her mother never knew what home her father would be spending the night at.

Chen went on to reveal that her grandfather apologized to her mother on his deathbed. Chen's parents are approaching 52 years of marriage this year.

Chen herself is coming up on 10 years of marriage to President and CEO of CBS Corporation Leslie Moonves. The couple wed on December 23, 2004 and have one child together -- four-year-old son Charlie.

WATCH: Break Time: The next Anna Nicole