Since disgraced comic Bill Cosby started his prison sentence a little over four months ago, his family hasn't visited him.
Per the news outlet, the prison is 20 miles from the family's estate in the Philadelphia suburbs.
"He doesn't want to have them in that environment," Wyatt, who said he visits Cosby regularly, told AP. "Why put them in that position, to make it turn into some form of a circus?"
A rep for Cosby did not immediately respond to Fox News' request for comment.
On Thursday, it was reported that Cosby was moved to the general population last week after spending four months in special housing as he acclimated to the prison. He now has a single cell in a two-story unit at the newly built prison in Montgomery County.
A jury last year convicted Cosby of drugging and sexually assaulting a woman at his home in 2004. On Sept. 25, Montgomery County Judge Steven T. O'Neill sentenced Cosby to three to 10 years in state prison. He is appealing the conviction.
Camille — she's Cosby's wife of more than 50 years — made just one brief appearance at each of her husband's two criminal trials, and their three surviving daughters stayed away. Camille did, however, file an ethics complaint against the trial judge last year, accusing him of bias in the case.
Wyatt told AP that Camille continues to strategize on her husband's behalf behind the scenes.
Cosby, a Philadelphia native, rose to fame in the 1960s as the first black actor to star in a prime-time television drama with the hit show, "I Spy." He became known as "America's Dad" for his portrayal of family man Cliff Huxtable on the top-rated "Cosby Show" from 1984 to 1992.
Dozens of women have accused him of sexual assault or misconduct over a 50-year span, including five who testified at his retrial last year. Cosby and his lawyers and agents have repeatedly called the encounters consensual.
The Associated Press contributed to this report.