The brother of the man who reportedly beat up disgraced Subway spokesman Jared Fogle is speaking out.

The alleged attack on Fogle occurred on Jan 29 in the recreation yard at Colorado's Englewood Prison where Fogle is serving time for child pornography posession and paying for sex with minors.

"He had to hit him. That's how the whole thing started," the brother of Fogle's alleged attacker Jimmy Nigg told People. "He just can't be around child molesters. He doesn't like them."

Nigg's brother, who is in prison on weapon charges, reportedly knocked Fogle to the ground and punched him repeatedly.

Nigg said his brother started the fight and subsequently faced 10 days in solitary confinement.

"He can't figure out why you would do that to kids. He says, 'I can't be around those people.' "

A U.S. Bureau of Prisons rep would not confirm the incident "based on the need to ensure safety and security in our institutions and in accordance with legal requirements including the Privacy Act."

Four4Four: Can stars breathe easy after nude pic news?