The old adage says an Englishman's home is his castle. But a court says Robert Fidler's castle can't be his home.

A judge ruled Monday the 66-year-old farmer will go to prison if he doesn't demolish a mock-Tudor castle he built without planning permission.

Fidler built the property south of London in 2000, keeping it hidden behind straw bales. Local authorities ordered him to tear it down in 2007, saying it breached planning rules designed to protect the countryside, and Fidler lost several legal challenges against the order.

High Court judge Ian Dove said Fidler would be jailed for three months for "intentional defiance" unless he complied by June 6, 2016.

Fidler, who claimed the local council was out to destroy his life, left court without speaking after the ruling.