California beaches are public by law. But in Malibu, mansions of the rich and famous create an unbroken chain of high walls and locked gates, making it nearly impossible for other people to actually use the beach.

"I think the homeowners are being very selfish," beachgoer Julie Milton said.

"There is one access here – then there is nothing for five miles," said Steve Hoye, of the group Access for All.

Homeowners made a deal with the state years ago where they could turn beach cottages into multimillion-dollar mansions – provided they give the state a five- to nine-foot easement or right of way for public access.

But that never happened. Homeowners built fences and cement walls, effectively turning public beaches into private ones.

Now the California Coastal Commission is demanding public-beach access near those homes, which is essentially "taking" some of their property.

Homeowner Mary Ann Kearns said that would ruin the beaches.

"It is ridiculous. They leave their pampers, beer cans, boom-boxes. It’s filthy," she said. "There are miles and miles of public beach in Malibu. Why can't they go where there is water, toilets and lifeguards?"

Some access points have no toilets and only one parking space.

Jody Siegler, president of the La Costa Beach Homeowners Association, said homeowners also face potential legal problems.

"It's not because of the actions of homeowners," she said. "It’s because of liability. How are we going to manage these access points?"

But Hoye rejected that excuse.

"That's not what it's about," he said. "It's about privilege."

Some say it's also about privacy for Hollywood bigwigs like Ryan O'Neal and Julie Andrews. Neighbors agreed to an easement, then fenced it off. Dreamworks record mogul David Geffen agreed to this easement but locked the gate.

Depending on how the conflict plays out, the beachbound may eventually be able to hit the sand without having to detour around the homes of the stars. But until then, the best view they’ll get of some Malibu beaches will be through a gate.