No one serious injuries were reported. City health officials aren't recommending rabies shots, but victims were advised by the Marine Mammal Center to take antibiotics to ward off infection.
Experts say the rogue sea lion could be protecting his harem of mates or might have brain damage from toxic algae.
Celeste McMullin, who was bitten Monday, said she saw the animal lurking nearby before her swim but didn't think much of it.
"I was swimming along, and I felt a brush under my feet. And I thought, 'These feel like whiskers.' So I stopped, and the animal popped up. He/she looked at me."
McMullin then tried to swim away, but the sea lion followed, biting and bumping her continuously until she made it back to shore. She ended up with six bites: two puncture wounds and four cuts.
Marine Mammal Center veterinarian Frances Gulland said the animal may soon leave the area and advised swimmers to avoid the lagoon in the meantime.
"The migration has started, and the animals are moving north to Washington state and Oregon," she said.