The 36-year-old man identified only as Bruce told KGO-TV that he was waiting to cross the street in the city's Mission Bay neighborhood in front of Gus's Community Market when he was punched and knocked to the ground.
"I was right on the ground and in that exact second I was trying to shield my head and prevent any worse injuries," he said. "I couldn't protect my child. I was on the floor and he was in a stroller that was slowly rolling away, so it's definitely very scary as a parent."
The alleged attacker, identified by police as Sidney Hammond, 26, pummeled the man while on the ground before officers patroling nearby intervened. The attack did not appear to be racially motivated, police said.
"It was then that officers pulled Hammond off the victim," the San Francisco Police Department told Fox News. "The attack appeared to be random, and no words were exchanged during the incident."
Hammond was arrested and is charged with assault likely to cause great bodily injury and child endangerment. He is being held in the San Francisco County jail.
The victim was treated at the scene for non-life threatening injuries, police said. In response to the arrest, Bruce said he hopes Hammond "can be somewhere where he can't harm anyone else or if he needs help he can get the help he needs."
Hammond was arrested last month for an alleged assault and stealing around the same location, the news outlet reported.
The incident came amid a wave of violent attacks on people of Asian descent. An Asian woman was attacked by a woman with a hammer in New York City over the weekend and two Asian women were recently beaten in Baltimore with a cinder block.