The UN refugee agency says it is alarmed by a mob attack on Rohingya Muslim refugees in Sri Lanka as senior government officials called for stern legal action against the perpetrators that included Buddhist monks.

The reaction of the United Nations High Commissioner for Refugees came a day after a group led by Buddhist monks stormed a United Nations-run safe house for 31 Rohingya Muslims, claiming that they were terrorists and demanding they be sent back to Myanmar, prompting police to relocate them.

Dozens of protesters from Sri Lanka's majority Buddhist community led a mob Tuesday that entered a multi-storied house at Mount Lavinia on the outskirts of the Sri Lankan capital.

Finance and media minister Mangala Samaraweera on Wednesday condemned the attack and called for strong action against perpetrators.