"No borders! No nations! Abolish deportations!" the crowd chanted at one point, according to video footage from the scene that was shared online.
Law enforcement officers from the Federal Protective Service were heard declaring an "unlawful assembly" around 10 p.m. local time and ordering the crowd to leave the area. The FPS is a unit of the Department of Homeland Security.
"If you trespass on federal property with a weapon … you will be arrested," a recording played to the crowd said.
The federal agents were seen responding to the crowd with tear gas and other crowd-control tools.
The same ICE building was targeted Wednesday, hours after the inauguration of President Biden in Washington.
That night, some members of a crowd of about 150 rioters caused damage to the ICE building and also marched against the state headquarters of the Democratic Party.
One Portland business owner appeared on "Fox & Friends" on Friday morning, saying the nearly nightly protests in Oregon’s largest city over the past year have been "devastating" for businesses and residents.
"They come out here and they just literally destroy whatever they can, throwing things around," Stacey Gibson said. "They’re just going to pick it up and throw it wherever they can. The damage that happens to the businesses down there is just devastating."
On Friday, White House press secretary Jen Psaki couldn’t offer a response to reporters who asked whether President Biden planned to address the chronic unrest in the Portland and Seattle areas.
"I haven’t spoken with him specifically about those events," she said, "but it’s something our national security team ... the homeland security adviser is closely monitoring, of course."