Some of the trapped Fortaleza citizens have spoken with family members using cellphones from within the collapse, said a fire department commander, Cleyton Bezerra.
Ten survivors have been found so far, along with an unidentified body, Bezerra said. At least nine people were still missing. A puppy was also freed, the BBC reported.
One person was believed to be stuck in an elevator, the BBC reported.
Rescue workers were concerned that parts of the building could collapse further as they listen for noises from under the debris.
The local newspaper Diário do Nordeste tweeted aerial video of a cloud of dust settling over central Fortaleza, while local journalist Donizete Arruda posted video of the rubble.
"I just heard a boom and the building came down at once. A man came out drenched in blood and with a broken arm; he was the only one we could help," Daiane Moreira, who was working next to the building at the time of the collapse, told journalists at the scene.
Fortaleza Mayor Roberto Claudio said it was not immediately clear what caused the building to collapse: "The only information we have is that it was an old building. We don't have specific information on licenses, permissions," Claudio said, promising a "rigorous investigation."
The Associated Press contributed to this report.