Addressing the crowds, Hanau Mayor Claus Kaminsky called the past week among the “blackest and darkest” the town had ever experienced during peacetime, the Germany News agency dpa reported.
But he qualified that those who want to pull apart society won't succeed, "because we are more and we will prevent that."
The armed suspect, a 43-year-old German man, fatally shot nine people in the Frankfurt suburb on Wednesday before apparently killing his mother and himself.
Five of the victims were reportedly Turkish citizens. The attacker left rambling texts and videos in which he espoused racist views, called for genocide and claimed to have been under surveillance since birth.
Turkey's ambassador to Germany, Ali Kemal Aydin, warned that Turkish immigrants in Germany are experiencing more and more hate crimes in the country.
"This cannot and this must not continue," Aydin said at the protest.
The racist killings were Germany's third deadly far-right attack in recent months. The rampage followed October's anti-Semitic attack on a synagogue in Halle and the slaying in June of a regional politician who supported Chancellor Angela Merkel's welcoming policy toward migrants.
The killings come at a time when the Alternative for Germany has become the country's first extreme-right political party to establish itself as a significant force in decades. All of the victims had immigrant roots, and many are accusing the party of producing a climate where extremism and racism can flourish.
There have been daily candlelight vigils for the victims since the shooting and another big protest against racism on Saturday that also drew thousands. A memorial for the victims is planned for later this week.
The Associated Press contributed to this report.