Demonstrators waved Israeli flags at the rally outside the Alt Oper opera house in Germany's financial capital. Some held up signs reading "Support Israel Now!" and "Israel has the right to exist."
They were joined by a small group of Iranian dissidents with their country's flags, though police said there was no trouble.
President Mahmoud Ahmadinejad has caused international outrage by dismissing the Holocaust as a myth and questioning Israel's right to exist.
Germany's interior minister, Wolfgang Schaeuble, has said that his country would have to accept a visit by Ahmadinejad if he decided to come during the World Cup.
Holocaust survivor and scholar Arno Lustiger told the rally that any welcome for Ahmadinejad would "be a provocation to all German Jews."
Ahmadinejad has not announced any firm plans to attend. However, Germany's Central Council of Jews has said the presence at the tournament of one of Ahmadinejad's seven vice presidents, Mohammed Aliabadi, already is a provocation.
An estimated 1,200 people, many of them German Jews, demonstrated against Ahmadinejad before Iran's loss to Mexico in its opening World Cup game in Nuremberg last Sunday. Iran played Portugal on Saturday.
The prospect of a possible Ahmadinejad visit has been a delicate issue for the German government, which is involved in diplomatic efforts to defuse a standoff over Iran's nuclear ambitions.
Schaeuble has refused to meet his Iranian counterpart, Mostafa Pourmohammadi, during the World Cup.
His ministry said the Iranian embassy in Berlin had asked whether Schaeuble would meet Pourmohammadi during a visit to an unspecified Iran World Cup game. Despite reservations, a ministry statement said, a meeting would have provided a chance to condemn Ahmadinejad's statements and to press the case for early release of a German tourist held in Iran after his boat allegedly strayed into Iranian waters.
However, it said an Iranian official's recent statement that the man would not be released early removed a "significant basis of business" for any meeting.
German Jewish leaders have worried about possible shows of support for Ahmadinejad by Germany's far right.
Police rejected an application from the far-right National Democratic Party to stage a rally in Frankfurt Saturday, and the party did not appeal the ban.
Iran will play its last Group D match against Angola in Leipzig on Wednesday.