Mustafa Barghouti, who is running a distant second in opinion polls, was forced by three Israeli plainclothes officers into a police car outside the Lion's Gate entrance to the walled Old City, witnesses said.
Barghouti flashed a "V" sign with his fingers before he entered the car.
It was the second time Barghouti was detained by Jerusalem police during the campaign.
Israel has promised to allow free and fair Palestinian elections and agreed to limited campaigning in the disputed city. But it insists that only Palestinians with proper permits be allowed into the disputed city of Jerusalem.
Barghouti lives in the West Bank.
The status of Jerusalem is one of the most contentious issues in the Israeli-Palestinian conflict. Israel captured east Jerusalem in 1967 and annexed it. The Palestinians want the eastern section of the city for the capital of a future state.
Israel has agreed to allow east Jerusalem Palestinians to vote Sunday. More than 5,000 Palestinians will be permitted to vote in east Jerusalem, but tens of thousands more will have to travel to outlying suburbs to vote.
Barghouti is expected to receive only a small percentage of the vote.
The leading contender, Mahmoud Abbas (search), had canceled tentative plans to visit the Old City in a campaign stop, fearing a large Israeli security presence could embarrass him among voters.
Barghouti was detained by Israeli police in the Old City last month, but released hours later.
Israeli police had no immediate comment on Friday's arrest.