Updated

In a tedious exercise that could determine whether veteran Rep. Charlie Rangel keeps his seat, the New York City Board of Elections on Thursday began to count the roughly 2,000 remaining provisional and absentee ballots in the disputed primary between him and his top Democratic rival.

The 13th Congressional District race appeared decided last week on election night, with Rangel seemingly holding a sizable lead. State Sen. Adriano Espaillat, his rival, even gave a concession speech.

But the vote margin shrank, and Espaillat went on to claim he had received scores of complaints from residents claiming their votes had been suppressed.

Now, the remaining uncounted ballots could determine the race. Board of Elections commissioners told Fox News they expect to wrap up their count by the weekend or early next week.

At each table Thursday, a bipartisan team of four elections board employees -- two Democrats and two Republicans -- recorded the number of ballots that had already been validated by the elections board. These include absentee, affidavit, military and federal ballots.

Also at each table were two observers for the Espaillat campaign plus a watcher and a lawyer for Rangel. Both campaigns were allowed lawyers and observers, but Espaillat only sent observers.

The counting process remained generally civil, although at times the two camps sparred over ballot irregularities and bureaucratic minutiae.

Espaillat says that if their voter suppression allegations are proven true, a new election must be held.

"Our country has to rely on an election process and election system that is verifiable, that is transparent and that brings about confidence to everybody," the candidate said. "We cannot have a Florida-type situation in New York State."

Meanwhile, lawyers for both Rangel and Espaillat went to state Supreme Court in the Bronx Thursday for a hearing on claims that Latino voters were blocked from casting ballots and were improperly asked for ID in last week's primary.

Board of Elections commissioners, though, refute that charge, claiming they've been unfairly attacked by Espaillat's campaign.

On Thursday afternoon, Bronx State Supreme Court Justice John Carter ruled that he will have the final word on the Board of Elections results once they are in.

Fox News' Laura Ingle and The Associated Press contributed to this report.