Prime Minister Gordon Brown was dealt a devastating blow to his leadership late Thursday when one of his most ambitious ministers resigned — hours after Britain voted in European elections — and called on him to step down.

Work and Pensions Secretary James Purnell became the fifth minister to abandon Brown's Cabinet in recent days. Purnell called on the prime minister to resign for the good of the Labour party in a resignation letter seen by The Associated Press.

Purnell was the first of the five ministers to openly criticize Brown and ask him to step down.

In his letter to Brown, Purnell writes: "I am calling on you to stand aside to give our party a fighting chance of winning. As such I am resigning from Government."

Purnell, 39, was seen as a fast rising star in the Labour party but he said in his letter that he is not seeking to lead the party himself.

Purnell's resignation comes as Brown gears up for a for a major Cabinet reshuffling and tries to recover from a scandal over legislators' expense claims.

Brown's Downing Street office said the prime minister was "disappointed" by Purnell's resignation and that he only learnt of it at 10 p.m. It added: "his (Brown's) focus over the coming days will be on restructuring the economy through the downturn and strengthen it for the future."

Opposition leader David Cameron said the government was falling apart.

"With this resignation, the argument for a general election has gone from being strong and powerful to completely unanswerable," he said.

Ministers, aides and even the host of the British version of "The Apprentice" TV show spoke with Brown at his Downing Street office Thursday, as the beleaguered prime minister tried to reorganize his Cabinet to reassert his authority and revive his ailing government.

Some lower-ranking Labour Party lawmakers were seeking support for an attempt to oust Brown and hoped that the party's expected poor results in elections Thursday for the European Union parliament and local town halls would strengthen their case.

Brown's chief whip Nick Brown — in charge of party discipline — said dissidents were actively working to topple Brown, including veterans loyal to former Prime Minister Tony Blair and some maverick lawmakers.

An e-mail by rebel legislators that was leaked to British media tells Brown: "You can best serve the Labour Party and the country by stepping down."

Opposition legislators and analysts say a scandal over lawmakers' allowances has exposed Brown's failings — revealing a lack of command over his party and an inability to deliver promised political reforms.

Lawmakers from all parties have been seen as greedy for charging the public for such things as large screen TVs, a mole catcher and repairs to a swimming pool. Critics blame the ruling Labour Party for not tightening Parliament's lax laws.

Even though members of the opposition Conservatives have been responsible for some of the most outrageous claims, the party has surged ahead in opinion polls. Leader David Cameron is considered all but certain to win the next election.

Brown must call a national election before June 2010. Many legislators fear if Labour fails to ditch Brown before then, the party will lose many of the 349 seats it holds in the 646-seat House of Commons. Others wonder if Brown can focus on resolving Britain's economic downturn while his own future is in doubt.

"It's weakening his authority all the time, which, in my view, has to be damaging for the country's ability to manage the crisis," said Howard Elcock, a politics professor at Northumbria University.

Cabinet minister Hazel Blears, in charge of local government, quit her post Wednesday in what was seen as a deliberate attempt to damage Brown.

Business Secretary Peter Mandelson, once Blair's closest ally and now a defender of Brown, acknowledged there is unrest — but urged Labour activists to consider the consequences of axing a second leader in two years. Blair resigned in 2007 amid pressure from Brown's supporters.

Lawmakers forced to acknowledge mistakes over their allowances — including Treasury chief Alistair Darling — are expected to be fired, and Brown may reach out beyond his party's ranks for new blood.

Businessman Alan Sugar, the "The Apprentice" host, was spotted Thursday at Brown's office, stoking rumors he could join the government.