Key independent lawmakers said Monday that they will stand by Australia's embattled government after the parliamentary speaker temporarily stepped aside over allegations of sexual harassment and fraud.

The opposition has called on independents Tony Windsor and Rob Oakeshott to bring down the minority government with a no-confidence vote in response to the scandal enveloping the government's choice of House of Representatives speaker, Peter Slipper.

Slipper stepped aside Sunday while police investigate allegations leveled by a former staffer last week that Slipper misused taxi payment vouchers while traveling by limousine in Sydney on three occasions this year.

The same staffer, James Ashby, 33, is also suing Slipper in the Federal Court for sexual harassment, alleging that Slipper only employed him as a media adviser in pursuit of sex. Slipper, 62, who is married with two adult children from a previous relationship, denies all the allegations.

Oakeshott described the scandal as the "darkest days" of the Parliament since it was elected in 2010. But he said it did not affect his pledge to support Prime Minister Julia Gillard's minority Labor Party government.

Windsor said his agreement with Gillard to support her government had not been breached.

"If there's corrupt activity (within government) or maladministration, let's see it," Windsor said.

The Greens party, which supplies a third lawmaker to support the government, issued a statement maintaining support and warning against Parliament acting like a court.

The opposition needs at least two of these three lawmakers to defect to bring down the government with a vote of no confidence when Parliament next sits on May 8. Such a motion would need at least 76 votes in the 150-seat House of Representatives.

Gillard on Monday defended her choice of Slipper for the job.

"I don't claim to know Mr. Slipper personally or well, but I formed a professional judgment about his ability to do the job," Gillard told reporters in Singapore.

Slipper has been dogged by allegations of abusing entitlements for years and has been ordered to pay back thousands of dollars in expenses. He has explained these invalid expense claims as honest mistakes.

Ashby's lawsuit alleges that the former conservative government had been made aware in 2003 that another male staffer had complained of being sexually abused by Slipper, who was then a senior government lawmaker.

Archbishop John Hepworth, leader of the Traditional Anglican Communion, said Monday that he will ask Slipper this week to stand aside from his position as chancellor of the breakaway church while both the civil and criminal cases are ongoing.

Hepworth, a former Roman Catholic priest who recently claimed to have been raped by fellow priests decades ago, said the sexual harassment allegations against Slipper were more serious than the fraud.

"The claim that goes to moral failings is of perhaps even greater significance than the claims of financial dealings," Hepworth told Sky News television.

A Catholic inquiry last year found no substance to Hepworth's rape allegations against the church.