Lawmakers seeking criminal referral to Justice Department over ex-IRS official Lerner

File: May 22, 2013: Ex-IRS official Lois Lerner is sworn in on Capitol Hill, in Washington, D.C.

File: May 22, 2013: Ex-IRS official Lois Lerner is sworn in on Capitol Hill, in Washington, D.C.  (AP)

Lawmakers on a House committee are trying to send a criminal referral to the Justice Department seeking prosecution against embattled ex-IRS official Lois Lerner, Fox News has learned. 

The move comes on top of efforts to hold Lerner in contempt of Congress over her refusal to testify on the IRS targeting scandal. A meeting of the House Oversight and Government Reform Committee to consider contempt is set for Thursday. 

But before that, the House Ways and Means Committee will meet Wednesday to prepare a letter to the Justice Department citing possible criminal activity by Lerner. 

Specifically, Fox News has learned the letter will argue Lerner violated the constitutional rights of citizens, gave misleading information to investigators and inappropriately released private taxpayer information. 

These accusations generally relate to the scandal over the agency's practice of singling out conservative groups seeking non-profit status for extra scrutiny.   

Republicans argue that Lerner played a key role in that practice as head of the Exempt Organizations Division. 

Criminal referrals from Congress to the Justice Department are rare -- and the Justice Department is under no obligation to pursue such a case. 

The last major effort of this sort was in 2008, when the leaders of the House oversight committee sent a criminal referral to the Justice Department after they claimed baseball star Roger Clemens failed to tell the truth during a hearing about performance-enhancing drugs. 

In that case, the department accepted the referral and prosecuted Clemens. He was eventually exonerated in court. 

Asked for comment Monday on the latest pending referral, IRS Commissioner John Koskinen said: "They're doing what they think is right, and I'm sure the Department of Justice is doing what they think is right."

Lerner, speaking before the House oversight committee last year, defended herself against the allegations in the IRS case. 

"I have not done anything wrong. I have not broken any laws. I have not violated any IRS rules or regulations," she said. 

Lerner, though, refused to answer questions from the committee, and did so again last month. 

Fox News' Chad Pergram contributed to this report.