Updated

The outgoing IRS commissioner claimed Friday that his agency's practice of singling out conservative groups for extra scrutiny was "foolish" and "horrible" -- but not illegal.

That determination, however, is up to the Justice Department to make as it begins to formally probe what went on at the agency over the last three years.

While the soon-to-be-former Acting Commissioner Steven Miller repeatedly rebuffed the most serious allegations Friday regarding his agency, Attorney General Eric Holder has said criminal actions could be brought.

There appear to be at least three prime areas where possible criminal violations could be investigated -- whether the IRS lied by concealing the program; whether it broke the law with its targeting of conservative groups; and whether it leaked confidential information. Holder also said civil rights violations could be examined.

Lawmakers were closely focused on the first allegation during Friday's hearing, claiming they were at least misled when Miller and other IRS officials declined to tell Congress about the practice.

More On This...

    Miller knew about it in May of last year, but did not fully disclose what was happening during a hearing two months later. Other IRS officials in the loop also did not come forward.

    Miller said Friday that he answered all questions "truthfully" last year.

    Rep. Dave Camp, R-Mich., chairman of the House Ways and Means Committee, said after his committee's hearing Friday even an "omission" can rise to the level of wrongdoing. He also complained about "an arrogance that came across" in the hearing room.

    "This investigation has just begun," Camp said.

    Federal law does prohibit false statements by members of the government. It bars anyone from knowingly concealing information or making a materially false statement. The punishment is up to a $250,000 fine and/or five years imprisonment. Federal law also bars U.S. officials from failing to report information about the violation of any "revenue law."

    A memo Friday from House GOP leaders, obtained by Fox News, said acknowledgement of the targeting program "directly contradicts repeated IRS testimony and written responses to Congress" over two years.

    Inspector General J. Russell George also testified Friday that he told the general counsel of the Treasury about his investigation in June 2012, and Deputy Treasury Secretary Neal Wolin "shortly thereafter." Those would appear to be among the highest-ranking officials to date said to have known about the IG probe. The Treasury Department, though, said they did not know about the actual findings of the probe until March 2013.

    The second area investigators could look at is whether the IRS broke the law in its targeting of conservative groups for additional scrutiny. Federal law prohibits attempts to interfere with the administration of "internal revenue laws." The punishment is up to a $3,000 fine and/or one year imprisonment.

    Asked at Friday's hearing whether that practice constituted a crime, Miller said: "It is absolutely not illegal."

    He then clarified that he didn't think it was, and that the inspector general's office indicated it wasn't.

    Finally, Justice could look into whether the IRS leaked confidential information, as some Republican lawmakers have suggested.

    Federal law does prohibit the unauthorized disclosure of information. The punishment is up to a $5,000 fine and/or five years imprisonment.

    Camp, at the start of the hearing, brought up several prior allegations against the IRS involving the possible disclosure of information. This included the question of whether donor information from the National Organization for Marriage -- an anti-gay marriage group -- was leaked last year.

    Miller, though, said he would be "shocked" if IRS information was shared with other agencies.

    Still, George said his office was continuing to conduct other investigations, as congressional committees and the FBI do the same.

    House Speaker John Boehner asked earlier this week "who's going to jail" over the scandal.

    Miller indicated Friday that, at the least, the agency is not done with resignations. So far, two officials have resigned or retired, though Republicans say that's not enough to address the problem.