Updated

Top officials from the agency accused of blowing through taxpayer dollars on luxury getaways and other expenses endured a public drubbing Monday on Capitol Hill, as one invoked his right to remain silent, others apologized profusely and lawmakers tore into them for the "gross abuse of taxpayer funds."

Members of the House Oversight and Government Reform Committee at times yelled at the representatives from the General Services Administration called to testify Monday, demanding strict punishment. The acting chief of the agency later assured lawmakers that he's ordered a few GSA officials to repay the government for their personal expenses and will refer any "criminal activity" that is uncovered to law enforcement.

The hearing, the first of several examining a 2010 Las Vegas-area conference by the GSA, stretched for hours. It focused in large part on Jeffrey Neely -- the western regional commissioner now on leave over his role in organizing that $820,000 conference.

Neely, in an uncomfortable exchange, pleaded the Fifth a half-dozen times when asked basic questions by Rep. Darrell Issa, R-Calif., chairman of the committee. "Mr. Chairman, on the advise of my counsel I respectfully decline to answer based upon my Fifth Amendment constitutional privilege," he said each time. Neely later left the hearing, refusing to answer questions from Fox News.

Rep. Elijah Cummings, D-Md., top Democrat on the panel, said the allegations against Neely document an "indefensible and intolerable pattern of misconduct."

Cummings, referring to internal documents allegedly showing Neely gloating about the money he was spending, accused Neely and his wife of blowing through federal dollars, as if they believed they were "some kind of agency royalty who used taxpayer funds to bankroll their lavish lifestyle."

"They violated one of the most basic tenets of government service. It's not your money," Cummings said.

GSA Inspector General Brian Miller also told the committee that fear of retaliation from Neely was a "significant factor" among employees in the agency. "They apparently had a very hostile environment," Miller said.

"It seems clear that Mr. Neely has a lot to answer for," Cummings said.

Acting Administrator Dan Tangherlini later said he has "demanded reimbursement" for "private in-room parties" from Neely and two other officials with the agency. He also said he's canceled 35 previously planned conferences.

Other agency officials, both current and former, apologized for the 2010 Western Regions Conference and condemned the over-the-top spending documented in the inspector general's report.

Martha Johnson, the administrator who resigned after the report was made public, called the Western Regions Conference -- which had been escalating in cost for years -- a "raucous, extravagant, arrogant, self-congratulatory event that ultimately belittled federal workers."

She personally apologized, while defending the work of the GSA as a whole. "I am extremely aggrieved by the gall of a handful of people to misuse federal tax dollars, twist contracting rules and defile the great name of the General Services Administration," she said.

David Foley, a GSA official who was captured on video joking about the agency's spending at the 2010 conference, also apologized at the hearing. Foley said that he didn't know the "over-the-top" expenses were being paid for with government money.

Issa complained earlier Monday that Neely is still being paid -- one official estimated his salary at about $170,000. The congressman noted that Neely is on administrative leave along with four other employees of the General Services Administration.

Issa told Fox News that while the hearing was to be about broader issues of spending and accountability at the agency, he hopes Neely gets booted off the payroll.

"Remember, this is an individual who's still being paid today by the American people," Issa said. "He hasn't been fired."

Two other GSA officials were fired after the inspector general report found the agency spent more than $820,000 on the 2010 conference.

Miller said Monday that investigations are ongoing, and that his department is looking at possible bribery and kickback schemes.

The Oversight Committees also released internal memos that showed GSA officials debated last year whether to give Neely a bonus for his job performance. The officials were aware at the time that the inspector general was investigating the conference spending.

Johnson granted Neely a $9,000 bonus over the objection of Deputy Administrator Susan Brita.

Brita wrote in a November 2011 email, that "based on what we know already" about the conference and a questionable awards program, "I would not recommend a bonus."

Johnson wrote in an email, "yes on a bonus" in part because Neely had to serve in an acting capacity "forever and a day."

Lawmakers at this point are looking beyond the 2010 conference itself and into other areas where the agency may have spent money imprudently, such as on employee incentive programs.

Issa said the conference is the "tip of the iceberg."

The Associated Press contributed to this report.