The new Secret Service director, already under fire months into his tenure for an incident involving allegedly drunken senior agents showing up at the White House, was told in no uncertain terms Tuesday by a frustrated congressional panel that he's got to crack down on his force. 

"Dude, you don't have to earn their trust. You're their boss. They're supposed to earn your trust," Rep. Chris Stewart, R-Utah, told Director Joseph Clancy. 

Stewart was responding to Clancy -- trying to explain why the Secret Service has been caught up in a series of controversies and security incidents -- saying it will "take time" to change the culture and he, as director, needs to "build trust" with the workforce. 

Clancy's testimony before a House Appropriations subcommittee on Tuesday comes after two senior agents drove through a White House bomb investigation scene after returning from a party earlier this month. 

But the director's explanation drew skepticism from lawmakers on both sides of the aisle. 

More On This...

"I find this testimony shocking," Rep. Nita Lowey, D-N.Y., said. 

Clancy said initial reports he heard from an anonymous source were that the agents were "inebriated." But he said the matter is now being investigated by the Homeland Security inspector general's office, and said he hasn't pursued an investigation of his own. 

Further, he said the agents have been transferred to "nonsupervisory" positions, yet are still getting paid while that probe is underway. 

The incident has proven to be the rare instance where both parties appear united -- they made clear that they expect the culture to change immediately, and that the latest incident is unacceptable. 

"This will not stand," Rep. Hal Rogers, R-Ky., said. He called Clancy's statement that he's not personally investigating "hogwash." 

In the March 4 incident, first reported by The Washington Post, two agents were returning from a party for a departing spokesman, and drove into the scene of an active suspicious package investigation. (The package turned out to be a book.) 

The agents reportedly were not given a sobriety test and were allowed to go home. Clancy could not say why they were not given that test. 

Clancy also testified that while the incident happened on a Wednesday, he did not learn about it until the following Monday. He said there was "no excuse" for that information not to have come to him earlier. 

But he stressed that he's seen some video footage of the incident, and the agents who interrupted the scene were driving at a "low rate of speed." 

Clancy said he didn't want to act "improperly" or too soon, but vowed: "There will be accountability." 

He said he would ultimately review a "table of penalties" in deciding what that accountability might look like. 

Rogers and others suggested people should be fired over the incident. 

The Inspector General has launched a probe into the widely reported allegations. "The IG's job is to conduct thorough and independent investigations, and that's what we will do in this situation," Inspector General John Roth stated in a press release. "Once we have completed our investigation, we will report our findings to the Secretary, the Congress, and the public."

The latest incident follows a string of high-profile security incidents at the Secret Service, including the individual who jumped over the White House fence and made it inside the residence.