Hegseth emphasized Monday that Team USA won 293 medals at the competition and there only two protests.
Fencer Race Imboden and two teammates were winners of gold in a team foil competition at the games in Lima, Peru, Friday and he took a knee on the podium at the medal ceremony. Imboden won the gold with teammates Gerek Meinhardt and Nick Itkin, both of whom stood for the anthem.
On Saturday, American athlete Gwen Berry also raised her fist at the end of the national anthem that played at the medal ceremony for the hammer throwing competition. Imboden took to Twitter to explain that his pride in America has been "cut short" and that he was "sacrificing" his moment on the podium to call attention to issues, including that President Trump "spreads hate."
His statement caught the attention of Crenshaw, a Texas Republican congressman who lost his eye in an explosion in Afghanistan.
"If you think this is 'sacrifice,' you are delusional. All you’ve done is haphazardly denounced the entirety of our country because of its imperfections. You and your fellow athletes have only succeeded in politicizing one of the few things that still unite us: sports," the former Navy SEAL wrote on Twitter.
Hegseth wholeheartedly agreed, saying Imboden's protest was disrespectful to the flag.
"People look at this and they're sick of it. Dan Crenshaw is right, that is the farthest thing from sacrifice I have ever seen. Look around, buddy, look around in this country, the men in blue, the folks that wear camouflage and what they do for us every day, and you're talking about sacrifice?" he said.
The athletes could face disciplinary action from the U.S. Olympic and Paralympic Committee.
“Every athlete competing at the 2019 Pan American Games commits to terms of eligibility, including to refrain from demonstrations that are political in nature. In this case, Race didn’t adhere to the commitment he made to the organizing committee and the USOPC,” Mark Jones, vice president of communications, USOPC said in a statement on Saturday.
Fox News' Robert Gearty contributed to this report.