Ryan Lochte said the shocking protest incident against him last week almost ended his time on "Dancing with the Stars."

"Last week, I wanted to quit the show," he revealed to reporters after performing again on week two Monday night. "I was like, the 'Dancing with the Stars' show doesn't deserve this' and I just wanted to quit. I was really down on myself but 'Dancing with the Stars' came together as a family and they really supported me and gave me that positive influence that kept me going back to rehearsal, kept me being on the show. I owe it all to them."

Last week, two male audience members rushed the stage towards Lochte as he and partner Cheryl Burke were receiving judges' feedback during the live premiere show on Sept. 12. Studio security guards tackled them before Lochte or his pro were touched.

The men were later arrested. Also, several female audience members were ejected from the audience for yelling, "Liar!" at Lochte. All wore tee shirts with Lochte's name with a line through it. The group wanted to protest Lochte for saying he was robbed during the Rio Olympics. It was later discovered he and some swimming teammates had actually vandalized a gas station.

The bizarre incident, the first time security had to take action during "DWTS," made Lochte understandably rattled.

"I've always known that there's like haters out there, obviously, but I never got to really witness it in person and that was really shocking for me," Lochte said backstage on Monday night.

"One of the things—how I kept myself going is, they're not going to keep me down. No one will. The only person that's going to keep me down is myself and if I stay positive and keep moving forward, hopefully they'll start seeing that and seeing how I want to become better and I want to change and how they're not going to break me down. I'm going to just stay positive and keep going."

Monday, for TV theme night, Lochte and Burke did a quickstep to the song from "The Muppet Show," with the athlete dressed up in a green suit in a tribute to Kermit the frog.

He received a fairly low score of 24 out of 40, with Judge Julianne Hough sighing, "I saw so many mistakes," and Bruno Tonioli complaining, "You have the posture of Fozzie Bear."

Lochte was fine with the critique, saying, "I've really learned about myself this week 'cause this has been the biggest emotional roller coaster ride for me this week and being able to block certain things out and trying to stay positive and challenge myself, challenge my body was definitely a big thing and I was able to overcome it and I'm happy, even though I messed up on my dancing a little. There's room for improvement so hopefully I get that chance to stay on the show and just keep going."

Lochte's mom and girlfriend, Playboy model Kayla Rae Reid, have been supporting him in the audience.

"My mom, she's like calling me nonstop. I'm like, 'Mom, stop calling. I'm okay. I'm okay.'  She's like, [imitating her high voice], 'I'm just making sure you're okay.' So, I mean, the love that they're showing me has definitely really helped," Lochte smiled.

But Lochte explained to FOX411 his girlfriend hasn't been attending rehearsals: "She hasn't watched any because I want to surprise her at the show. She hasn't wanted to. She has her own life. She can't just go sit for six hours and watch me dance. It could be kind of boring."      

After last week, though, Lochte should hope his time on the show is a little more mundane from now on.

With a Tuesday elimination show looming, he vowed, "I’m not going to quit. I'm going to keep my head up and keep going."

Lochte said he hasn't taken any extra security precautions but noted, "I think the show has definitely stepped up the security and I'm so grateful for that."