"I don't understand why you want to know," Aiken shot back. "I don't understand why it's any of your business.
"At some point, [the question] becomes just really rude, you know?" he said in an interview that aired Thursday on "Good Morning America."
The "American Idol" runner-up has been the subject of gossip about his sexuality ever since he became an overnight star in 2003.
And this year, the question became especially sordid after a former Green Beret claimed that he hooked up with Aiken through a gay Internet site.
Sawyer went right after Aiken, asking at the top of the interview if he was "ready to come out and say you're gay."
"That would not make sense for me do to that," Aiken said.
"You think I'm rude for asking?" Sawyer asked Aiken.
"I've gotten to a point where I feel it's invasive. Forget it. What I do in my private life is nobody's business anymore, period.
"I don't think you're rude because I figure people have a job to do," Aiken said.
"I just don't understand why people care, to be honest with you. I'm not spending my time with this anymore.
"This is a waste of my time."
Sawyer's questions were not popular with fans and gay groups.
"Media speculation about people's sexual orientation is not something we support," said Damon Romine, an official of the Gay and Lesbian Alliance Against Defamation.
"Coming out is a very personal, private decision that needs to be made by the individual."
"GMA" viewers also weighed in.
"A person has a right to his or her privacy," one viewer wrote on the "GMA" message board.
"Clay is trying to preserve this. I agree with him. What difference does it make?"
"Mr. Aiken seems tired and beaten down by this issue," wrote another viewer.
"People with no lives buy into the media speculation on his sex life ... Now, Mr. Aiken sees this is a no-win."
Wrote another viewer: "If Clay Aiken wants to be a role model and/or a hero he needs to put all this to rest by coming out or proving his heterosexuality -- not dancing around the subject like he is trying to hide something."