House Majority Leader Eric Cantor on Sunday downplayed the argument that he lost his House seat because of his immigration stance and hinted that his 22 years in elected government might be over.

Cantor acknowledged his views on immigration, which include some legal status for children of illegal immigrants, played a part in his GOP primary loss Tuesday in Virginia to Tea Party-backed challenger Dave Brat.

“That infuriated people on both sides. Sure,” Cantor told ABC’s “This Week.” However, Cantor called his stance a long-held, principled position and said he never supported “amnesty” for the millions of Americans living illegally in the United States.

“We may not be able to dissect [the loss] now,” said Cantor, who also faced accusations of losing touch with district voters. “I don’t think any one particular reason [determined] what the outcome was. …. Frankly, that’s not what I’m focused on now.”

Cantor said he would talk to his wife before determining what he would do next and hinted several times about a potential role outside elected office.

“I believe I can play a role not just in government, but in the private sector,” said Cantor. He also told ABC that a gratifying number of people have already contacted him about a future role.

“As I go forward, I will continue to find ways that I can be influential in making sure that we continue that drive for solutions,” he said. “We've got to demonstrate that conservative ideas ... is the answer to so many working middle-class people's problems right now."

Cantor, who is Jewish and spent roughly nine years in the Virginia Legislature, told CNN on Sunday that he didn’t agree with the theory that anti-Semitism was a factor in his loss to Brat, an economics professor.