Updated

The House lawmaker who has tried to rally his colleagues to speak up on behalf of the Pakistani doctor imprisoned for his role in helping track Usama bin Laden said Tuesday -- on the 11th anniversary of the Sept. 11 attacks -- that "we've abandoned this hero."

Rep. Dana Rohrabacher, R-Calif., renewed his call Tuesday for the U.S. to do more to bring attention to Shakil Afridi's plight and pressure Pakistan to free him. He addressed the case after Afridi detailed his miserable prison conditions in an exclusive interview with Fox News.

But Rohrabacher expressed grave disappointment with how Washington has handled the case to date.

"I have been for the last two months pleading with everybody who I can get a hold of that we must intervene with Pakistan to free Dr. Afridi and end his horrendous incarceration," the congressman told FoxNews.com. "He stood up for us ... and now we're not even speaking out in a loud voice and demanding justice for him."

He added: "We should be speaking out right now and instead we've abandoned this hero."

More On This...

The congressman made his latest plea to colleagues at the weekly GOP conference meeting Tuesday morning. He says Congress should at least be publicly recognizing his "heroism" on the 11th anniversary of the 2001 terror attacks -- as Afridi helped the CIA ultimately track bin Laden and bring him to justice for that deadly attack on American soil.

Rohrabacher accused both Congress and the Obama administration of "inaction" on the matter. However, he said he plans to speak with Sen. Rand Paul, R-Ky., on the other side of Capitol Hill about tackling the Afridi issue.

In the face of resistance from leadership, Paul is pushing a bill to freeze U.S. aid to Pakistan unless Afridi is released. He wrote in a letter to Senate Democratic Leader Harry Reid on Monday that he's prepared to use "any and all means" to secure a vote, including holding up Senate business.

In a statement Monday, Paul said the U.S. "should not give foreign aid to a country whose government is torturing the man who helped us kill Usama bin Laden."

Afridi, in his interview with Fox News, described brutal torture at the hands of Pakistan's Inter-Service Intelligence. Afridi said he was blindfolded for eight months and handcuffed with his hands behind his back for 12 months, leaving a debilitating effect on his eyesight and limbs.

He also said his captors told him "the Americans are our worst enemies, worse than the Indians." Afridi said the country's claims of cooperation with the U.S. are a sham to "extract" billions in U.S. aid.

But Afridi told Fox News he helped the CIA out of love for the U.S. "I have a lot of respect and love for your people," he said.

Rohrabacher said he wasn't surprised by the purported comments from the ISI. But he said "it broke my heart" to read Afridi's comments about his respect for America -- "knowing we have not done right by him in his greatest hour of need."

"I just hope that on 9/11 we don't forget one of the true heroes that has emerged," the congressman said.

The State Department has not commented recently on the Afridi case. Secretary of State Hillary Clinton said in May she didn't believe there was "any basis" for holding Afridi. "His help, after all, was instrumental in taking down one of the world's most notorious murderers. That was clearly in Pakistan's interests as well as ours and the rest of the world," she said.

Fox News' Dominic Di-Natale contributed to this report.