Updated

His presidential campaign in turmoil, Republican Herman Cain refused to say Wednesday whether he will ask his former employer — the National Restaurant Association — to terminate confidentiality restrictions on women who accused him of sexual harassment in the 1990s while he was head of the trade group.

The candidate was supposed to take questions after a speech to health care professionals but he ultimately refused and left the hotel through a back door.

"I'm here to visit with these doctors and that's what I'm going to talk about, so don't even bother asking me all of these other questions that you all are curious about, OK? Don't even bother," a testy Cain told a throng of reporters who were peppering him with questions.

When pressed about the allegations, Cain raised his voice and said "What did I say? Excuse me. Excuse me!" as hotel security led him through a hotel hallway jammed with journalists in a Washington suburb. "What part of 'no' don't people understand?"

Try as he might to project an image of campaign business as usual, the former Godfather's Pizza chief executive appeared frazzled and couldn't escape the questions that have dogged him since the allegations surfaced three days ago — two months before the leadoff Iowa caucuses and just as polls show him at the head of the GOP field alongside former Massachusetts Gov. Mitt Romney.

In the latest twist, a lawyer for one of his accusers is asking the National Restaurant Association to free her from a confidentiality agreement so she can talk openly about her allegations and respond to Cain's claims that the complaints were "totally baseless and totally false."

"I know her very well," lawyer Joel P. Bennett told CNN late Tuesday, "and I'm sure she would not make a false complaint."

Cain's campaign manager, Mark Block, repeatedly refused to say whether the campaign was in discussions with the trade association over letting the woman talk freely. Block said the campaign would address that question "when it's appropriate."

A spokeswoman for the restaurant association, Sue Hensley, said Tuesday night that the group had not been contacted by Bennett. Messages seeking comment Wednesday were not immediately returned.

The pressure on Cain only increased early Wednesday when a pillar of the GOP establishment suggested that the Georgia businessman should ask the association to waive the agreement so that the woman can talk openly about her allegations.

"What are the facts?" asked Mississippi Gov. Haley Barbour on MSNBC. "If you have a confidentiality agreement that keeps the public from finding out something that the public is interested in knowing the facts, you ought to go on and get the facts out."

"Herman Cain's interest is getting this behind him," added Barbour, a former Republican National Committee chairman.

As the day began, Cain spoke to the Northern Virginia Technology Association, saying: "There are factions that are trying to destroy me personally, as well as this campaign." He didn't say to whom he was referring, but he said "the voice of the people" is stronger.

Over the past two days, Cain has admitted he knew of one agreement between the restaurant association and a woman who accused him of sexual harassment. He has said the woman initially asked for a large financial settlement but ultimately received two to three months' pay as part of a separation agreement. Cain also acknowledged remembering one of the woman's accusations against him, saying he stepped close to her to make a reference to her height and told her she was the same height as his wife.

He has said he is not aware of agreements or settlements with any other women, though Politico — which first disclosed the allegations — reported that the trade group had given settlements to at least two female employees who accused him of inappropriate sexual behavior.

The New York Times reported Tuesday night that the trade group gave a female employee a year's salary in severance pay, $35,000, after she said an encounter with Cain made her uncomfortable working there. The newspaper cited three people with knowledge of the payment to the woman, who was not Bennett's client.

For Cain, Wednesday was supposed to be the culmination of a three-day attempt at courting official Washington — and the GOP old guard that seems to be tilting ever more toward Romney.

A former talk show host, Cain is a self-styled political outsider who has attracted tea party support and, for now at least, weathered a series of stumbles that have many GOP luminaries questioning his ability to run a viable campaign much less win the party's nomination. Conversely, Romney is running his second national campaign and has spent the past few weeks shoring up support among the GOP establishment for a nomination fight many Republican insiders think is his to lose.

Cain has spent the past three days defending himself and giving conflicting accounts as to what happened in the 1990s.

By Tuesday night, Cain had begun to try to pivot toward Congress and the war for lawmakers' endorsements that could mean critical on-the-ground support and campaign cash. Cain's rival in Iowa, Romney, has a sophisticated network of surrogates in Congress trying to coax their colleagues into his camp. So far, they've rounded up at least 33 endorsements. Cain has none.

But lawmakers remained interested.

The delegation from Cain's home state, Georgia, helped set up a series of private events intended to introduce Cain around Capitol Hill.

Cain dined near the Capitol with a gathering of Republican senators Tuesday night. On Wednesday morning, after a speech in nearby Alexandria, Va., Cain was heading to Capitol Hill for a speech to House members on health care.

From there, it was back-to-back events set up by Rep. Tom Graves, R-Ga. First, Cain was to meet and greet House members at the discreet Capitol Hill Club for a conversation about health care policy. Then it was on to the Republican National Committee, where Cain was to speak with members of the Georgia delegation, a spokesman for Graves said.

At some point, Cain was to meet House Budget Committee Chairman Paul Ryan of Wisconsin. Ryan is meeting presidential candidates in his role at the Republican National Committee.