NASHUA, New Hampshire -- Herman Cain hasn't paid a lot of attention to New Hampshire - visiting the state only a handful of times since he jumped into the presidential race. But on Thursday the former businessman returned to the Granite State, making several stops, including a sparsely attended rally.

His campaign portrays itself as a grassroots effort, one that plans to capitalize on name recognition for a strong showing in New Hampshire's critical "First in the Nation" primary.

Cain only has one paid staffer in the Granite State and today's event was moved from a small venue in Hollis to a hotel ballroom in Nashua to accommodate the expected turnout. The room was big. The crowd was not.

About 100 people turned out to hear a wide ranging speech in which Cain touched on his flat tax plan, the need for energy independence and foreign policy. He also lashed out at the media.

"They're spending more time focusing on when I'm not talking than when other candidates are talking," he said, referring to his awkward difficulty earlier this week answering a Milwaukee newspaper editorial board question about Libya.

"They've even got it down to ... it's 11 seconds," he added. The incident was captured on videotape and showed Cain floundering for an answer, compounding the impression he lacks foreign policy experience. However Cain tried to spin it as a positive.

"I think before I speak. I know that's a novel idea," he said. "Who knows every detail of every situation in every country in the world? Nobody."

Thursday morning Cain was scheduled to sit down with the Manchester Union Leader, New Hampshire's largest newspaper. However in an effort to avoid a repeat of the Wisconsin gaffe his campaign would not allow the session to be videotaped. Cain's campaign spokesman, JD Gordon said he told the paper earlier in the week the session would be limited to 20 minutes but this morning the Union Leader said that would not be a sufficient amount of time and the meeting was cancelled.

It's fair to say the last few weeks have been difficult for Herman Cain. Allegations of sexual harassment were not the headline grabbing stories his campaign was hoping for, shifting focus away from his '9-9-9' flat tax plan and forcing him to defend his moral character.

Despite the recent negative publicity Cain remained upbeat, telling the crowd in Nashua, "I'm still smiling and I'm still inspired."

Seeking to return to more positive political footing he told supporters, "yes, we're going to be here a lot. But here's the message ... the question I want to ask. If you believe you're better off than you were three years ago, vote for a politician," said Cain. "If you don't believe you're better off, vote for a businessman who wants to put this country back on the right track."

It was his portrayal of himself as a political outsider and someone who knows how to create jobs that propelled Cain to the top of national polls. But the harassment allegations seem to be taking a toll on the businessman's White House bid.

A FOX News poll released on November 16 shows Cain at 15%, a decline of 9% since last month when he was leading all candidates with 24%. Conversely, Newt Gingrich has seen his numbers surge to 23% and Mitt Romney continues to hold steady with 22%.