Dozens of police officers were called in to control a raucous crowd Tuesday at a luncheon with Rep. Paul Ryan, as protesters repeatedly interrupted a speech given by the Republican congressman. 

Police arrested three protesters who "resisted our attempts to remove them," local Police Chief Brad Wendtlandt told FOX6 News in Milwaukee. A total of 20 were escorted from the Greenfield, Wisc., event. 

The protesters demonstrated inside and outside the local Rotary Club where Ryan was speaking, criticizing him over the state of the economy. As one protester was escorted away, she shouted: "I'm being arrested because I spoke out against my congressman." 

Ryan continued to give his speech despite the interruptions. He told FOX6 that he's available to speak to constituents if they come to see him at his office. 

"I have listening hours. I have office hours in my office for people who schedule meetings; that's what I've been doing all month and they could have set a schedule," Ryan said. 

Supporters of the House Budget Committee chairman also demonstrated outside the Rotary Club.