WARSAW, Poland – These are tough times for political pollsters.
In recent elections in Israel, Britain and Poland pre-election polls failed spectacularly to predict election results, sending shock waves through entire nations on election night. Now many research pollsters are analyzing these fiascos in search of ways to do better in the future.
In Britain, Prime Minister David Cameron's Conservatives won a clear victory on May 7 after polls had predicted a near tie with Labour, generating expectations of a hung parliament. A day after the election the British Polling Council launched an independent inquiry into the polls' inaccuracies.
In Poland, the Association of Market and Opinion Research Organizations, said that it plans to carry out a similar evaluation after a presidential election in which the incumbent, Bronislaw Komorowski, lost to a little-known challenger, following surveys that for month had suggested he would glide to easy re-election.
In Israel, Prime Minister Benjamin Netanyahu's Likud won after polls had showed the opposition Zionist Union ahead. Leading pollsters in Israel said the discrepancy had more to do with last-minute voter shifts than with any flaws in their surveys.
So far, there are no definitive conclusions on where thing went wrong in the Polish and British cases but experts point to several factors — a key one being political correctness. In short, people lie to the pollsters because they are ashamed to admit their true intentions. In Britain, there appears to be the phenomenon of "shy Tories" — people who secretly harbor the intention of voting Conservative, still stigmatized as a "mean" party, telling pollsters they'll vote for the more socially progressive Labour. In Poland, people may have been embarrassed to admit they were voting for the right-wing presidential challenger, saying they supported the more moderate incumbent. And sometimes non-voters declare an intention to vote due to the stigma of not voting.
"People feel forced to declare a preference even when they don't have crystalized preferences," said Miroslawa Grabowska, director of the CBOS polling agency in Warsaw. Among undecided Poles, she said, many may have skewed the predictions by saying they would vote for Komorowski of the ruling centrist party, Civic Platform — an easy and uncontroversial choice.
Professor Leighton Vaughan Williams, director of the Betting Research Unit and the Political Forecasting Unit at Nottingham Business School, also points how lack of enthusiasm in one camp can distort polling data. He spoke of a "lethargic Labour" effect in Britain, with Labour Party voters not turning out in the numbers expected last month.
After Labour's loss, some political observers accused uncharismatic party leader Ed Miliband of running a lackluster campaign.
"They stayed home. They weren't lying to the pollsters, but in a sense they were lying to themselves," Vaughan Williams said. Vaughan Williams said other, secondary, factors that the polls did not reflect included tactical voting and a swing to the incumbent once voters were in the polling booths.
Polling insiders say they have been aware for years of methodological pitfalls of their research and constantly seek ways to overcome them.
One of them is the falling number of households with home phones — the traditional target of polling — and the rising number of cell phones, which could skew results by leaving out younger voters more likely only to have cell phones. Jan Kujawski, director of research with Millward Brown in Poland, said his organization has avoided this problem by questioning people on home phones and on cell phones to "cover the whole population."
Grabowska says that her institute gets around the cell phone problem by questioning people in their homes. But that brings its own problems, with people less likely to answer honestly in person than when they can remain anonymous, she said.
In Britain's election, the bookies proved to be a much more accurate predictor of the election results than opinion polls. Vaughan Williams explained that the betting markets are more accurate because they take account of all polls as well as other information.
"The betting market is like one mind that combines the collective wisdom of everybody," he said.
Associated Press writers Jill Lawless in London and Tia Goldenberg in Jerusalem contributed to this report.