Polls, Politics and Parties, Part 3: How predictive are polls in Slovakia?
Before launching into an extended discussion of public opinion in Slovakia, I thought it would pay to look at the quality of the tools we actually have. I approach this, however, as a non-specialist and look forward to input from others on how this compares to other countries and how better to measure what I am trying to get at.
I also undertake this knowing that whatever the results, I will still look at the topography of party support based on polls. They are the proverbial lamppost under which we search for our lost keys–the keys may not be there but everywhere else is too dark.
Finally, I distinguish here between accuracy and predictiveness. I have little doubt, knowing the experts who do this work in Slovakia, that the polls get a read on Slovak opinion that is close to what people actually think (sometimes better, sometimes worse but usually close). That is not the same, however, as figuring out which of those people will actually come out to vote and how they will make up their minds in the voting booth itself. When I talk below about “error,” I talk about the difference between what polls say and how ballots are actually cast rather than to mistakes by pollsters. There may be a technical term for this that I don’t know and I’d be happy to learn it.
From the perspective of somebody who wants to know the result–or wants to make some money in the odds markets, the prediction value of polls for all elections in the sample (Slovakia’s parliament in 2002 and 2006, the European parliament in 2004 and 2009) is not particuarly encouraging. The difference between results and the average of final polls was 2.5 percentage points which is 36% of the value of the actual result for the parties in question. Even among major parties in higher turnout parliamentary elections of 2002 and 2006 the average poll got the average party result wrong by an absolute value of 2.1 percentage points or about 24% of the party’s actual result. The maximum error recorded was 5.4 percentage points (31%). While some polls occasionally came close on specific parties, the poll average never did better than 0.8 percentage points (4%).
Nor do the levels of error seem to be decreasing. In fact the levels and percentages of error are remarkably consistent from one election to the next when differentiated by the category of election.
|Parliamentary Election||Year||Raw poll error||% poll error|
This means that any guess about any party’s electoral results based on average public opinion polls, whether made in this blog or anywhere else is, at best likely to be 5% off in either direction and that the error will average (if the past is any guide, as it seems to be) around 24%. For a party exactly at the 5% threshold, a 24% error produces a range between 4.0 and 6.25. For a party with 30% support, that same average error produces a range between 28.0 and 43.75! For low-turnout European parliament elections the potential range is even wider. This resulting range is the combination of normal margins of survey error (a small part of the total) and a much larger component related to the likelihood of people to actually turn out to vote, regardless of their preferred parties.
The task, then, is to figure out whether there are any ways to figure out specific locations where the errors are likely to emerge and to try and correct for them. This means looking specifically at time periods, parties, and polling firms, something I will do in the next post.