Self-Promotion Theatre Presents: My interview on polls with the Slovak Spectator

Thanks to Michaela Stankova of the Slovak Spectator for asking good questions about changes in public opinion in Slovakia.  Her questions, in fact, serve as prompts for some of my future blog posts, but in the meantime you can read the interview here:

http://spectator.sme.sk/articles/view/38874/2/polls_consistently_show_coalition_slumping.html

Euroblindness, Part IV: Slovenska slepota

To even a first time reader of this blog, it is clear that I am addicted to the politics of Central Europe, but some long-time readers might also know that I also have a minor (but still unhealthy fascination) with the Eurovision song contest– coming up in less than two weeks, by the way.  These two things came together for me in a new way when I read SME‘s nice nice piece on Slovakia’s campaign songs from seasons past (http://volby.sme.sk/c/5376090/najlepsie-predvolebne-piesne-od-hzds-po-obamu.html), a follow up to their piece last week on the new HZDS theme song for 2010 (http://volby.sme.sk/c/5368283/hzds-nakrutilo-vivat-slovakia-ii.html).

Watching Vivat Slovakia again (alternative version here) and listening to the other political songs included by SME, I am struck at the degree to which Slovakia’s political songs bear resemblance to some of its Eurovision songcontest, which should not be taken as a complement to either.  This raises interesting possibilities for new mashups and co-op marketing opportunities:  Eurovision already has strong political element and a much-debated electoral system so why not complete the circuit by formalizing the role of entertainment in politics.  Ladies and gentlemen, I present:

Next round to be held in 2014, finals in Strasbourg.

Finally, I cannot let a discussion of political songs go by without what is, in my mind at least, Slovakia’s best political song:

Dashboard News: What to make of Median (again)

The final April poll is out in Slovakia and because it is “Median” it is more or less what we might expect, which is to say that for the parties whose numbers are most interesting, Median’s numbers are utterly unreliable.  Actually in technical social science terms they are quite reliable in that they “measure the same way each time” and almost certainly invalid in that, at least from the perspective of somebody who is interested in how people will actually vote they are almost assuredly not a good “approximation to the truth or falsity of a given inference, proposition or conclusion” (For more on the differences, see http://www.socialresearchmethods.net/tutorial/Colosi/lcolosi2.htm).  In other words–and this is much the fault of the Slovak media as it is of Median–DON’T RUN STORIES ABOUT WHO WILL GET INTO PARLIAMENT OR NOT USING MEDIAN DATA.

(For a prime example, see SME prieskum-medianu-nepustil-do-parlamentu-smk-a-most-hid.html, though the headline does at least specify that the poll is from Median, which is a warning to more informed readers.  Slightly less irresponsible, but rather odd is the TA3 headline, PRIESKUM: Keby boli voľby v apríli, vyhral by ich Smer-SD which emphasizes that Smer “would win” despite the fact that this is the one thing everybody does know and that the Median data for this month actually shows the largest single drop in support for Smer of any Median poll I can find.)

So what can we make of Median’s numbers.  In the spirit of Stephen Stucker, we can still make something.  Median’s trendlines are not usually out of line with the rest of the parties.  So here’s what we learn.

In Median’s poll Smer shows an enormous drop–almost six points.  This comes after two months of Smer numbers in Median that were unusually high and bucked the trendline of the other polls, so this is probably simply a return closer to the norm for Median on Smer and brings it in line with the other polls, though it is worth noting that Median always polls high for Smer (with the exception of only two months in the last year it has produced the highest numbers for Smer, usually between 3 and 4 points above the average of the other three polls).  So Median confirms Smer’s drop but doesn’t tell us how far, though around 36 or 37 looks like a good current guess.

SNS shows a big rise in Median’s numbers, one that pulls it back to January levels.  The curve is almost exactly the same–though a bit higher–as that of FOCUS, suggesting that the FOCUS number is more than just statistical noise and that SNS may have actually benefitted from the Fidesz victory in Hungary.  Only Polis showed no SNS bounce in April and so we can probably take the bounce seriously and perhaps move SNS slightly higher in its probability of passing the threshold, though its long-term trend is still resolutely negative.

Median is the only poll to give HZDS a bounce in April–the other three show stability or a slight drop.  In this case it may be the Median trend that is out of line.  As with Smer and SNS, Median consistently polls high for HZDS and so its claims on all three of these need to be discounted a bit.

Current coalition parties overall.  It is notable that even though Median tends to poll high for the coalition parties, and does so again this month, the overall trend (small graph at the right) shows an unbroken drop in total coalition preferences over the last 5 months by about 9 points overall (A disclaimer: it matters where you begin your trendine.  Carry this graph back two months and it appears that January was actually an unusually high peak compared to the months before or after.  But carry it back 6 months and the trend of decline re-emerges quite distinctly.)

For SDKU the situation is muddled.  Median polls very high for SDKU this month, but it does not always do so, suggesting that the party may have shown an increase.  The evidence from the other polls is mixed: Polis also shows an increase, though not as big as the one in Median; MVK and FOCUS show slight decreases.  This is one party without clear trends at the moment.  It appears to be holding steady between 13 and 14.

Like both MVK and Polis, Median shows a drop for KDH but from the highest level the party has ever received in a Median poll.  FOCUS also showed a drop for KDH (but from a much lower baseline).  Here we have significant disagreement, with FOCUS pulling the lower end and MVK near the higher with the two less established polls both showing higher.  It is hard to say what this means. I will have to think about it.

This is the first Median poll to show SaS above the 5% threshold, but Median has consistently polled very low for SaS (as do all new and small parties), and what is important here is that the party shows the same almost unbroken upward trend and degree of increase in Median as in every other major poll.  The party will fall back some degree when voters are in the booth, but it is hard to imagine it falling so far short that it misses the threshold.

The overall rise for the “right” parties is almost perfectly reciprocal to the decline of the coalition, a steady increase of 6 points since November and more from the summer of 2009.  This conforms quite well to the findings of every other poll.

Median has a big Hungarian problem. About 11% of Slovakia’s population is Hungarian and past elections show that the Hungarian parties/coalitions together receive about the same percentage of the vote.  Yet only three times in the last year has Median’s total of preferences for Hungarian parties exceeded 8% and five times (including this month) it has been below 7% (this month the two sum to 6%).  This suggests a major problem with Median’s methodology or its network or both and so it is hard to give credit to its numbers for Hungarian parties or its trends.  For the record this month shows a slight drop for MKP-SMK and a slight increase for Most-Hid but these likely mean little and they certainly do not merit the SME headline mentioned above.

From my college era I remember a slogan (likely propagated by beer companies to prevent excesses that would hurt their PR and inspire anti-drinking legislation), “If you must drink, drink wisely.”  A note to SME (and Pravda and TA3 and all the others:

If you must cite Median, cite Median wisely.

The new pollster in town: A quick read on AVVM

If a previously unknown astronomer announces the discovery of a major comet, major news outlets might be forgiven for not immediately running with the story.  It could, of course, be another Hale-Bopp (or Deep Impact) or it could be dust on the lens.  This is the problem the Slovak media faces with regard to a recent poll by a survey firm that has never before entered the party preference fray: AVVM, the Agency for Research on Public Opinion. Do media outlets pick up AVVM’s press release and run with it or do they wait for a second or third poll from this firm to see if it is doing good work.  The answer is that they run with it (with the apparent exception, so far, of SME).

So what do we get from a poll by AVVM?  Perhaps something, perhaps not.  AVVM’s website is the simplest imaginable, an out-of-the box 4 page website (one of which is blank) with no logo or other distinguishing characteristics that would say “take me seriously.”  On the other hand AVVM’s director Martin Palasek has worked previously (and maybe still does) for other firms.  I will say this:  AVVM certainly did a great job of getting out the news of its first poll.  A quick search for the firm produces no records from any time before this week, but in the last two days the poll story has already been picked up by dozens of Slovakia’s news outlets.  Before make any judgment about the firm itself, I will wait for first hand reports from those in the public opinion field in Slovakia who may know better.

If we cannot judge from the firm’s records, perhaps we can begin to judge the firm by its results.  This is always difficult, of course, because we have no way of knowing that AVVM is not right and the others wrong.  But to do this we at least need to put the AVVM numbers in context and array them against those of other firms whose accuracy we can judge (and whom we have judged in the past).  The graphs below show individual polls and trendlines with the AVVM result in bright yellow.  The conclusions:

  • The poll’s results are well within the expected range for every party except the two largest, Smer and SDKU.  For these AVVM shows a strikingly low figure for Smer (about 10% below the normal range, though in the same direction as the trendline) and a rather high figure for SDKU (about 15% above the normal range, for which there is no ready trendline justification.  Of course anomalies for only two out of eight parties is rather normal for polls, and without a track record we have no way of judging whether these are outliers for the firm itself.
  • The poll tends to show lower results than other polls for parties of the current coalition–its number for Smer is exceptionally low but its numbers for SNS and HZDS are on the low end of the range–and higher results for the Slovak parties of the opposition–its number for SDKU is exceptionally high and its number for KDH is on the high side of the range, while only its number for SaS is near the middle.
  • The AVVM poll differs from the major polls in showing Most-Hid well ahead of SMK.  Other polls show both low (Median) or both high (Polis) or MKP-SMK above Most-Hid (FOCUS, MVK)
  • The AVVM poll also shows a surprisingly high result for the HZDS splinter AZEN (a result which I find unlikely) and for the revived SDL (again not likely to be translated into election results).  This could account for the unexpectedly low levels of support for Smer, since these otherwise obscure parties lay in the same ideological space.

So who benefits from AVVM’s report this week (regardless of its accuracy)?

  • Most-Hid, which here (and nowhere else) appears to be the only viable Hungarian party.  If nothing else, this is a valuable antidote to the upcoming Median poll which, if the past is any guide, will show Most-Hid below the threshold (because of Median’s unusual questioning methods).
  • Opponents of the current coalition.  A poll showing Smer to be weak may help embolden the opposition.  A continued weak poll showing for HZDS may add one more impression to an overall image that it will be unable to cross the threshold discourage its marginal voters from “wasting their vote.”
  • The leaders of AZEN and SDL.  These parties have little chance but this must make them feel as if they are close to a breakthrough.

So what to make of AVVM’s poll overall?  It’s not absurd, not far off, but there’s also no reason to pay it much heed just yet.  With any luck the party will publish a June poll so that we can compare it to final results and make a better evaluation.  Until then I’m not adding it to my averages but I will keep track of it and give the firm an individual analysis.

Smer

SNS

HZDS

SDKU

KDH

SaS

MKP-SMK

Most-Hid

Polls, Parties and Politics, Part 7: How (not) to help voters translate preferences into votes

Poznamka:  Vitajte!  Prepac, ze vsetko tu je len po anglicky ale 1) moja slovencina je bohuzial slaba a 2) mnohe z mojich citatelov nehovoria po slovensky.  Prepac.  Mam, vsak, “google translate buttons” na pravej strane (bohuzial grafy a tabulky su len po anglicky.  Ak chcete preklad, daj vediet: pozorblog@gmail.com alebo nechaj “comment”).

Poznamka II: HN uz publikoval slovensky preklad (neviem kto to robil, ale dakujem): http://hnonline.sk/slovensko/c1-43853830-kovac-junior-sa-uniesol-sam-volim-hzds

In recent years thanks to interactive web technologies, a variety of news outlets, civic groups and party organizations have begun to create online tests to help voters figure out the relationship between what they want and the party that most closely fits their ideological preferences (for example http://www.quizrocket.com/who-should-i-vote in the US, http://www.selectsmart.com/FREE/select.php?client=GLAR in the UK, and most recently and impressively, http://www.euprofiler.eu/).  This trend has finally made it to Slovakia in the form of “Voličomer“, Pravda’s voting test which asks “How will people with similar opinions to yours vote?  The same, similar or completely different?  Fill out this questionnaire and you will find out right away.”

What I found out right away is that Voličomer leaves rather a lot to be desired:

  • The first time that I filled out the survey, I tried to exhibit the profile of a European social democrat, the party family closest to my own:  moderately statist on economic questions and liberal on cultural questions.  What came up first on the list, however was the Communist Party followed by the tiny HZDS-splinter AZEN, though with with a variety of other left wing parties nearby including Smer.  I was willing to accept this result as reflective of the absence of cultural liberalism from most left wing parties in Slovakia (though I have a hard time believing that the KSS is the closest alternative).   
  • I then tried to fill out the survey as a supporter of Slovakia’s opposition.  Here the results were rather good, with SaS coming first and SDKU coming second on the list, though oddly with AZEN again appearing near the top of the list.
  • My next effort was to to represent a member of one of the Slovak national parties, with criticism of both Hungarians and Roma and mixed answers on economic questions that are less important for such voters.  The result was below.  As with my social democratic effort, an extreme party popped up first–the radically xenophobic NS–followed by Meciar’s HZDS and the Slovak National Party.  But in between HZDS and SNS appeared the Party of the Hungarian Coalition, MKP-SMK!  It is virtually impossible for me to think of any meaningful quiz in which these parties appear next to one another except on in which the adjectives “Slovak” and “Hungarian” are erased and respondents are simply asked about the intensity of their national feeling (which would be an interesting exercise but it is not what Pravda is trying to do with Volicomer).  
  • A second attempt to represent a supporter of a Hungarian party produced an equally odd pattern that included, in the top five, two HZDS splinter parties (ND and AZEN), the Christian Democrats (KDH), the Party of the Hungarian Coalition (MKP-SMK)–but not until third place–followed in short order by an obscure worker’s party (ZRS), Smer, the Roma party and the radically anti-Roma NS.

The problem here is that Volicomer does not seem to be set up in a way that could produce a meaningful answer except in certain circumstances or by accident.  Setting aside the fact that like almost all such tests Volicomer is based only on policy questions and not on relevant circumstances such as who respondents voted for in the past or their affection for individual politicians, its failure to identify party preference has more a fundamental cause in its failure to deal with the linked questions of underlying issue dimensions and their salience.

Issue Dimensions

Voters’ answers to political questions tend to form a limited number of reasonably well-defined clusters, (in which case an individual’s answers each of the questions is, theoretically, fairly closely related to the answers o the others).  The content of these clusters varies from country to country and from one decade to the next, though most countries have long-lasting oppositions between pro-market and pro-state clusters on economic questions and many have similar clusters on cultural and religious questions, questions of foreign policy, regional and language policy and other sets of issues (for far too much detail on this question, see http://www.la.wayne.edu/polisci/kdk/papers/cleavage.pdf).  In some countries all the major political questions are more or less reducible to a single cluster, so that we can speak of  one-dimensional political competition.  In most, however, knowing voters’ or a parties’ position on one did not help guess positions on the others, producing two or three relatively independent dimensions.  (One of the core messages in my own courses on American politics is that despite our obsession with the all-out war between “liberal” and “conservative,” there are at least two dimensions in American politics: one economic and the other cultural.  For far to much detail on that question see http://www.pozorblog.com/2010/03/american-politics-in-a-nutshell/).

In Slovakia, I have argued, competition is at least two-dimensional.  With an economic dimension and a national dimension.  These have sometimes overlapped considerably but at other times have been almost completely unrelated, producing a two dimensional axis such as this graph that I’ve used in other settings:

This gets a bit complicated, of course, because “nation integrating” means, “Slovak nation defending” and its rival pole is not so much “Slovak nation integrating” as “Hungarian nation defending” but I simplify it here because the main area in which Hungarian parties and Slovak parties agree on this side of the axis is the need to integrate Slovakia into European structures.  Regardless of their labels, the important question is the degree to which these axes are independent of one another or somehow aligned so that positions on one are closely related to positions on the other.  This is an open question in changing circumstances.

I have some thoughts that in fact political competition in Slovakia is not defined by two dimensional but actually contains at least two additional “half” dimensions.

One of these is cultural, not in the national sense but in the religious sense and relates to issues of church and state and traditional morals.  The dimension seems to be at right angles to both of the others, forcing the inclusion of a third dimension on the graph, but unlike the others two dimensions, the distribution of people and parties on this dimension is quite asymmetric, with a relatively small share on the side of the church and traditional moral and the bulk of the population and the party system on the other side.

A second half-dimension, more speculative, is one that Tim Haughton and I have identified in Slovakia and other countries in central and eastern Europe: a “novelty” dimension.  We have argued that there is a relatively stable (if not dominant) bloc of voters who seek less corrupt governance and who seek new, untainted parties to achieve the goal, but who are invariably disappointed when those parties themselves become corrupt.  Thus although the dimension remains stable, the players on the “New and ‘clean'” pole are constantly changing.  Parties such as ZRS (1994), SOP (1998), ANO and Smer (2002), SF (2006) and today’s SaS and Most-Hid have at their peak occupied the “new and ‘clean'” end of this axis but slip gradually to the other end and, with the notable exception of Smer, which found other issues on which to build its base, disappear from political competition.

Which brings us back to the Volicomer question.  It is virtually impossible to understand the role of programmatic issues for party choice without a clear understanding of how those issues cluster together, how many clusters there actually are and how they are related to one another.  Except in certain circumstances, a simple additive list will ultimately produce gibberish (as Volicomer does on anything other than the economic dimension where the plurality of its questions are concentrated), while the best political quizzes begin with the question of dimensionality (see Idealog, http://www.idealog.org/, and Political Compass, http://www.politicalcompass.org/).

Salience

Even a quiz designed to account for Slovakia’s two-and-two-halves dimensions of competition will not produce particularly meaningful results unless it also accounts for the salience of the issue clusters.  All parties and voters emphasis that parties and voters place on them clusters when making their political decisions and these are not the same form party to party or from voter to voter.  In Slovakia in particular, those who tend to care about national questions put relatively little emphasis on other clusters of issues while those interested in economics usually put national questions in the background.  Knowing a person’s positions on issue dimensions, therefore, is important only if you know which of the dimensions is most important for that person.  By asking a high number of economic questions, Volicomer privileges the economic dimension and therefore produces acceptable, if not particularly useful results on that dimension while failing to produce anything meaningful on issues that are restricted to one or two questions.

Volicomer2: A (not so) Facetious Alternative

But things like Volicomer are hard to do, you might argue, and I should accept it as better than nothing unless I am willing to do the work to provide a better option.  Challenge accepted.  In the spirit of the old television game show “Name that Tune” (which I have never actually seen but which was part of the pop culture of my upbringing), “I can name that party in 4 notes.”  The flowchart below is a not-so-serious (but not entirely frivolous) attempt to integrate issue dimensions and salience to reduce the number of questions necessary for picking the major parties (and for the smaller parties the choice is largely random in any case):

Even a cursory look at the flowchart reveals the assumptions that I use when approaching Slovakia’s politics:

  • The “National Question” is the most polarizing and those who care about national issues are unlikely to care about much else.
  • The “National Question” is largely distinct from the economic questions, though those who care a little bit about national questions are more likely to prefer a statist economic policy (hence the “a bit” option that points to the choice between Smer and HZDS.  This was not always the case but the two axes have slowly come into closer alignment).
  • The choice between MKP-SMK is largely a personality driven one and therefore largely unpredictable on other bases.
  • The choice between SDKU, KDH, and SaS is one based on religiosity in the first case and novelty/”cleanliness” on the other.  In other words the two half-dimensions currently function primarily within the right wing of the political spectrum.  My guess is that by the next elections there will also be a “new party” alternative within the left wing that will siphon some votes from Smer, but we will have to wait to see about that.

I’m not convinced I’m right about any of this, so I’d encourage everybody who reads this to try Volicomer, try the quiz above and see which works better.  On this, as with everything, I’d love to see a larger number of reader comments!

Polls, Politics and Parties, Part 6: How (not) to second guess the polls…

Two-and-a-half months ago I finished up a blog post on poll predictiveness with a cliffhanger:

This question has driven experts to find a variety of proxy measures to figure out how to adjust polling numbers to reflect the final outcomes.  This post is already too long, however, so that will have to wait for another post.

By now readers have no doubt forgotten that they have been hanging on in suspense and have moved on to other important things, but I have not forgotten the promise.  Indeed I could not since this is one of the two biggest questions that shape how we look at poll numbers in Slovakia (the other is the threshold, which I will deal with in the next part of this series) and since the issue has surfaced in nearly every major Slovak daily.  No refer to just a few:

Each of these articles refers to methods by which one might try to put poll numbers in context, with the ultimate goal, the grail, to make a better prediction of election results.  On the one hand some articles, such as the most recent one from SME, say that this can’t really be done–that it depends too much on idiosyncratic behaviors and random events.  Other articles, such as the one in Hospodarske Noviny suggest that the firmness of voter commitment might offer some clues that would allow us to make better predictions.  Others have suggested factors such as voter turnout likelihood, measured either by surveys or by past polls.  Recent trends may also play a role in shaping how people decide when they actually enter the voting booth.  These all seem plausible, but rather than take any of these potential influences as real, it is worth testing them to see if they do have any relationship to the difference between polls and actually election outcomes.

The Role of Voter Loyalty

It seems quite plausible to assume that for two otherwise identical parties, the party whose voters are more committed to voting for it will perform better.  It seems plausible, the,  that polls (which usually don’t ask about degree of committment) will tend to underestimate the election-day performance of parties with more committed voters.  And this should give us a tool for making better predictions.

But it doesn’t.

At least it didn’t in 2006.  Below is the first in a series of graphs that looks at the relationship between potential sources of “poll adjustment,” such as “voter committment,” on the x- or horizontal-axis and the actual predictiveness of polls (specifically the difference between election results and poll results) on the y- or vertical-axis.  If a source of adjustment is useful, the graph should follow a straight line between lower left and upper right (in other words, parties which score “low” on whatever factor also underperform polls while parties that score high overperform polls).

The graph for “voter loyalty” (measured here as the difference in a March FOCUS poll between those supporters of a party who say they will “definitely” vote for the party and those who say they “might change” their minds) suggest that this data source does not allow us to improve our assessments.

In fact, there is a slightly negative relationship between loyalty and support.  The factor of loyalty might help explain why the longstanding SMK did better than expected and why the relatively new and weak SF did worse (which, if true, might suggest a discounting of current preferences for SaS), but it does not explain the overperformance of SDKU (except to the extent that SF voters may have shifted to that party, something not factored into the “loyalty” question).  More to the point, perhaps, it also does not explain the underperformance of HZDS, which in 2006 had the most loyal voters of all.  “Committment” may have something to do with it, but if it does, the interaction is too complicated for this single piece of information to produce any meaningful insights.  So HN may be right in pointing out that KDH voters are the most “solid” but that does not help much.

A related factor–perhaps very closely related–was cited by ING Bank in its own quite thorough 2006 analysis of the prospective election results which circulated before the elections.  The bank’s report, which unfortunately did not contain full footnotes, cited two measures of voter “discipline” and compared the share of “disciplined” voters committed parties to the share of voters overall.  Here again, a party with more “disciplined voters” should in theory have a better chance of outperforming the polls than a party with fewer disciplined voters.  Unfortunately, I do not know how they defined “disciplined” and I have a feeling that it is simply another way of calculating the same “loyalty” data above.  Since I don’t know whether it means anything, I analyze the results here only to demonstrate that the same results emerge for polls by both FOCUS and MVK (and therefore that this is not simply an artifact of FOCUS polling).

The trendlines here are essentially flat, suggesting no relationship.  The MVK results have a slightly positive trend, but not by much and the overall pattern is murky: again it works for SF and, with MVK, for SMK and SDKU but not for SDKU in the FOCUS survey or HZDS in the MVK survey.

The Role of Party-Voter Turnout

The voting decision is actually twofold: alongside the question of whom to vote for is the question of whether to vote at all.  The preference question itself does not measure whether those who prefer a party will actually make it to the polls on election day.    In theory those parties with voters who were more likely to go out and vote would do better than others and therefour outperform the polls.  We have data on this because in 2006 the pre-election FOCUS polls regularly asked about likelihood of participation.   The graph below combines the results for all major parties over four months and shows the relationship-line for the February data, the May data and the data for all four months aggregated together.

As with the loyalty data, the trendlines here point down, suggesting that knowing the likelihood of voter turnout for a party does not help figure out whether the party will perform better than the polls.  This figure works for the same parties as loyalty–SMK in the positive, SF in the negative–and fails for the same parties as loyalty–SDKU and SNS  outperform likely turnout while HZDS underperforms.  Of course there may be a reciprocal relationship here between SDKU and SF and between SNS and HZDS, but the point here is that the loyalty and turnout figures alone do not tell us that information and cannot be used to specify our predictions.

Previous Polls

An even more direct way to estimate the overperformance or underperformance of parties compared to polls is to look at the performance in the previous election.  In 2006 I set out to create a prediction model based largely on this principle, looking at the gap between past polls and past actual results using the 2002 Slovakia parliamentary election and 2004 Europarliament election and suggesting a relationship between differences in turnout (medium in 2002, very low in 2006) and differences between party polls and party performance.   The model did not work.  Or more precisely, it worked so idiosyncratically that it was not useful:  it worked well for SDKU and SF and badly for most of the rest.  The graph below demonstrates the weakness of that attempt: the

The graph shows two sets of dots because there is a slight comparability problem:  in 2002 the law forbade polls within the last two weeks of elections whereas in 2006 it did not.  The graph above shows differences between polls and results as a percentage of the poll figures in 2002 compared to 2006 for both the 2006 polls taken immediately before the election (white circles and colored rims) and those taken in a more comparable 2-4 week period before election (colored circles and gray rims).  In both cases, the results are the same:  predictiveness of polls in 2002 did not help assess predictiveness in 2006 except for SDKU, SMK and Smer (but only the 2 week prediction and  not the election day prediction).  It would appear that too many other variables affect the parties and polls over time for this method to work well.

What we need, then, is some kind of indicator that itself incorporates a significant share of the factors that shape poll predictiveness.

Oddsmakers and Experts

One way of assembling and integrating relevant factors is to find a smart and trustworthy person who has already done it.  It is even better if the person (unlike the author of this blog) stands to lose some tangible resources if the predictions are wrong.  There are not many public sources for this kind of information for Slovakia.  Political scientists, pollsters and journalists (precisely because they do have something to lose–reputation and perhaps even revenue) tend to keep their predictions to themselves.  And there is not yet a widespread public odds market in which individual buying and selling of shares offers a glimpse into public assessment of probabilities (a mechanism which has been extremely effective in predicting close races in the United States) except in SME‘s ambitious effort, which at least for now does not seem to involve real losses or gains or a particularly large base of participants (though that does not mean that the collective wisdom of SME‘s readers won’t be correct, and the consensus I see there is not implausible) .

What we have instead are two rather thin reeds: in 2006 the firm Tipos.sk allowed public betting on election questions, though the odds were established by the bookmakers themselves and only occasionally updated rather than in a true shares market (and which can be reverse-engineered into an assessment of the bookmaker’s own predicition); and in that same year an assessment by an analyst at ING Bank became public (though not necessarily with the encouragement of the bank).  I am sure there are other analyses floating around by political risk mangement firms, but I don’t have access to them (though if readers do and can share them, even for past elections, I would be extremely grateful).  The question is whether these expert opinions do much better than the raw data.  The answer is no, as the chart and graph below show.  In only 4 cases out of 16 did the expert opinion do better than the actual final polls in predicting the outcome (and in this cases for the likely suspects:  SF, with its weak base, and SDKU with its past history of significant underestimation) while in 7 of 16 cases the expert opinion did rather worse than the actual data.  Of course these analyses emerged more than a month before the actual election and, to be fair, they actually performed slightly better than the poll estimates from May polls, so experts do have their (slight in this case) value for those needing their predictions well in advance.

Party Actual 2006
election result
Actual results minus pre-election estimates
Final week
poll average

Tipos odds

ING Bank

HZDS 8.8 -2.7 -3.8 -3.7
KDH 8.3 -1.4 -1.3 -2.8
KSS 3.9 -1.2 -1.1 -0.9
SDKU 18.4 5.4 3.8 2.4
SF 3.5 -1.9 -1.0 0.3
Smer 29.1 1.5 4.2 4.6
SMK 11.7 1.4 2.5 -1.8
SNS 11.7 1.6 3.7 3.9
Average error in raw points 2.1 2.7 2.6
Average percentage error 21.7% 25.7% 22.5%

The best tool I can find: Pre-election trends

In the process of testing all of these various mechanisms for prediction, I decided to try one last method:  does a party’s trajectory actually affect its final outcome?  In this case the theory suggests it should not have much of an effect:  when polls are taken up to the last minute, then trends themselves should not shape the final decision.  It is possible, however, that a voters’ sense of trend might bring a reluctance to vote for a declining party or an enhanced willingness to vote for one on the rise.  And lo, when I ran the numbers the results came out shockingly clear:  in 2006 a party’s trendline for January through April had a clear relationship to the party’s tendency to over- or underperform polls:

This source of information correctly predicts 5 of the 8 and incorrectly predicts only 2 (and then only by narrow margins).  The dots are grouped tightly around the line, suggesting a strong relationship.  It is as if the election results followed the trendlines, but at levels they would have produced in July or August rather than June; the experience of going into the voting booth may force the kind of concentration that pushes voters to become the extrapolation of trends.

A strong caveat: I am hesitant to take this too far, especially since I have not been able to put together a time series for previous elections or in other countries, but the initial results suggest that the past trend is something worth attending to.

A word about prediction, caveat be damned. So what would this flawed (but not as flawed as the other methods I’ve looked at here) say about Slovakia’s upcoming 2010 election?  Well it is necessary to begin by looking at the 2010 four-month trends.  There are, of course, several ways to calculate the trend, averaging individual poll trends or aggregating all poll data together.  The results and relative positions of parties are reasonably similar, however.  The following table gives the range:

Party Trend Change
SaS Significantly positive +60% to +80%
KDH Moderately positive +10% to +13%
SMK Slightly positive +7% to +11%
MOST Slightly positive +2% to +8%
SNS Mixed -5% to +8%
SDKU Flat 0% to +1%
Smer Moderately negative -10% to -12%
HZDS Significantly negative -10% to -23%

This is good news for SaS (especially since if the question of “loyalty” and “discipline” are relevant anywhere it is for new parties such as SaS whose positive trend here may be enough for it to overcome the at-the-polls reluctance that kept SF from parliament in 2006) and not bad news for KDH, though this party seems locked into a 9.5%±1.5% performance zone regardless of external circumstances.  The news Smer is not good–but this is not unexpected, since the even party elites know that their party gathered a significant amount of soft support during its more successful years–but not catastrophic since Smer has a huge lead over the next largest party.  The worst news is for HZDS which has both the strongest negative trend in recent months and the least cushion to offset any losses.  If the trendline factor found in 2006 applies in 2010 then HZDS will not make it into parliament.  A big if.

For the other parties the prediction is far less clear.

  • I suspect this model does not apply at all to Most-Hid and SMK since the intra-ethnic competition takes a quite different form.  Of course this makes all of the above assessments rather useless since this is what everybody wants to know.
  • For SDKU the neutral trend is probably about right.  In the past the party has benefitted from last minute decisions by voters to stick with the tried and true on the right.  With a revived KDH and a solid alternative in SaS (barring any mistakes by the party or revelations about it in the next month), SDKU will likely not see the big jump that it saw in 2006 or 2002.  Its has a good chance of being the strongest of the right-wing parties, but probably not by too great a margin.
  • As for SNS, it is an open question.  In the last four months, SNS saw a general decline offset by a major rise in a single poll: FOCUS in April.  Take out that one data point and SNS shows a trend in the last four months that matches its sharp decline over the past full year.  It is for that reason that I am keen to see the April Median poll–not for level but for trend–and especially the May FOCUS trend.  If FOCUS in May puts SNS back under 6% then the party may be in real trouble.  The threshold will have a big impact on the shape the next government (the topic for the next post, I hope.)