Under the spotlight

x Sponsored content on Research Live and in Impact magazine is editorially independent.
Find out more about advertising and sponsorship.

June 23 is not a date that the UK’s political pollsters will be particularly looking forward to. The calamitous polling experience of 2015 is still fresh in the mind, new polling methods are still unwrapped in some cases – never mind properly put to use – and the fear of getting consecutive electoral events wrong is pervasive. The morning after may require a dose of Prozac rather than a celebratory hangover cure.  

Meantime, the industry looks on, as do the media, the academics, the activists and the bewildered public. Let’s pity them all, for the messages we currently send merely serve to reinforce a reputation so brutally scarred by last year’s events. EU referendum polling is now too ingrained to face down the mode problem: telephone polls show a relatively comfortable ‘Remain In’ victory; online polls suggest a close finish, with perhaps a Brexit outcome. Yes, we face the possibility of predicting the wrong result again. Batten down the hatches.

I’d like to provide some reassurance here, but I can’t and I won’t. You can be sure that much of my time has been spent trying to solve 2015 riddles in preparation for this next test – and I’m not alone in that. Pollsters all seek pinpoint accuracy and nobody should doubt our collective integrity or desire to achieve the right result. But I have little idea whether my significantly expanded methodological and technical polling armoury has the right weapons for this battlefield.

Of course, it’s right that we focus on why different data collection methods are producing different results. At ICM, our published polls are still conducted once a month by telephone, but to date have not included referendum questions.

So, while there’s no corroborative evidence in this building that our polls would also produce a significant ‘Remain In’ lead, the assumption should be that they will do just that.

I think I know the reason, and it’s nothing to do with the referendum itself. In six of our nine post-2015 phone polls to-date, the raw sample has recalled voting in Ed Miliband as Prime Minister last May. The other three recalled a pretty tight race too. So, phone polls continue to suffer from the single material observation of the BPC Polling Inquiry in 2015 – that raw samples are biased in favour of Labour.

Why should this matter? Well, Labour voters fall in the ratio of 2:1 in favour of remaining in the EU. Too many of them fixed in the sample will skew the referendum result towards that outcome. It’s as simple as that. Past vote-weighting Labour voters (down) helps marginally, but as 2015 proved, by nowhere near enough to eliminate the bias. In my view, while other factors apply, the phone polls are pushing the ‘Remain In’ share a little too far one way.

Online polls suffer from a different problem – too many UKIP voters. My own online tests have suggested this week-on-week, with all online polls overstating UKIP before the 2015 election – some quite sizeably so. Suffice to say, having too many UKIP types in the sample has obvious implications for EU referendum outcomes – pushing the results a little in the opposite direction.

Hence we have an exaggerated gap by mode – each side pushed a little away from the other. I believe the ‘truth’ is somewhere between, and it might well be that – as the public becomes more and more engaged with the four-month campaign – we’ll see this effect diminish. But then of course, we’ll probably walk into herding accusations.

There’s so much more to the debate than this, and it would be wrong to conclude that one-dimensional issues affect each data collection methodology. But let’s be honest and clear – we approach this referendum only too aware that numerous critical eyes are on us, desperate to prove the value of what we do, but only able to hope, rather than expect, our polls return to former levels of accuracy.

Martin Boon is director at ICM Unlimited