The pollsters’ challenges

Elections, we’ve had a few; and with each one comes a further analysis of the pollsters’ performance. But at last night’s IJMR lecture, Simon Atkinson, chief knowledge officer at Ipsos, reviewed the experience of pollsters in Britain and elsewhere, to give an overview – away from the knee-jerk headlines – of how the discipline had fared.

He said that historically, 85% of elections were ‘cut and dried’, leaving 15% ‘disruptive’. We’ve clearly been in a period where disruptive elections have come the new norm and that leaves specific challenges which he identified.

1.     Explaining the dynamic

Pollsters have been successful at describing and identifying the political environment, picking up the new populist sentiment such as identifying the America First ideology and a feeling among voters that the ‘system is broken’. In the UK, the polls for the recent election clearly indicated that Jeremy Corbyn was gaining ground.

2.     Measuring market share

For Brexit there were 35 surveys – 15 had Remain in the lead; 17 had Leave in front and three were a dead heat. “So they showed all along that the polls were too close to call,” said Atkinson. In the US, Ipsos had Clinton leading on the nationwide poll with a three point lead, the outcome was a two point lead – “we were getting it almost bang on,” said Atkinson, who pointed out that this level of accuracy would keep a brand owner very happy.

He added that there was very little indication that poll errors were getting bigger – “so the long term picture is of stability”.

3.     Sampling and analysis

Party support is now much more fluid, which means polling is dealing with a ‘moving target’. The UK referendum showed that attitudes and orientations were more important than traditional political allegiances when determining voting choice. Education has become a much more important variable.

4.     The issue of turnout

“The final poll is the crucial one and this is where turnout really comes into play,” said Atkinson. So for instance in the US, state-level polls turnout models didn’t work as there was a lower than expected Democrat turnout. In the UK referendum, 2015 non-voters did vote, so the turnout models made the results worse.

5.     Integrating opinion poll data with other sources

Atkinson talked about Ipsos’ Base Rate model which, based on international experience not just country-specific experience, includes two variables: incumbency and government approval rating. In the US, Base Rate showed it would be a Republican win, however Ipsos didn't share these results. He questioned when polling companies would be ready to use alternative sources with confidence and present their numbers in a different way.

6.     The public realm

Once the data is released, you can’t control it, which makes the job of reminding people that it’s not exact even more difficult. This has been exacerbated with the rise of social media.

And even if the polls are saying one thing, it isn’t always reported that way. For instance, even as the Brexit polls were showing it was too close to call and that Leave may be the outcome, many commentators chose not to believe the opinion polls, because it didn’t fit with their viewpoint.