NEWS9 June 2017

And the winner is...the exit poll

Brexit News Public Sector UK UK General Election Youth

UK – A mixed night for the pollsters as the new methodologies introduced after the 2015 general election were put to the test.

Polling station sign_crop

In the end everyone was divided. The pollsters were divided and the electorate was divided. Where previously the polling companies had been criticised for a herding mentality, this campaign – with their newly tweaked methodologies – meant a greater mix of predictions.

As everyone poured over their numbers – turnout, margin of errors etc – the biggest winner for the industry was John Curtice and the exit poll. It had predicted the Conservatives on 314 seats, Labour 266, SNP 34 and Lib Dems 14. It currently stands at 318 Conservative, 261 Labour, 35 SNP and 12 Lib Dem (at time of writing). Turnout was 68.7%.

In terms of share – which is where all the pollsters were looking to judge their success or failure – it was 42% Conservative, 40% Labour, 7% Lib Dem, 3% SNP and 2% UKIP.

In the main, this was a much bigger share for Labour than most had predicted. Survation’s final poll put Labour on 40%. Its head of political polling Chris Hopkins said: "We are really pleased that our telephone method has come good – and we published it this time. What’s becoming apparent is there’s not necessarily hard and fast rules to correct for errors made [in previous elections]. We didn't feel the need to change our telephone method – we were confident, but there hadn't been the acid test needed (as not published our telephone poll in 2015 )."

Those who fared less well often go underground at this point, but others were ready to analyse their figures. And there was much to dissect – from the youth turnout to UKIP defectors, the shift in Scotland to the vagaries of individual seats.   

Andrew Hawkins, chairman and founder, ComRes, said: "We calibrated our model on how voters behaved in 2015, and I've been on the record saying that the Tories were set for a 70-odd majority if behaviour seen in 2015 (in terms of turnout etc) continued – but it didn't.

"There was evidence of a late swing toward Labour. Some fundamentals shifted a bit – the perception of who would be the best Prime Minister had narrowed in the past few days. Fifty per cent of UKIP voters had said they'd vote conservative and didn't on the night. All the intelligence from the frontline didn't translate into the result on the night. There were lots of seats where the pattern of UKIP splitting and [more going to Labour than was expected] was enough to strip Tories of winning the marginals they'd hoped for."

For some pollsters the subjective judgement call on the turnout among the young went the right way, for others not. Ben Page, chief executive, Ipsos Mori said: "For the industry as a whole, some were very accurate and some were less so. It all depended on how you treated turnout. We were accurate on Conservative and Lib Dem in 2015 but over on Labour. Trying to learn from that, we went for verified actual turnout and that down-weights youth and Labour – but this time young people did something they've not done before. All the pollsters with very large Conservative leads made the assumption that the youth vote wasn't going to happen. Our fig leaf is that we were accurate within our margin of error.

"The reason the exit poll is a triumph is that you know the people who've voted so you don't need to make that judgement. This time the pollsters stuck to their guns – some are enjoying the victory and some are picking at their wounds."

YouGov predicted a hung parliament and that was good enough for CEO, Stephan Shakespeare. "We will look back on the 2017 campaign as the one where data science took a great leap forward in terms of understanding the electorate and how they vote. We invested in new research approaches – central to this was multi-level regression and post-stratification (MRP) analysis, which models types of voters and constituencies based on the sort of people they are and their past political behaviours.

“We went beyond static broad-brush clichés of the past voting groups – such as ‘Mondeo man’ and ‘Worcester woman’ – and instead got into the detail of the data. The events of the past few years show vividly the sharp and nuanced contours of an electorate which does not move in big blocks. By treating voters more as individuals, our data picked up some surprising movements at a constituency level. A good example of this was our estimate that Labour would take Canterbury, despite many commentators saying it was ludicrous."

The complexity of the electorate has been a theme in this outcome. Dr. Zsolt Kiss, co-founder of challenger polling firm, opinion.life, said: "Electoral behaviour and people’s political allegiances have become more volatile between electoral cycles. Coupling this societal change with pollsters’ inability to verify the true representation of these key and volatile groups in their samples can explain why some polls were significantly off.

"Last night’s results do not mean that opinion polling is obsolete, but that its methods need to change in response to societal changes. For broader market research, the difficulty of some polls to accurately identify last night’s results might not actually be that significant. People’s electoral behaviour is in many ways more complex and harder to measure compared to everyday purchasing decisions, for example.”

The finer nuance of the electorate is what Deborah Mattinson, founder of BritainThinks, has been concentrating on. BritainThinks’ swing seat focus groups gave a particular insight into how the public were viewing the political debate and campaigning.

"What we have, is a result that no one is happy with – Labour didn't win, the Tories ran the worse campaign in decades; what Labour did was defy expectations but for Labour moderates what does that mean? The smaller parties aren't happy; the joint Labour/Conservative share is the biggest since the 1970s. Voters told us that their greatest fear was a hung parliament and voters are usually good at getting what they want. This result highlights what happened a year ago [with the referendum splitting the country]. Did young people turnout? Looks like they did. But as overall turnout is not up that much, maybe older people didn't."

Her prediction is another election before too long and she urges the pollsters to come out fighting. "My feeling is the industry tends to be rather apologetic. Look at the final polls – some weren't right but they were within the margin of error; the industry shouldn't be cowed."

1 Comment

7 years ago

Measuring intention and past behaviour pose different challenges

Like Report