Have opinion polls had their day?

Opinion polls are part of the fabric of modern politics, fuelling the symbiotic relationship between politics and the media. Some academics argue that polls don’t just reflect public opinion, but help to shape it – in some countries polls are regarded as so influential that they are banned toward the end of the electoral process.

It is only recently in the UK, following last year’s general election in particular, that people have begun to question the accuracy of such polls; for years they were seen as the best source of information available to us. But the opinion poll is being challenged by the emergence of a new academic tool: the predictive market.

Predictive markets present a real alternative to conventional research surveys.  They have been proven to be more accurate in predicting a range of outcomes and in addition to often being more cost effective, they avoid many of the issues that conventional research surveys are prone to, such as sampling bias, respondent fatigue and dubious responses to overly long or irrelevant questionnaires.

•          Respondents self-select; they are free to answer whichever questions they want to, based on their own knowledge, from an array of questions covering different markets and subjects.
•          They only respond to a question, place a bet, if they think they know better than the market – so no forced questions.
•          There is a small amount of jeopardy.  If they bet incorrectly they will lose their stake – so more thought goes into each question.

A predictive market is the only true gamified research environment; respondents aren’t answering research questions, they really are playing. Analysis of the playing habits of participants in Predictive Markets has shown that they spend as long on the Predictive Market website as they do on social media sites and playing casual games such as Candy Crush and Angry Birds.

The chart below shows the betting on our predictive market as to whether the Scots would vote Yes in the Scottish referendum over a period of 10 months up to voting day. With the exception of the first four weeks, the outcome was never in doubt; the probability that the Scots would say Yes to independence was never higher than 50% and in the last three months was less than 30%. Even after the TV debates, the likelihood of a Yes vote only rose to 35% and that fell again to less than 30% shortly afterwards.

By contrast opinion polls showed an average of 43% Yes and 57% No (excluding don't knows) in July 2014 and August 2014 and in September, polls indicated that the vote would be even closer. On 6 September a YouGov poll gave those in favour 47% versus 45% for those against. But as the opinion polls narrowed and predicted a tighter outcome, the Predictive Market betting was moving decisively away from a Yes vote.

Similarly, the data below for the betting on the 2015 General Election show that, apart from some short-lived upturns, Ed Milliband never really got above 35% probability of being the next Prime Minister. David Cameron was seldom given a probability of less than 60-70%.

Probability of Ed Milliband being the next Prime Minister.

Probability of David Cameron being the next Prime Minister.

At the time of the general election the estimated voting outcomes published by most opinion polls were so close, 1% or 2% either way, that the differences weren’t even statistically significant. But, the predictive market was clear that there was only ever going to be one outcome.

By analysing many thousands of data points, Blinc and Media Predict (our US partner) have shown that predictive markets are far more accurate than traditional polling, and over a longer period, when it comes to election results. They have shown that women are no better at predicting outcomes for women than men are and vice versa! And they have shown that adults (regardless of whether they are parents) are capable of accurately predicting outcomes for children.

All of this is despite the absence of any adjustment to reflect the electorate or the national distribution of age, sex or anything else in the sample.

And predictive markets aren’t just useful for predicting political outcomes.  They can be used for optimising creative content, concept evaluation, NPD and a range of other applications.

In the US, the concept has been adopted by a number of leading companies to use in their business decision-making.  The last words on the subject go to the Harvard Business Review:

“Hewlett-Packard, Motorola, Intel, Best Buy, Microsoft, Google, and Pfizer have all employed internal prediction markets to assess likely product shipment dates, predict sales volume, and identify best-selling products. Their results have been impressive.” – Harvard Business Review, January 2015

By Justin Charlton-Jones, managing director, and Paul Barrow, founder, Blinc