Is the biggest problem with polling the way it is reported in the media?

The motion focused on how polls are communicated to the wider public and suggested that issues with their portrayal and interpretation are primarily down to how media organisations report the results.

Speaking in favour of the motion, Simon Atkinson, chief knowledge officer at Ipsos, argued there was a litany of common errors in how the media reports polls, including overly exaggerating small percentage changes, over-focusing on one poll result out of many, watching the lead rather than the share, and ignoring limitations, such as polls not being predictions but snapshots of public opinion at a point in time.

“Opinion polls have a lot to bring to the next six months, with independent insights into what people think and feel,” Atkinson said. “We carry out a vital service in telling the story of what’s happening and speaking truth to power about how the public see as priorities.

“When it comes to how they are reported, we do need to talk about those cheer-leading headlines, we do need to talk about the boring but vital caveats in the data and we do need to talk about a single focus group masquerading as a poll.”

Also speaking in favour of the motion, Ben Shimshon, co-founder and chief executive at Thinks Insight and Strategy, said that the size of the polling industry had massively increased in recent years, and the relationship between those companies and media organisations had changed from one akin to those between clients and agencies to that between a PR function and a newspaper.

Pollsters are “trying to land stories and get publicity for themselves”, Shimshon added, and noted that “political polling is a shop window for a lot of research agencies – getting coverage does to some extent rely on whether you can offer surprising or noteworthy findings”.

But he argued that media outlets “continue to engage with pollsters as if it was the relationship from 2001 to 2005” and have not got the “same alertness and scepticism to an agenda that they would to a press release from a corporation”.

“In the end, media companies are responsible for the content that they publish and the data that they give credence to,” Shimshon said. “Those who cover surveys need to arm themselves with the tools they need.”

Opposing the motion was Maria Sobolewska, professor of political science at the University of Manchester and co-author of the book Brexitland, who said there were three bigger issues affecting the polling industry before their polls reached media companies: representation, regulation and enforcement, and responsibility.

On the first point, Sobolewska said: “The British polling industry is very unrepresentative of ethnic minorities in this country. This is a huge problem despite some recent improvements in transparency.”

This issue then feeds into policy making. “Polling is how political parties these days formulate their policies and the manifestos they put to their voters,” she explained. “All parties are now interested in this growing electorate [ethnic minority groups], but they do not know what ethnic minority voters want to see in their manifestos and what their positions are on the issues of the day.

“We have heard some examples of bad polls and bad polling practices. This is a direct consequence of a data vacuum we have around ethnic minority polling. If you don’t know what they think and what they might want, someone will enter that space.”

She added there was a lack of statutory regulation to enforce punishment against companies which conduct bad polls, and argued that the polling industry had to take more responsibility for how its polling results were reported. “Whose responsibility is it to deliver polls to the news media for reporting? We have to remember that the media are a huge commissioner of polling.”

Lexie Kirkconnell-Kawana, chief executive at media regulator Impress, also spoke against the motion and said that if news media is the only way to disseminate polling companies’ findings, pollsters needed to take a more active role in how it is presented.

She also argued that it was a mistake to expect journalists to keep up with the complicated methods and technical language used by market researchers.

“To blame news headlines rather than the market research industry to own and determine how its work is going to be communicated is akin to a conductor blaming the orchestra if the audience is not up with the tempo,” Kirkconnell-Kawana added.

“News is a cornerstone of democracy – it is a way for us to make sense of complicated issues and news is one of the few ways for the public to understand the views of others and get access to communities they otherwise wouldn’t.

“Pollsters have become dependent on the news industry to communicate their impact. In that, they have a much more co-creative role in how their stories are told or not told. The market research industry should take more leadership and ownership of how polls are reported.” 

The end vote saw a slim majority ( 30 to 28 ) vote in favour of the motion.