OPINION21 August 2009

Stats in the news: Harmless nonsense?


Pollster OnePoll has defended its work against attacks from Bad Science author Ben Goldacre. But what does the presentation of survey data in the press do for the public’s trust in research?

Ben Goldacre has a bee in his bonnet about market research.

At an MRS-sponsored debate earlier this year, the Bad Science author accused the research industry of serving up “half-truths to order” and getting away with it by not being upfront about methodology.

He’s now got into an argument with OnePoll, which provides PR polls to UK newspapers, about the use of “PR-reviewed data” (a term he borrowed from Charlie Brooker). His comments drew an impassioned response from Harriet Crosse of 72 Point, which runs OnePoll, in the company blog.

Goldacre said he couldn’t possibly assess whether OnePoll’s results were sound or not, because “they won’t tell me anything about the questions they asked, the responses they got, or the people responding”. Secondly, he says, these articles are not news but advertising, pure and simple. Crosse’s response: “No shit, Sherlock.” She says everyone knows it’s advertising, including the reader, but they don’t mind “because they are entertained along the way”.

But the question of how much space the papers should give to this sort of ‘entertainment’ is just one aspect of how the media treat, or mistreat, data. And the worst abuses are the fault of the journalists and politicians who present the data as much as the researchers who gather it.

In our February issue we interviewed journalist Michael Blastland, who has written extensively on the abuse of stats in the media. He draws an important distinction between different types of nonsense.

“Most of what falls into the category of nonsense is harmless, designed to get a laugh or a bit of attention as much as anything, and with, possibly, some truth in it somewhere,” he said. “But then there is serious nonsense, where the subject needs treating and researching with care and rigorous independence of mind.”

There’s also the problem of robust research data being butchered by journalists who don’t know how to interpret it – or care. The British Crime Survey, for example, is routinely used to fuel tabloid scare stories.

Politicians are not above it either. The recent debate (if it can be called that) between the UK’s two main parties on future public spending plans was a wonderful example of how both sides are willing to obfuscate the numbers. As the BBC’s economics editor Stephanie Flanders wrote in her blog: “It all makes for a jolly knockabout in Parliament. But I pity the poor voter who has to decide what any of it really means for them or their own public services in the difficult years for the budget that lie ahead.”

It all feeds a belief that stats are up for grabs to anyone who can turn them to their advantage, and it does nothing for the public’s already shaky trust in quantitative data.

For more on trust in public data and market research, look out for the upcoming September issue of Research.