Science fiction: The creeping threat of survey abuse

The perfect sitcom (that’s (rd+vf/a+s, in case you were wondering), the perfect day (quality = O+NS+Cpm/T+He) and the perfect rugby kick (the formula’s too long to bother with here) have two things in common. The first is that they are basically made-up research with no statistical basis. The second is that that didn’t stop them getting extensive mainstream press coverage.

Now Simon Singh, doctor of physics, MBE, author of several of the UK’s bestselling science books and occasional journalist, is taking aim at the ‘experts’ who give birth to these spurious research findings and the journalists who give them notoriety. In a lecture for British Science Week on Sunday he announced a (so far unspecified) prize for the most ridiculous equation that he discovers in 2010. He’s inviting you to email your submissions to voys@senseaboutscience.org and, in that open way that scientists have with debate, to suggest a name for the award.

“Everybody seems happy with these equations,” he said in his lecture, “but for me, they are a real problem.”

The reason he is unhappy, and the reason it concerns all of us, is not just that the public are getting a false impression of any type of research, it’s that they are becoming unable to distinguish potentially important, well-researched insight from made-up garbage that’s designed only to sell a product or an idea. Singh quotes a fellow scientist, Len Fisher, who comes up with some of these equations (how to pour gravy is one) – but only when he believes there is real intellectual content. Fisher is continually asked to provide pseudoscience solely to market a product, which he refuses to do. “Some of them [the PR companies] seemed genuinely astonished that prejudging issues and then providing proof is not science,” he says.

The same goes for the ‘surveys’ that make the news, complete with the ‘experts’ who validate them with sensational quotes which may or may not be supported by the evidence. Or the sensational internet polls that ‘prove’ some pre-determined editorial point. For many members of the public, just as the formula for the perfect chip (commissioned by Tesco) is science, so these surveys are market research. In this, the market research industry has common cause with Singh and his fellow academics: when MR is so easily traduced, then important research will inevitably be lost in the noise.

So who does Singh blame? Not, you may be surprised to learn, the PR companies that give life to this. Or, indeed, their clients who have paid for the pre-determined marketing-based result. He blames the people who lend their names and their doctorates to the press release.

“It would not happen if there were more mathematically literate people working in newsrooms who could put a stop to it,” he says. “But when someone comes up with this we all know it’s not real.” Therefore, he reasons, the only way to stop it is to stop it at source.

For mathematicians, who (most of the time) value the respect of their peers and their integrity above a cheap headline and a cheque from a PR company, this might be possible. But for market research, where anyone with an internet connection can produce a survey with alluring statistics and pie charts attached, the brake is harder to apply. Can anyone hold back this tide?

Tim Phillips is a Research contributor and writer of the Talk Normal blog.