Researchers offer up ‘half truths to order', says Goldacre

UK-- The research business and its critics clashed last night at a debate on whether market research is becoming more of a “manipulator” of public opinion than a “messenger”.

Ben Goldacre, journalist and author of Bad Science, was the first to land a blow at the Debating Society event, held in a committee room at the House of Commons. Arguing for the motion, Goldacre slammed market researchers for offering up “half truths to order”.

Goldacre claimed that market research has nothing to do with real methodological rigour and that after findings were served up to journalists and clients, researchers would hope “to get away with it and then run like buggery.”

A lack of transparency in the disclosure of methodology seemed to be at the heart of Goldacre's concerns.

Before concluding his opening statement, he criticised the Market Research Society, “the organisation responsible for policing this industry”, for not doing anything to address the situation.

Ben Page, managing director of Ipsos Mori Public Affairs, argued against the motion. He said: “If we operated in this way, we would go bust. The reason that people pay us millions in this business is because research allows them to make better decisions.”

Ten minutes into the debate Page turned his sights on the media, blaming them for much of the flack levelled at research. “Bad polling happens,” he said. “So does bad journalism. The media reporting of survey research does far more damage than bad surveys.”

He cited the national press's “manipulation” of the British Crime Survey, criticising the Daily Mail in particular for filleting out detail from the survey for the purpose of hysterical headline-making.

Page concluded by saying that Goldacre's view was “laughable and a gross distortion” of the situation.

David Walker, managing director of the Audit Commission, and ex-editor of the Guardian's Public magazine, weighed in on Goldacre's side. He paid particular attention to the flaws in the creation of social research projects where he believed that “too often possibilities of the scope of research are limited” because the client's view takes primacy over the issues that respondents face.

It fell to Andrew Hawkins, chairman of opinion pollsters ComRes, to conclude against the motion. He began by admitting that in a previous conversation, Goldacre had praised ComRes as being “less dodgy” than other agencies. “However”, he continued, “I'm not entirely sure that I'm as flattered tonight as I see that Ben sets the bar quite low.”

Hawkins said: “If market research is manipulating public policy, it's doing a very bad job of it. Broad public opinion is for capital punishment, an end to abortion and an end to MPs wages and expenses. If polls were successful at manipulation all these things would be law.”

He went further, claiming that there has never been such scrutiny of standards. “The motion suggests that we have moved away from some Golden Age of rigour and scientific excellence,” Hawkins said. “I'd like to turn the motion on its head and say that there has never been so much scrutiny and hunger for accuracy.”

Once the floor was opened up to other participants it became clear that Goldacre and Walker were going to have a tough time convincing many of their position. Firefish's Jem Fawcus said: “This debate isn't about the quality of market research, it's about the incompetence of journalists.” The inability of national journalists to understand and relate research findings found a considerable amount of favour in the room.

Michael Brewer of Clerestory received the evening's most sustained round of applause for his contribution, which neatly cut through the clichéd salvos against the press. “I'm not sure that the accuracy of polls or dodgy journalism is the issue here,” he said. “I think that agencies are shirking their responsibilities to ensure that their data is used and communicated properly.

“Yes, there are examples of irresponsibility in the press, but when that happens who is challenging the reporting? Who is challenging the use of skewed data? Agencies have a responsibility to see their work through and the leaders of the market research business need to take more of a lead in this.”

In summing up, Page sang a paean to today's research standards. “Accuracy is increasing. Our sophistication in understanding the research process has never been more detailed,” he said. “Research is better than ever and there is more scrutiny than ever. The idea that we're turning into spin doctors just has to be rejected.”

But Goldacre was having none of it. “Some blame lies with journalists, it's true,” he said, “but I am amazed at the lack of critical self-examination in this room. This industry has to set out standards and police them.”

The motion was defeated by 37 votes to 17.

Author: Marc Brenner