The old and the new

The audience at the debate, held at the offices of GfK, were mostly senior researchers, which made for a cosy event – a little too cosy perhaps – with plenty of namedropping and in-jokes.

On the ‘traditional’ side were industry stalwarts Bill Blyth of TNS and Phyllis Macfarlane of GfK. On the ‘innovation’ side were John Kearon of BrainJuicer and Ray Poynter of the Future Place, both of whom have championed new research methods.

As the four speakers took their turns, it became clear that there was more agreement than disagreement – at least about where the problems lie, if not about how to respond.

Ray Poynter (who recently predicted the death of surveys in 20 years) highlighted what he sees as being wrong with research: unrepresentative online access panels, people’s inability to report their own behaviour and the perception of conventional research methods as “slow, boring and crap”.

But Phyllis Macfarlane said “the rumours of the demise of the traditional market research industry and particularly the demise of the survey are much exaggerated”.

Ad hoc surveys, she pointed out, only account for about a quarter of what the industry does, she said, so the majority is safe from the challenges affecting survey research.

However, Macfarlane voiced concern about some online techniques. “If I hear one more person say ‘I don’t need surveys, I can just scrape the web,’” she said, “I will either shoot them or myself.” As for access panels, she agrees that they are a problem, but is yet to be convinced that we have a replacement. “I think there is a place for them in our toolbox,” she said.

John Kearon of BrainJuicer spoke next for the ‘innovation’ team. His goal is to “make market research sexy – because I think it is sexy. Understanding the human condition – how intriguing and fascinating is that? That is a really cool thing to be doing.”

But quantitative MR, he said, rests on just one pillar of science: statistics, which has “kind of run out”. Meanwhile research has barely touched other sources of knowledge such as psychology, the social sciences and behavioural economics.

“The top 10 research factories… are innately conservative, because the clients almost wanted the industry to be innately conservative – it’s an insurance policy more than it is inspiration. If I was being really naughty I’d say what they’re best at is M&A, but that will not stand them in good stead for the new world.”

The final voice from the ‘traditional’ camp came from Bill Blyth, who warned of the danger of clients losing trust in the scientific basis of MR. “I think there has been a tremendous downward commodity spiral in the industry. We’ve let in poor quality research, there’s been price depreciation. It’s Grecham’s law: bad data drives out good. Self-selecting samples are valid up to a certain point but where there’s no barriers to entry, no professionalism, wow, different panels get different results – when I was a child that was called sampling frame bias because of variable coverage.”

Industry associations such as Esomar and the MRS, he said, need to do more. “Our professionalism should be guaranteeing the quality of our work and I think we’ve let ourselves down here. Look at the price for online research now – the barriers to entry are nonexistent. We need more leadership in our professional skills.”

By the time the four speakers were done, the distinction between ‘traditional’ and ‘innovative’ had become fuzzy – their remarks were characterised more by an agreement that, whatever happens, things can’t stay as they are.

As the debate opened up to the floor, talk turned to the age of the attendees. Where were the young people? Attempts to engage young researchers typically involve steering clear of ‘boring’ methodological discussions – but that doesn’t sound like a solution. Do young doctors complain about conferences with too many papers about medicine? Do young software developers roll their eyes at people talking about software all the time?

Ray Poynter suggested the industry has problems with people development. “You make [prospective graduate recruits] stand up and give presentations, and you get the brightest, sharpest people, and then you give them crud to do for two years and they’re not allowed to present. By the time they do present, they’re sort of stodgy, like us.”

Adam Phillips, who chairs Esomar’s standards committee (and who described himself as “quite old”), said: “When I joined the industry one of the things that attracted me to it was the debate about how we did it. There was a sense of evolution and of discovery. If you think of the problems we confront, they have continued to evolve, but now we don’t have the debate. It’s unattractive to new people joining because they can’t sense that there’s progress. As an industry, we have conferences, I hear the most anodyne crap being spouted by people. I’d like to see much more debate, adversarial and questioning. I think that’s how we will move our technology on.”

Phillips struck a chord with his comments, and John Kearon went on to suggest that some of the responsibility lies with buyers. “Clients do not want the debate,” Kearon said. “I have seen so many times the very senior leaders of research within the world’s biggest companies try and lock down and snuff out debate because it causes them personal problems in the organisation.”

Anjul Sharma agreed, saying: “It’s amazing how quickly the shutter comes down when those sorts of discussions start to happen.”

Peter Mouncey, editor of the International Journal of Market Research, said he finds it much harder to get good methodological papers than he used to. “Most scientists have a body of knowledge which is generally still in printed journals. We seem to be gradually moving away from that,” he said. Part of the problem, he believes, is that the research industry is less concerned with scientific arguments than it once was because it is “not owned much by researchers anymore, it’s owned by PLCs who have a very different agenda about the way they manage data”.

So how can we fix this? If the industry is to nurture the serious methodological debates and the continuously accruing body of market research knowledge that Phillips and Mouncey hope for, it needs to have a culture of openness.

Tom De Ruyck of Insites Consulting (one of very few attendees under the age of 30 ) said that the atmosphere of open discussion fostered in social media circles needs to spread further, urging agencies to “publish everything, be open about everything – what works and what doesn’t work”.

Whether the wider industry can summon up this generosity of spirit may determine whether it ends up moving with the times or getting left behind.