OPINION22 April 2010

The old and the new


An Esomar debate in London last night pitted ‘traditional’ research methodologies against ‘modern’ ones – but the discussion went on to focus not on the age of the industry’s methods, but on the age of its people.


The audience at the debate, held at the offices of GfK, were mostly senior researchers, which made for a cosy event – a little too cosy perhaps – with plenty of namedropping and in-jokes.

On the ‘traditional’ side were industry stalwarts Bill Blyth of TNS and Phyllis Macfarlane of GfK. On the ‘innovation’ side were John Kearon of BrainJuicer and Ray Poynter of the Future Place, both of whom have championed new research methods.

As the four speakers took their turns, it became clear that there was more agreement than disagreement – at least about where the problems lie, if not about how to respond.

“There was more agreement than disagreement – at least about where the problems lie, if not about how to respond”

Ray Poynter (who recently predicted the death of surveys in 20 years) highlighted what he sees as being wrong with research: unrepresentative online access panels, people’s inability to report their own behaviour and the perception of conventional research methods as “slow, boring and crap”.

But Phyllis Macfarlane said “the rumours of the demise of the traditional market research industry and particularly the demise of the survey are much exaggerated”.

Ad hoc surveys, she pointed out, only account for about a quarter of what the industry does, she said, so the majority is safe from the challenges affecting survey research.

However, Macfarlane voiced concern about some online techniques. “If I hear one more person say ‘I don’t need surveys, I can just scrape the web,’” she said, “I will either shoot them or myself.” As for access panels, she agrees that they are a problem, but is yet to be convinced that we have a replacement. “I think there is a place for them in our toolbox,” she said.

John Kearon of BrainJuicer spoke next for the ‘innovation’ team. His goal is to “make market research sexy – because I think it is sexy. Understanding the human condition – how intriguing and fascinating is that? That is a really cool thing to be doing.”

But quantitative MR, he said, rests on just one pillar of science: statistics, which has “kind of run out”. Meanwhile research has barely touched other sources of knowledge such as psychology, the social sciences and behavioural economics.

“The top 10 research factories… are innately conservative, because the clients almost wanted the industry to be innately conservative – it’s an insurance policy more than it is inspiration. If I was being really naughty I’d say what they’re best at is M&A, but that will not stand them in good stead for the new world.”

The final voice from the ‘traditional’ camp came from Bill Blyth, who warned of the danger of clients losing trust in the scientific basis of MR. “I think there has been a tremendous downward commodity spiral in the industry. We’ve let in poor quality research, there’s been price depreciation. It’s Grecham’s law: bad data drives out good. Self-selecting samples are valid up to a certain point but where there’s no barriers to entry, no professionalism, wow, different panels get different results – when I was a child that was called sampling frame bias because of variable coverage.”

Industry associations such as Esomar and the MRS, he said, need to do more. “Our professionalism should be guaranteeing the quality of our work and I think we’ve let ourselves down here. Look at the price for online research now – the barriers to entry are nonexistent. We need more leadership in our professional skills.”

By the time the four speakers were done, the distinction between ‘traditional’ and ‘innovative’ had become fuzzy – their remarks were characterised more by an agreement that, whatever happens, things can’t stay as they are.

As the debate opened up to the floor, talk turned to the age of the attendees. Where were the young people? Attempts to engage young researchers typically involve steering clear of ‘boring’ methodological discussions – but that doesn’t sound like a solution. Do young doctors complain about conferences with too many papers about medicine? Do young software developers roll their eyes at people talking about software all the time?

Ray Poynter suggested the industry has problems with people development. “You make [prospective graduate recruits] stand up and give presentations, and you get the brightest, sharpest people, and then you give them crud to do for two years and they’re not allowed to present. By the time they do present, they’re sort of stodgy, like us.”

“When I joined the industry one of the things that attracted me to it was the debate about how we did it, but now we don’t have the debate”

Adam Phillips

Adam Phillips, who chairs Esomar’s standards committee (and who described himself as “quite old”), said: “When I joined the industry one of the things that attracted me to it was the debate about how we did it. There was a sense of evolution and of discovery. If you think of the problems we confront, they have continued to evolve, but now we don’t have the debate. It’s unattractive to new people joining because they can’t sense that there’s progress. As an industry, we have conferences, I hear the most anodyne crap being spouted by people. I’d like to see much more debate, adversarial and questioning. I think that’s how we will move our technology on.”

Phillips struck a chord with his comments, and John Kearon went on to suggest that some of the responsibility lies with buyers. “Clients do not want the debate,” Kearon said. “I have seen so many times the very senior leaders of research within the world’s biggest companies try and lock down and snuff out debate because it causes them personal problems in the organisation.”

Anjul Sharma agreed, saying: “It’s amazing how quickly the shutter comes down when those sorts of discussions start to happen.”

Peter Mouncey, editor of the International Journal of Market Research, said he finds it much harder to get good methodological papers than he used to. “Most scientists have a body of knowledge which is generally still in printed journals. We seem to be gradually moving away from that,” he said. Part of the problem, he believes, is that the research industry is less concerned with scientific arguments than it once was because it is “not owned much by researchers anymore, it’s owned by PLCs who have a very different agenda about the way they manage data”.

So how can we fix this? If the industry is to nurture the serious methodological debates and the continuously accruing body of market research knowledge that Phillips and Mouncey hope for, it needs to have a culture of openness.

Tom De Ruyck of Insites Consulting (one of very few attendees under the age of 30 ) said that the atmosphere of open discussion fostered in social media circles needs to spread further, urging agencies to “publish everything, be open about everything – what works and what doesn’t work”.

Whether the wider industry can summon up this generosity of spirit may determine whether it ends up moving with the times or getting left behind.


12 years ago

It's all happened before - it is merely another variation of ancient "Quant versus qual "battles. Why can't people accept that we have a variety of tools and we should nurture innovation but never forget the old principles of disciplined sampling and frank confrontation of sources of error. How do you measure anything without asking people questions? But, as always, the right people and the right questions.

Like Report

12 years ago

Two contributions from me. The Futility of Debating Old v New Good researchers don't give in to this false debate. They evolve methodologies or applications of methodologies. There is no such thing as 'traditional' or 'conventional' methodologies. Rather there are 'established' methodologies and new developments. Not in competition but used in complimentary and colloborative ways. Involving Young People Loved the Edge Learner Forum session at Research 2010 - talented young people doing 'engagement' projects not research projects and making a difference on-the-ground. I'm currently in York visiting the best in the business at this kind of work www.inspiredyouth.org My stepson is involved in Inspired Youth so I declare my interest but I also know that their work with young people is amazing so check it out.

Like Report

12 years ago

As often with these type of matters the solution is al about being creative incombining old and new. Do not follow ol new trends blindly, and do not stick to much to old school research. But at least I am certainly trying to implement new sources and technology in to research I do. Jorrit Lang

Like Report

12 years ago

It was mentioned on the night, but ESOMAR members tend to be older, more senior people in the industry, partly due to the higher subscription price. Perhaps we could open up these debates (and I hope there will be more of them - this was excellent) to younger non-members to give them a flavour of the organisation and encourage their participation, in much the same was as MRS is encouraging younger people. Over to you David Smith.

Like Report

12 years ago

MRS and you people represent a continuous stimulous for those working in the industry abroad. Please keep up the debate. And yes Paul, I completely agree: methodologies are not "conventional" or "innovative", they rather work more or less well to study a very complex matter that is consumer and market behaviour. Each method has got pros and pitfalls. To me the ability of researchers to me is to learn and apply the most appropriate method to client's needs. Best wishes from Italy, Luca Meyer

Like Report

12 years ago

Sorry for the delayed response on this topic but coming from the cognitive neuroscience research world and forming Sands Research Inc. (www.sandsresearch.com) with Steve Sands in the neuromarketing segment, I can relay there will be a large crowd of young people coming into neuromarketing from that field. The Society of Neuroscience is the fastest growing scientific research organization in the past two decades and the investigations / applications are now coming from research to the market. That wave is coming into market research. It will not replace existing methodologies but will be an important adjunct to provide a more complete picture on consumer insight. Ron Wright CEO Sands Research Inc.

Like Report