This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Find out more here

FEATURE10 September 2010

Meet the man who wants to make surveys history

Features

In his book Consumer.ology former market research manager Philip Graves argues that companies who buy surveys and focus groups have been fooled by ‘pseudo-science’ and that true consumer insight can only come from observing and analysing behaviour. He talks to Brian Tarran.

Graves, you’ll gather, is no big fan of market research. More specifically, he doesn’t go in for the concept that you should ask people questions, analyse their answers and use that information to make crucial business decisions.

So committed is he in his belief that he’s written a book about it, Consumer.ology [sic], in which he seeks to undermine once and for all the “pseudo-science” of market research. “My point in writing the book is to say that at the moment businesses, organisations and politicians are putting a lot of faith in something that is extraordinarily haphazard as a tool,” says Graves. “Is market research always wrong? Of course not. Is your horoscope always wrong? No.”

On the road to ruing
It wasn’t always like this. Graves was once a fully paid-up member of the market research community. Brand strategy and research manager at Pizza Hut, market research manager at Eagle Star. “I worked for a couple of years for a traditional market research agency, a small one, specialising at that time in financial services research. So yes, my background into research was fairly traditional,” says Graves.

What started him down the path to where he is today was a focus group he was conducting in Leeds on personal pensions. “I was with this group of men and they told me they wanted a range of funds, financial security, independent advice – all the same things that were coming through from the other groups I had done.

“At the end of the group, after I’d turned my tape recorder off and put my notebook away, one guy asked another guy: ‘So what are you going to do about getting a pension?’ And he said: ‘Well, we’ve got this guy who comes round our house who deals with all our insurance stuff so I’m going to speak to him about it.’ I realised what he was describing was the man from the Pru – he was describing an industrial insurance salesman, with no reference to financial security, no sense of what range of funds would be available, certainly not getting independent advice. And the other guys started asking for the salesman’s number. And so off I went to write my report about independent advice and financial security and the like, and off they went to do something completely different.”

“My point in writing the book is to say that at the moment, businesses, organisations and politicians are putting a lot of faith in something that is extraordinarily haphazard as a tool”

After this “initial trigger”, as he calls it, Graves started to study psychoanalysis and transactional analysis. “It dawned on me that all these people who go through a process of psychoanalysis do so because they can’t get to the bottom of their own thoughts,” says Graves. “That then opened my eyes to the extent to which we are not very good at explaining and understanding our own thoughts. Then, when you start to look at what does make a difference in terms of how people behave, there is the unconscious mind and the emotional brain that is shaping decisions we take, and we don’t have a direct connection to it. We become awful witnesses to our own behaviour, and also hopeless at identifying what it is we do and don’t want.”

This won’t come as a surprise to many researchers – it is widely accepted that what people say they’ll do is often very different from what they actually do. This explains why attitudinal data is often analysed alongside behavioural data. Graves’s question is why people bother with attitudinal data at all? In his view there is no place for it in the decision-making process. “I would put it all to one side,” he says. “I would do away with it.”

Telling tales
While Graves’s view of research is likely to enrage certain sections of the research community, it’s possible that his animosity towards asking questions of consumers will stand him in good stead with those who argue that the role of research, especially in the social media world, should be to listen to consumers and let them steer the conversation. Graves himself is certainly less opposed to the concept of listening – though it’s not without its drawbacks, he says.

He cites fashion retail chain Zara as a “great example of a company who really listen to their customers”. “In Zara stores,” he says, “one of the tasks their managers have is to listen to what their customers tell them. So if someone comes in to complain that a belt is a bit short, say, that’s a valuable piece of feedback and they as a business are geared up to channelling that information back into the marketing, product design and product manufacturing base.”

Listening in to online conversations, though, is merely “OK”. “It’s a bit like listening to what people are saying about you in the playground. You need to understand the psychology of human communication,” says Graves. “For example, if something bad happens to me as a consumer I’ve now got a story to tell and it’s going to have all the elements of story – a good guy, a bad guy, a setting, a turning point and a hero. And I’m going to tell that story because of how it will make me look and because it’s a way of conversing that I know will engage people.”

The act of telling a story actively primes others to share their own tales and before long, Graves says, “we’ve got everybody in the playground sticking the boot in”. Companies, he says, should be alert to this, and ready to go in, deal with negative sentiment and attempt to create a positive story for its customers to tell. “But the notion that what people choose to talk about is representative of what they do” – and thus contains insight to explain customer behaviour – “just doesn’t bear scrutiny to me,” says Graves.

Where do we go from here?
So if asking questions is off the table, and listening isn’t all its cracked up to be, what are companies left with? Graves advocates observational research – watching what consumers do and analysing their behaviour.

If a company is developing new products, he says, prototyping and live testing of concepts on store shelves is the way to go. This, we suggest, might be a tough sell commercially. Companies would have to get quite far along in the product development process before they realise they have a dud on their hands. Surely that creates a bigger risk of more expensive failures than a few concept testing surveys would – no matter how flawed Graves says they are.

“We’ve been seduced by this illusion that we can ask people what they think and learn from it”

He disagrees of course. “Testing new concepts with consumers is so bad as an exercise. Companies feel that they are minimising their risk by doing that research and it’s giving them confidence to go forward, but in reality they are just as likely to be ditching a really good idea at that stage,” says Graves.

“If you are sensitive to loss, as a lot of businesses are, be sensitive to the loss of just kicking out a perfectly good, perfectly marketable idea because the process you are using cannot give you any reliable sense of whether people are going to buy it.”

It’s at this point Graves points to a list of brands that have launched on gut instinct and been successful, despite having scored poorly in research tests: Red Bull, Absolut Vodka in the US, Baileys, the Heineken ‘refreshes the parts’ campaign. “Market research was all for stamping on them and throwing them out the door,” he says.

These and other so-called “market research failures” are well documented. But it is, I suggest, an unfair argument. The story of creative genius triumphing despite the warnings of customer experts makes for an engaging tale, and reflects well on the teller. If, as Graves said earlier, people choose to tell a story because of how it makes them look, it’s hardly surprising we hear this sort of story more than the one in which the humble survey led to the development of a successful product.

Researchers looking for a ‘good news’ story to tell might point to the continual growth of their industry in the 60 or 70 years since it emerged. Surely, they might say, an industry responsible for more misses than hits wouldn’t have enjoyed such a run of success.

No one would argue that the survey is a perfect tool – nor for that matter is a depth interview, or a focus group, or any other method that involves asking respondents questions. Asking questions invites focalism and encourages people to post-rationalise, neither of which are helpful in understanding the consumer’s true sub-conscious decision-making process.

Researchers are well aware of the limitations of their range of tools. What we’re left to ponder is whether the companies who pay them to wield those tools are equally knowledgeable, for Graves’s prediction of an end to the old paradigm of market research depends on those buyers.

He says: “We’ve been seduced by this illusion that we can ask people what they think and learn from it.” But there’s every possibility that companies have really been seduced by the fact that asking people is cheaper and less time-consuming for companies than the alternative methods of research – no matter how much more reliable they might be.

‘Consumer.ology, The Market Research Myth, the Truth about Consumers and the Psychology of Shopping’ by Philip Graves is published this month by Nicholas Brealey Publishing.

23 Comments

7 years ago

As someone who has conducted thousands of interviews in the name of market research I fully agree with Philip Graves.

Like Report

7 years ago

It's actually the same debate that has gone on for the 25 years I have been a professional researcher. How do we understand what people do, rather than what they say they do? I believe that sometimes asking questions to help understand behaviour has a role to play (in understanding and predicting behaviour), although I agree with the critics who say we waste money on surveys focusing on attitudes. I also think that much analysis of online conversations, eg on Twitter, tells us little about behaviour (in fact Twitter has gone backwards from 'what I'm doing now' to 'what I'm thinking now') but we are getting genuinely new insights through new online methods of observing and analysing behaviour - or online behaviour at least. And we have to listen to these commentators like Martin Lindstrom and Philip Graves don't we? As Maureen Lipman's character in the BT ads used to say - they've got an ology!

Like Report

7 years ago

The initial "Eureka" moment here could seem slightly flawed in that Philip instantly assumes that the "guy who calls round the house and deals with all the insurance stuff" is a man from the Pru type figure who is offering only one range of products. We too have a man who calls round the house to deal with these things but he is an Independent Financial Adviser who offers us a whole range of products/services uniquely tailored to, and fully meeting, our requirements. Maybe I'm wrong but has poor research led to writing a book about how research/surveys are not going to be needed in the future?

Like Report

7 years ago

Reminds me of some work Malcolm Gladwell's done on the nature of choice and consumer selection. Sometimes consumers just don't know what they want in a product. http://www.ted.com/talks/malcolm_gladwell_on_spaghetti_sauce.html

Like Report

7 years ago

It is interesting that Graves' "initial trigger" took him to the path of dismissing asking questions instead of questioning if he was asking the right questions. I guess that would have been a tougher road to take for him. I have seen the birth of very successful products based on asking questions about both attitudes and behaviors. We need to know not only what people do, but why they do it. They might not always be able to formulate what they want in the future, but it is our role as market researchers to tease that out with the right questions and combinations of techniques. Unfortunately so many bad surveys have been created particularly in the era of cheap online survey tools when everybody thinks they can write one, that they have overshadowed the outcome of really good and professional research. Hopefully, as the recession eases up and research budgets come back to life again, companies will be able to invest in quality research and we can see the death of bad surveys. Michaela http://www.relevantinsights.com

Like Report

7 years ago

This might be oversimplifying, but it sounds to me like this will take one existing methodology, which is admittedly flawed and replace it with another methodology, which will also be flawed.

Like Report

7 years ago

As someone who has done 600+ market research projects over 25 years using various techniques, I can tell you the following: 1. Surveys are not perfect, and alas are often used when other techniques would yield better insights. 2. Surveys work very well (are reliable and accurately predict behavior) in some markets/customer groups more than in others. Anyone doing research for more than 5 minutes can tell you that, including Phillip I'm sure. 3. Observational methods are not perfect, can be very misleading, and simply don't apply to many research situations. 4. In-depth interviews and focus groups have their own set of limitations--using them (or any method) inappropriately will yield misleading results. 5. The bigger issue is over-relying on any single method--there is an appalling lack of triangulation and contextual consideration in many research projects. Bottom line: no market research method is perfect, and buyer beware of any author/salesman proffering a one-size-fits-all solution.

Like Report

7 years ago

Yes this has been a debate for a long time but I think Philips long term projection is worth considering in a world now awash with lots more real behavioural data. Traditional MR (asking what people need, want, like, might do) has a part to play in the decision making but it will be an increasingly marginalised input alongside the real/hard information. So the balanced perspective Kathyrn outlines will be even more balanced by inputs external to what MR has the ability to provide. So what would an MR industry look like if no new data was collected & researchers had to just use what data already exists?.

Like Report

7 years ago

In its best manifestation, the prinicpal tool in qualitative research is the qualitative researcher, not the questions that she asks. Qualitative research is not a simple process of transferring information from one place to another and totting it all up - 2+2 = 4. this is precisely why it is different from quant. It's an exploration by a creative map maker who gives us the colours and shapes to describe the new world she finds. Perhaps the key to his views is in the apparently very limited experience that Graves had as a researcher.

Like Report

7 years ago

“In 20 or 30 years,” he says, “we will look back with genuine embarrassment at what we currently do." ? Actually, to misquote Keynes (?), in 20 or 30 years, we will all be dead.

Like Report

Results 1 to 10 of 23