FEATURE1 November 2009
FEATURE1 November 2009
What should researchers make of the proliferation of high-tech methods for measuring people’s thoughts and feelings? Are the really smart researchers the ones sticking with the low-tech approach? Tim Phillips investigates
When Spring Research was looking for a tool to capture the emotional response of its subjects to advertising campaigns, it could have used galvanic skin response monitors, or placed its respondents in EEG skull caps, or put them inside an MRI scanner while they stared at a tiny screen for an hour.
Instead, says Steve Phillips, they gave them the Emotimeter – that is, a knob to twiddle and some emoticons to select.
The knob allowed participants to vary the rate of a simulated heartbeat, to record their feelings of arousal. The emoticons were a proxy for the type of emotion they were experiencing – useful across linguistic and cultural boundaries. In short, says Phillips, the Emotimeter is a quick and inexpensive way to test advertising – far from perfect, but a huge improvement on the existing qualitative techniques he was using.
“A lot of creatives moan about focus group research, saying that people in the real world don’t spend two hours discussing ads,” he says. He wanted a test that would capture gut response without asking the respondents to rationalise it, but realised that the test didn’t need to be complex. “We don’t want to get too complicated. We wanted a simple visual mechanism, something that was as consumer-focused as we could get.”
“Market researchers are increasingly using neuroscience to make sense of what’s going on in our unconscious mind by asking our conscious mind what’s happening”
Creatives were not looking for exact results, just a steer, Phillips says. “The Emotimeter was created to be simple and cost-effective… I see this struggle [between technology and simplicity] all the time in research, but in our business we are really focused on keeping it simple.”
The Emotimeter notwithstanding, neuroscience is growing swiftly as a component of research, despite its expense. Market researchers are increasingly using neuroscience to make sense of what’s going on in our unconscious mind by asking our conscious mind what’s happening. Phillips estimates that while 20 years ago advertising was a 50-50 split between emotion and information, now emotion is 95% of the message. Knowing when to stimulate emotion is a tough challenge for traditional research, but it’s the bread and butter of neuroscience.
Agencies and their clients alike are enthusiastic, even if sometimes their enthusiasm outstrips their ability to produce insight from these methods. “Some customers have come to the conclusion that MR methodologies are far too old – they start looking for technology and then just buy it because it is different,” says Per Nystedt, content manager at eye-tracking firm Tobii Technology.
Nonetheless he points out that in many areas – for example usability testing – the returns on investment for his technology are well-established. And he also defends the agencies and their clients who are pushing the technology into areas where the evidence for its effectiveness doesn’t yet exist, as long as they are honest about it. “Some market research companies exist at the cutting edge. They capitalise on technologies that haven’t been proven. But we all need the early adopters because they are the ones that explore the technology, they are the ones who invest in discovering whether those gut feeling results are valuable.”
The problem is, even if the results aren’t as robust as whatever the “far too old” methods would have produced, bad data still looks very much like good data.
The most rigorous of the neuroscience research agencies are frustrated that their peers often overclaim and underdeliver. “We take pride in how we apply the technology… We work hard to create a reliable base of evidence,” says Carl Marci, who combines his role as CEO and chief science officer at Innerscope with a part-time faculty position at Harvard Medical School as an assistant professor as well as a departmental headship at Massachusetts General Hospital.
Innerscope uses a special vest to collect biofeedback from subjects to measure emotional intensity in response to a stimulus as it happens. The firm claims it was able to predict the buzz generated by Super Bowl advertising with far greater accuracy than traditional methods. But Marci worries about the indiscriminate use of technology. “You really have to know what you are doing with this stuff. Eye-tracking on its own will pick up the blank stare. My concern is with companies that engineer their own devices and don’t do it with a degree of rigour.”
Although neuroscience typically uses a small test sample size – usually fewer than 50 subjects – the research is expensive. For Marci that means that it is best suited to answering big questions with maximum potential impact. Unfortunately the technology is sometimes presented as a ‘black box’ to solve the problem, simply because clients do not have the background in science that they might have in traditional research techniques, and find it hard to make an assessment of the methods being proposed. “You have, for example, a beer company doing advertising research and all of a sudden they are talking to neuroscientists and taking advice,” he says.
Accredit where it’s due
Innerscope has responded by putting its methods through an examination by the Advertising Research Foundation’s accreditation process, the first neuroscience-based firm to do so. Neurofocus, which also brings high-tech methods to the traditional research process, attempts to do a similar job by explaining the rigour of the research techniques carefully before attempting to sell their services. “There is a lack of conventional methodologies to predict human behaviour,” says Thom Noble, Neurofocus European managing director and a former FMCG brand manager. “Increasingly I found myself working on global brands which had a very high emotional quotient. The product differences were very small, but trying traditional research to get a handle on how your customers engaged with you – that was hard.” While the cost of neuroscience-based research may still make it the preserve of the large FMCG clients on Neurofocus’ and Innerscope’s lists, using research to predict the future is relevant to every client. So what’s the role of traditional MR skills?
According to Martin Hayward, the director of strategy and futures at Dunnhumby, insightful, clever ideas are more important now than ever – simply because hi-tech methods are swamping us with data. “There are already many organisations that have managed to amass huge amounts of data and they can’t do anything with it, he says. “You need intuition and you need to know where to look in that data.”
“For the first time the amount of detail that is available and our ability to get it exceeds our ability to deal with it”
Martin Hayward, Dunnhumby
Join the club
Dunnhumby knows the value of intuition, having built its business on its analysis of Tesco Clubcard data. Its breakthrough was realising that it did not have to build the giant data warehouses that Tesco’s rivals were building in the late 1990s. Better to ask the right questions, then find the cheapest and quickest way to get a reasonably robust answer. Dunnhumby says its analysts found results using only around 5% of the Clubcard data – answers that were robust enough to produce directed action quickly. The ability to ask the right questions and react quickly to the answers is still paramount, Hayward says. In that case, the ability to use technology to visualise data we already have might be more important than collecting more of the stuff.
“For the first time the amount of detail that is available and our ability to get it exceeds our ability to deal with it. Organisations already can’t make decisions as fast as they are getting the data. No client can say that they haven’t got insight already. It arrives on their desk every day. We’re still using bar charts and pie charts, there’s just 200 pages of them now. There’s lots of short-term excitement about bells and whistles but most of the decisions we are making are about pretty mundane stuff.”
The result is that “we all drift towards simplicity and clarity”, Hayward says, which raises the ultimate problem of technology: it doesn’t help unless it leads to beneficial decisions that would not have been otherwise made. Becoming better measurers doesn’t help if we measure the wrong stuff. Hayward is frustrated that some of his clients still obsess on the wrong measures of success – for example, valuing short-term emotional response from customers rather than long-term loyalty.
Vince Nolan, owner of 2CV, is “a big fan of technology in general,” but also worries that hi-tech methods might obscure insight rather than creating it. “I’ve been researching videogames for 15 years, and I was the first to use SMS as a tool, so I’m certainly not a Luddite,” he says, “The problem is that we get very siloed into advocating one technology to gather information. In the excitement of a new technology people can lose sight of the fact that we might not need to generate even more information. There is already information around everywhere. You need to know where to look for it.”
He warns that when an agency becomes wedded to one technology-based method of collecting data, it naturally advocates it to clients, whether or not it will be better than existing methods. “Online we already record a lot about people’s behaviour that isn’t being examined. In a grown-up research industry we will accept we have spent too much of our time getting more information.”
Nolan’s final objection to an infatuation with hi-tech echoes Hayward: “We are looking in the wrong place. The space between us is ultimately more important than the space inside us. How our minds work is only a small part of the story. We need to accept that we are social animals, and that our decisions are driven by more of a group than an individual mind,” he says. “I know client heads are easily turned by technology, because it’s exciting and new. So there is a greater degree of attention paid to embryonic methods compared to the traditional low-technology methods. There is a tendency to want to see the end of what we call traditional market research. It comes in for a terrible hammering.”
Ultimately hi-tech methods will find their place in the ever-changing list of techniques that researchers use. But just as telephone research, the internet, focus groups, semiotics or any of research’s innovations cannot survive without the insight of the people who perform them, so no one is claiming that a market researcher’s ingenuity is outdated. Well, almost no one. “A client was joking with me the other week – ‘We don’t need you Vince, we’ll soon be able to see the information you give us without your help’,” says Nolan. “But I think reports of our death due to technology have been exaggerated.”
?Eye-tracking: A camera follows and records the direction and duration of a person’s gaze.
Pros: It’s quick, relatively inexpensive and portable.
Cons: The fact that someone is looking at something doesn’t tell us what their emotional response is.
Galvanic skin response: Sensors measure the electrical conductivity of the skin, which indicates emotional arousal.
Pros: It’s non-invasive, inexpensive, can be measured over a long period of time and gives immediate feedback to stimuli.
Cons: It doesn’t distinguish between types of arousal.
EEG: Electroencephalography records electrical activity at the scalp from the firing of neurons.
Pros: There’s no delay in response – effects are recorded as the emotion occurs.
Cons: You have to wear a carefully applied skullcap with electrodes attached, the signal is susceptible to noise, and it only measures the surface of the brain.
PET: Positron emission tomography produces a three-dimensional image of metabolic processes in the brain.
Pros: It shows changes taking place in the brain as they occur.
Cons: The equipment is expensive, and you’ll have to persuade your subjects to be injected with a radioactive isotope.
MRI and fMRI: A large doughnut-shaped scanner that record blood flow in specific areas of the brain.
Pros: It maps activity in particular parts of the brain, so can distinguish rational and emotional responses. Resolution is better than PET.
Cons: Apart for the multi-million pound cost of the scanner, subjects can get uncomfortable and stressed – and they’re not allowed to move.