A different order

x Sponsored content on Research Live and in Impact magazine is editorially independent.
Find out more about advertising and sponsorship.

While I was growing up, my father ran a series of small businesses. No-one in my family had ever worked in a large organisation – they were mostly farmers, rural GPs or suchlike. So, during my first few years at Ogilvy, I assumed big businesses operated much like small ones, only on a larger scale. This, I quickly learned, is far from the case.

In my first month as a trainee, I spent two weeks in the creative department of what was then called Ogilvy & Mather Direct. One of my tasks, working under an experienced American copywriter, was to write a solicitation letter, cross-selling a fairly obscure financial product. Call it beginner’s luck, but the letter was quite a success; the response rate was twice that of the control and the ROI was 5:1 in the first six months alone (it was a fairly profitable product).

“Wow,” I said. “Let’s mail some more people with the same letter.” I was pretty certain that if you had shown my father any sure-fire way to make £5 for every £1 he spent, he would have done quite a lot of it.

“No,” it was patiently explained to me. “It doesn’t really work like that in big organisations.” The client had been allocated a budget and a target for this piece of activity, and they had spent their budget and exceeded their target. So there was nothing more to be done. “Maybe next year.”

This seemed strange to me. What is the point of measuring anything if the results do not – indeed, cannot – change what you do next? If it doesn’t alter subsequent actions, measurement is merely a glimpse into the rear-view mirror. It doesn’t affect where you’re headed.

It now occurs to my older, more cynical self that measurement is often seen more as an obligation than an opportunity, and that much of what passes for ‘marketing accountability’ is arse-covering or blame-avoidance performed under the guise of rigour. The same, presumably, goes for market research when it is used ‘for support rather than illumination’. Perhaps the majority of research or measurement is commissioned and used not to inform decisions in advance, but to defend them in retrospect. This can be a massive waste of time and effort.

But what if you could use behavioural science to solve this problem? I believe you can; all you need to do is change the way you present findings, and inject a forced question at the beginning of the process. This is what Richard Thaler, the author of Nudge, calls ‘managed choice’.

What we typically do when we present results is to say, ‘here are the results, now what do you plan to do?’ The problem with this approach is that it allows any decision-maker to do what they were planning to do anyway, with cherry-picked parts of your research tacked on to lend confirmation bias. If this happens, the research or measurement is being used defensively. It has not changed behaviour – all it has done is allow someone to defend a course of action they were planning to take anyway.

If, before presenting the results, we asked ‘what do you plan to do if the research reveals X, Y or Z’, our research or evaluation becomes a great deal more influential.

Let’s go back to that mailing I wrote in 1988. What if we had asked the client before the mailing – or in advance of presenting the results – ‘What will you do if the ROI is a) 1:1 or less, b) 2:1 c) 3:1 d) 4:1 or e) 5:1 or higher?’ How much more likely are they to say: ‘Well, with 4:1 or more we’d go and find some extra budget?’

Now you’ve changed the game. You’ve reduced confirmation bias and created conditions under which your research findings might change what the company does in response. At the very least, it would be awkward for them to say, regardless of the results, they would do exactly the same thing.

Market researchers already know about ‘order effects’ – the extent to which the sequence in which questions are asked changes the answers people give. In this case, by simply reversing the order in which research is presented, we might be able to make it more powerful.

Human behaviour and decision-making is path dependent. Often, the most important – and under-used – technique in changing behaviour is simply to experiment with doing the same thing in a different order. If you sign a tax form at the top rather than the bottom, you are more likely to tell the truth. 

Rory Sutherland is vice-chairman, Ogilvy & Mather, UK