Econometrics: an able servant, but never the master

But in this extract from the IPA’s Advertising Works, Camelot Global commercial director Richard Bateson reminds us that models are only as good as the data that goes into them, and more evidence is always needed to prove the return on marketing investment.

“For many marketing-focused organisations, econometrics can be both a godsend and an albatross. In many ways a good econometric model can be the holy grail, separating out the key influences on sales shifts and allowing you to evaluate the return on marketing investment (ROMI) of factors as diverse as TV advertising, promotions, the weather, the economy and competitor communications – who wouldn’t want one?

Another dimension of econometrics I find particularly powerful is the ability to look across a diverse brand portfolio and understand the interdependencies of each of your products, halo effects and even potential cannibalisation. Many marketers I have worked with have found modelling to be hugely informative as a marketing tool, both as a measurement tool in understanding the effectiveness of past activity, but also as a planning tool for future activity.

Within the National Lottery, econometrics is particularly important as we need to be answerable to our players and all our stakeholders as to where we invest our budgets and how much we return to National Lottery Good Causes for every marketing pound spent. It cannot answer all questions, but does provide one strong tool within our measurement suite.

And therein lies the rub – I believe that econometrics is only one tool and should not be treated as gospel or looked at in isolation. When looking at the effect of past activity, results should align with your gut instinct, you should be able to see the same indicators in your other tracking tools, and even qualitative consumer research should back up your findings.

My main concern with econometrics is the danger of thinking ‘if it says it in the model it must be true’ – models are only as good as the data put into them, the skills of the analyst and the other information available in order to look at the outputs holistically. Lots of murky factors can be hiding in the base figure and, perhaps more importantly, ground-breaking new activity which is driving a valuable ROMI into the business can be lost because it is not robust enough to be picked up in a model which learns from the past and from deviation.

My second concern is therefore that dependency on models which tell us what worked in the past can make us risk averse in the future. Few clients would take a leap into social if all eyes are on the econometrics model, the Orange association with cinema probably doesn’t stack up in the world of econometrics, and no-one would ever develop 90-second films. Marketing still needs bravery alongside the science bit.

So my words of advice to any potential author of an IPA Awards entry in the future are: use your econometric findings wisely; do not let jargon dominate; do not fall into the trap of thinking it is the only proof you need; explore all avenues to make your case stack up; don’t use econometrics as your end goal – use it to say ‘the model suggests this is what is happening, do we have more evidence to support that?’.”

Advertising Works 21, published by Warc, showcases 24 prize-winning 2012 IPA Effectiveness Awards papers in full, including John Lewis, Department for Transport, Walkers and Yorkshire Tea. It’s available to order online here.