This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Find out more here

OPINION15 February 2013

Econometrics: an able servant, but never the master

Opinion

Camelot Global commercial director Richard Bateson reminds us that econometric models are only as good as the data that goes into them, and more evidence is always needed to prove the return on marketing investment.

But in this extract from the IPA’s Advertising Works, Camelot Global commercial director Richard Bateson reminds us that models are only as good as the data that goes into them, and more evidence is always needed to prove the return on marketing investment.

“For many marketing-focused organisations, econometrics can be both a godsend and an albatross. In many ways a good econometric model can be the holy grail, separating out the key influences on sales shifts and allowing you to evaluate the return on marketing investment (ROMI) of factors as diverse as TV advertising, promotions, the weather, the economy and competitor communications – who wouldn’t want one?

When looking at the effect of past activity, results should align with your gut instinct, you should be able to see the same indicators in your other tracking tools, and even qualitative consumer research should back up your findings

Another dimension of econometrics I find particularly powerful is the ability to look across a diverse brand portfolio and understand the interdependencies of each of your products, halo effects and even potential cannibalisation. Many marketers I have worked with have found modelling to be hugely informative as a marketing tool, both as a measurement tool in understanding the effectiveness of past activity, but also as a planning tool for future activity.

Within the National Lottery, econometrics is particularly important as we need to be answerable to our players and all our stakeholders as to where we invest our budgets and how much we return to National Lottery Good Causes for every marketing pound spent. It cannot answer all questions, but does provide one strong tool within our measurement suite.

And therein lies the rub – I believe that econometrics is only one tool and should not be treated as gospel or looked at in isolation. When looking at the effect of past activity, results should align with your gut instinct, you should be able to see the same indicators in your other tracking tools, and even qualitative consumer research should back up your findings.

My main concern with econometrics is the danger of thinking ‘if it says it in the model it must be true’ – models are only as good as the data put into them, the skills of the analyst and the other information available in order to look at the outputs holistically. Lots of murky factors can be hiding in the base figure and, perhaps more importantly, ground-breaking new activity which is driving a valuable ROMI into the business can be lost because it is not robust enough to be picked up in a model which learns from the past and from deviation.

My second concern is therefore that dependency on models which tell us what worked in the past can make us risk averse in the future. Few clients would take a leap into social if all eyes are on the econometrics model, the Orange association with cinema probably doesn’t stack up in the world of econometrics, and no-one would ever develop 90-second films. Marketing still needs bravery alongside the science bit.

So my words of advice to any potential author of an IPA Awards entry in the future are: use your econometric findings wisely; do not let jargon dominate; do not fall into the trap of thinking it is the only proof you need; explore all avenues to make your case stack up; don’t use econometrics as your end goal – use it to say ‘the model suggests this is what is happening, do we have more evidence to support that?’.”

Advertising Works 21, published by Warc, showcases 24 prize-winning 2012 IPA Effectiveness Awards papers in full, including John Lewis, Department for Transport, Walkers and Yorkshire Tea. It’s available to order online here.

1 Comment

7 years ago

Couldn't agree more! When I started studying econometrics over 20 years ago, much was drilled into me about the inaccuracy of models with academic papers stating that, at best, econometric models may only explain 50-60% of the market situation. Getting one of mine to accurately predict a market to a level of 82% seemed, at the time, to be unusual, if not suspicious! One of the things that I did learn at the time, though, was that the obvious elements of the models, highly correlated but of dubious causality quite often led to inaccuracies and that constructors of models do need to look at and test second and third levels of influencing data, (i.e. what influences the chosen factor or metric and what influences that) to try and model a situation. In my own experience 82% felt pretty good but in these days of being 99% certain before embarking on an investment, too much reliance on this technique can blinker the marketer at best or ruin a brand at worst.

Like Report