This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Find out more here

Sunday, 23 November 2014

Experimental phase

My first job upon graduation was as an academic researcher, schooled in the art of experimental design, mindful of the need for testing hypotheses on an a-priori basis, and knowledgeable about univariate statistics such as t-tests and Chi-squares. So imagine my shock when entering the market research industry to be faced with designing surveys, not experiments, and combing sets of data to answer clients’ business issues rather than testing specific hypothesis.

Of course, as market researchers, we develop an incredibly nuanced understanding of the way in which consumers are acting and thinking. We can use this to deduce the reasons why certain behaviours are exhibited. And the multi-faceted nature of market research allows us to properly reflect the messy world in which we live rather than attempt to artificially isolate one variable at a time to see its impact. I gradually came to understand the benefits of this more pragmatic approach.

“Maybe this is a point where the market research industry needs to reflect on its heritage in the social sciences, where one of the key tenets is to design experiments with the express purpose of testing hypotheses”

And yet we are still faced with knotty problems for which our market research approaches aren’t always the most effective at answering; that may even – at times – obscure the answer. To give an example: recently we wanted to understand which of a number of approaches was the most effective way for salespeople to present information about different mobile devices. It turned out that the most sensible way to uncover the answer was to create an experimental design, isolating the factors that we wanted to explore rather than collecting what would be hard-to-capture data over a longer period of time using surveys.

Experiments are not a recent innovation, essentially being the cornerstone of the scientific method for hundreds of years. In some business categories, such as direct mail, this is familiar territory and indeed market research uses experimental approaches in some areas such as advertising pretesting. But the logistics of running experiments have historically been considered onerous and expensive, requiring investment of relatively large sums (particularly if we wanted large and representative samples) to test quite specific phenomenon.

The growth of the internet helps to counter this, as it is becoming ever easier to undertake randomised controlled experiments quickly and cheaply. Many companies now routinely undertake ‘bucket testing’ to optimise their website – presenting slightly different formats and propositions to their customers and gauging which is most effective.

Datafication, the way we transform our environment into a quantifiable data format, means we have many more means at our disposal to create experimental designs. At GfK, for example, we have created a design prototyping tool which allows consumers to see high fidelity graphical images of objects (devices, packaging etc) on screen from a variety of different angles. Consumers are presented with subtly different designs and their reaction times tested to obtain implicit preference scores. 3D printing is also used to produce a physical representation of the original prototypes and revised versions which can be tested in the same way. Of course, while this is not a ‘real life’ controlled trial (the next step in the process), technology allows us to undertake experiments to get much closer to an approximation of the real world that otherwise simply would not be possible.

Testing times

While technology now makes it much easier for us to undertake experiments, the business environment is also more favourable towards them. First, as business challenges get tougher trying to find competitive advantage gets harder. Experiments allow us to identify consumer insights that would be hard to achieve by other means. Many of the insights around consumer behaviour that behavioural economics point to, for example, are often only identifiable through the use of experiments. Second, experiments provide a level of proof that may not always be possible from more mainstream market research data, where causality may often be inferred from the data rather than determined from it.

Maybe this is a point where the market research industry needs to reflect on its heritage in the social sciences, where one of the key tenets is to design experiments with the express purpose of testing hypotheses. The advance of technology and online research platforms described here provide an opportunity to return to the rigour of this social science legacy: where we state our objectives, formulate tangible hypotheses to reflect those objectives and then design experiments to test hypotheses.

Colin Strong is managing director of GfK’s technology division in the UK

Follow us on
Follow us on Twitter

Readers' comments (1)

  • Colin, great suggestion, to build a body of knowledge through the systematic formulation and testing of hypotheses. Better yet, how about starting this process by understanding the underpinnings of purchase decision-making, rather than removing ourselves from the transaction?

    Unsuitable or offensive? Report this comment

Have your say

Please add your comment. You can include links, but HTML is not permitted.
Your email address will not be displayed on the site. All comments are moderated.

Mandatory
Mandatory
Mandatory
Mandatory