Social dominance

Gareth Price (above left) was the author of the first piece. He’s a researcher working in the brand insight division of Precise, and is responsible for developing the company’s social media research offer. His article was based around the findings of a Pew study which showed that traditional surveys and social media sentiment often diverge when it comes to measuring things like voting intention. Price’s argument, then, was that researchers need to accept that social media is different from traditional research and can never directly replace it.

Then there was Mark Westaby, the founder of SpectrumInsight. He sent us a blog post based on some internal research that found that Twitter-based ‘social’ insight correlates closely with traditional market research surveys regardless of demographics. Mark’s argument was that this finding opens the way for Twitter to be used as a legitimate market research tool over periods for which traditional market research would be prohibitively expensive, as well as enabling insights to be captured that would otherwise be missed.

We asked them to go head-to-head to debate the issues and answer the question once and for all: Can social media analysis ever be a replacement for traditional market research?

GP: Can it complement traditional methodologies? Absolutely. Replace? In some circumstances it might enable us to answer a question that’s otherwise difficult to answer. Will we ever be in a position where we no longer need to ask questions of respondents and collect all information passively? I don’t believe so.

At present, the people using social media don’t make up a representative sample of the population, so it cannot replicate a quantitative tracking survey, for example. It can help explain why opinions towards a brand may have changed. However, if we need to quantify the size of an issue, social media research can only do that in a specific context and can’t tell us how this is reflected among particular demographics and the population at large.

The unstructured nature of the data means we can’t always make it do what we want it to do either. Because of this, you have to think about it in more of a qualitative way. Even if this is at scale, if you’re advocating that it replaces all forms of research, you’re effectively making the argument for no longer doing quantitative research. Social media research informs traditional methodologies. It’s a holistic process.

MW: In many circumstances, social media analysis, which I’ll call ‘social insight’, isn’t just a replacement but is actually vastly superior to traditional market research for reasons that will become obvious.

Your opening salvo, Gareth, throws up that old favourite of the ‘representative sample’, which is how every traditional market researcher tries to attack social insight. However, what exactly is a ‘representative sample’ and what are such samples really representative of? The answer certainly isn’t ‘the real world’ because real people don’t behave in the nice ‘representative sample’ way that traditional market researchers love so much.

On any single day the ‘representative sample’ of people who use Tesco will not be the same as on the next day, or the day after; and that goes not only for every other supermarket but also for every single brand the supermarkets sell.

Furthermore, a ‘representative sample’ in itself is going to provide very limited insight. Knowing what the population might do is only going to tell me about Mr and Mrs Average. But it’s Mr and Mrs Not-Average that I really want to understand. But as soon as we drill down into our ‘representative sample’ to try to find out about Mr and Mrs Not-Average, our statistically significant data becomes not very statistically significant.

This is the fundamental problem with traditional market research: it’s limited to the samples it can collect. And the problem with samples is that their statistical significance is limited to the population they represent which, as we’ve seen, aren’t really very representative of what actually happens in the real world.

Social insight provides an ideal way of addressing all of the the above problems. For starters, let’s take the Tesco example used earlier. We track tweets from millions of unique supermarkets users, who in fact represent between 1 in 10 and 1 in 20 of all consumers who ever use a supermarket. With these numbers we’re not just tracking a sample but the population itself.

Furthermore we’re doing so on a 24/7 basis, which means we can track what consumers are thinking and doing literally by the hour.

“A ‘representative sample’ in itself
is going to provide very limited insight.
Knowing what the population might do
is only going to tell me about
Mr and Mrs Average. But it’s
Mr and Mrs Not-Average that I really
want to understand”

GP: I’m confused on a couple of points, Mark. You seem to be arguing that extracting data from a representative sample of the population no longer matters. In certain instances that may be the case but in polling, for example, it’s vitally important. The main reason Nate Silver has been so successful in predicting elections is that he’s able to aggregate lots of polling data that’s representative at the local and national level.

I think you’re wrong to say that you can’t drill down into a representative sample either. As long as you achieve results above a statistically significant level, you can talk about and investigate the implications in more detail (e.g. understanding attitudes by demographic – ABC1s v C2DEs, women vs. men, etc.).

If we’re talking about the fact there isn’t a ‘Mr and Mrs Average’ anymore, and everyone is a sample of one, then in-depth qualitative research is the only way to understand their behaviour. This can take the form of ethnography, observing people in store, using eye-tracking or even running focus groups. If you look at an agency like BrainJuicer they use all sorts of innovative ways of understanding a truer-to-life model of human behaviour. This can also take the form of social media analysis but at a very granular level.

I think we’re talking about quite a hypothetical level of social media use too. To suggest that lots of people are using these networks to convey every aspect of their shopping experience is either naïve or wilfully ignorant.

Social media research can help us answer some of the questions some of the time. Understanding the results it delivers means understanding the context in which people share experiences. It also means understanding in which context it works and in which context we need to turn to a different methodology.

MW: Regarding your comment about getting away from Mr and Mrs Average, I’m far from talking about a “sample of one”. Indeed, my point is entirely the opposite: social insight enables us to understand how Mr and Mrs Not-Average behave because it gives us access to lots of Mr and Mrs Not-Averages when – as you rightly point out – the only way traditional market research can hope to find out about them is through qualitative research.

What I’m arguing is that the importance of extracting data from a representative sample of the population – by which I take it you mean the UK population – has declined enormously over recent years to the extent that it is now of relatively minor value. The reason is that brands must now respond to a world where consumers have greater freedom of choice and the ability to make purchases on a 24/7 real-time basis, as a result of which understanding how actual consumers are behaving in real-time and on a continuous basis is of much greater value than having a snapshot of the UK population. The one major exception to this, I agree, is political polling, but that’s hardly the basis on which to argue that social insight cannot do a better job than traditional market research in many instances.

On top of this is the problem that people say one thing when answering even the most beautifully crafted questionnaire and then behave in completely the opposite way. Market research surveys rely on the cognitive responses of people who participate in them when, of course, human behaviour is driven mainly by emotions. The result of this is that a consumer can answer a question completely honestly and yet act quite differently without even being aware they are doing so.

The beauty of social insight here is that we’re using people’s natural language, unprompted by a market research question, which is important because people cannot disguise the language they use in written form – and even when deliberately trying to do so, they rapidly revert. This means we can find out far more accurately what consumers really feel and believe than is possible in even the best focus group.

“Social media research can help us
answer some of the questions some
of the time. Understanding the results
it delivers means understanding the
context in which people share
experiences”

GP: I’d like to start by reiterating that I do think social insight can do a better job than traditional methodologies in some instances. However, I think it’s important to emphasise the fact that different methodologies work in different contexts and help you to answer different questions.

As to your point about surveys and cognitive versus emotional response: Are you really suggesting that people always truly think and act in exactly the same way they do when they subsequently share this behaviour on social media? And that they always share it?

Many of the limitations you draw attention to in collecting survey data also apply to social media research. The level of usage is incomplete and the ways in which networks are used are interchangeable. Post-rationalisation can apply to a very short as well as a very long time-frame.

I’ve written quite a lot about digital dualism and how people inaccurately draw a distinction between the ‘real’ and the ‘virtual’ world when, of course, there is only one world. However, behaviour is always contextual and character limits, privacy settings and the type of people we’re friends with or follow all affect how we share information.

Short posts, for example, have the dual advantage of encouraging more people to join the network (as they make less demand on our time) and give the impression of a more active community.

Also, our social media use isn’t an expression of our subconscious desires and attitudes (something, incidentally, an Implicit Association Test can actually help you understand, for example).

I think we’re still at the stage where we need to prove the value of social media research. This means demonstrating business outcomes beyond replicating survey results and understanding it has a place, not the place in market research. That means accepting its limitations, as well as underlining its strengths.

MW: We are absolutely not interested in replicating market research, not least because with social insight we can do things that simply cannot be done in any other way. Nor do we believe social insight has the place in generating insight – but it can outperform traditional market research in crucial areas and to suggest that “we’re still at the stage where we need to prove the value of social media research” is to ignore, or misunderstand the very points that I’ve been making.

As for: “Our social media use isn’t an expression of our subconscious desires and attitudes” – I am not suggesting it is. The fact is, however, that consumers publicly express views and opinions on social media that are not cognitively biased by the intervention of market research questions. In addition, you mention that: “The level of usage is incomplete and the ways in which networks are used are interchangeable” and, of course, you are correct. But the use of social media is already so great that it represents every level of society, ranging from socio-demographic categorisation to people of all ages, including many over 65.

In summary, we are not saying that it’s about one methodology trumping all others; and nor are we saying that social insight is a panacea. What we are saying is that social insight can generate new insight that traditional market research will never be able to determine, while in many cases outperforming it.

GP: I also believe that social media research can generate insights that traditional market research would struggle to determine. However, it’s also important that we continue to be aware of the advantages of complementing what we do with other methodologies – which aren’t limited to the questionnaires and surveys you’ve focused on throughout. Only then can we select the best approach in each instance.

Gareth Price is knowledge leader at Precise. Mark Westaby is director at SpectrumInsight.

• MRS is running a Social Media Research conference on 26 September in London with speakers from Mumsnet, Unlilever, and YouGov. Click here for details.