OPINION14 July 2014

Measuring social is harder than you think


Counting social media is easy, it’s the measurement that’s hard says Ray Poynter, as the industry collaborates on evaluating social media campaigns.

At first glance social looks like an easy medium to measure. In one sense, social is the most measurable of channels, with every element from views, to shares, to clicks, to plays leaving an electronic trace of what has happened. However, as social media spending grows, advertisers are becoming increasingly concerned that they are struggling to validate the effectiveness of their campaigns and activities.

It turns out that when dealing with social, counting is really easy, but measuring is much harder. There are many reasons why measuring social is hard but some of the key reasons are:

  • Social is usually used in conjunction with other channels. The evaluation of social has to measure direct effects and also how it interacts with other elements. For example, to what extent does TV advertising impact Twitter, and to what extent does Twitter impact TV advertising, and how does the combination of the two impact sales?
  • Even within social there is a complex interaction between Paid (buying placement), Owned (using the brand’s own pages/sites) and earned (the word of mouth and hoped for viral effect); unpicking cause and effect can be difficult.
  • The metrics for social are often used to manage campaigns or activities. While this can improve the performance of the campaign, it can also confound the measurement.
  • Social is often less controllable, from a measurement point of view, than traditional media. For example, creating a control sample that will not be exposed to a social message may be impossible.
  • Because social is the most measurable of all media, problems tend to arise from having too many choices of data rather than too few. With too many measurements, some of them are going to look positive, even if only by chance.
  • Social is often not time bound in the way traditional campaigns are, i.e. there may not be a clear before, during, and after. For example, the Cadbury Gorilla ad uploaded in 2007 to YouTube is still collecting plays, shares, and comments.
  • Although the objectives of a social campaign/activity should always be linked to business objectives, they are not always linked directly to them. Social is often used to achieve tactical ends, such as collecting information, encouraging trial, or simply generating engagement. Evaluation needs to be tailored to the objectives – not simply tied to ROI.
  • The budget for social activities/campaigns is often smaller than for, say, TV. Also, the number of social campaigns/activities may be larger, which means the measurement approaches need to be appropriate to the circumstances – for example, a full blown media mix modelling approach is not always going to be viable.

To illustrate the difficulty, imagine a campaign based on TV advertising, social media, and point of sale, designed to encourage people to buy a specific coffee and to post a message to their friends about it, in order to encourage both purchase and copying behaviour. If somebody posts or reads a comment, does that make them a) more likely to buy a coffee, or b) does being more likely to buy a coffee make it more likely they post or read something? The answer, from a variety of studies, is sometimes a), sometimes, b) and usually both. One problem for social is that there is often a lack of the counterfactual – i.e. how many coffees would have been bought by these people if the campaign had not run?

Aren’t some people already measuring social?

There are companies measuring some aspects of social, and many of them are doing a good job. However, when we talk to the companies spending the money, they tell us there is no consistency of message, no consistency of method, and no body of knowledge they can turn to when they want to determine the right approach.

There is a growing realisation that the lack of established practice about the evaluation and measurement of social is costly and is holding back innovation. In order to tackle this issue, a cross-industry initiative has been formed. This project, named #IPASOCIALWORKS, has been created to provide advice on measuring and evaluating social.

#IPASOCIALWORKS brings together the Institute of Practitioners in Advertising (IPA), the Market Research Society (MRS) and The Marketing Society, along with the key social platforms (Twitter, Facebook and LinkedIn). This initiative is believed to be the first time such a wide collaboration across the industry has been formed, which reflects the urgent need to establish clarity on how to evaluate the business value of social.

The project comprises the following elements:

  • The creation of a collection of peer reviewed case studies that show ROI delivered via social. The key benefit of these case studies is that they highlight the steps required to validate claims about ROI. You can access the case studies online.
  • Industry events focused on the need to move from counting to measuring, covering things like the need to ‘bake’ the evaluation process into the design of social campaigns and activities.
  • The creation of a Guide To Evaluating Social; I am the lead author along with a team of industry luminaries advising on its content and format. The guide will be launched in the autumn.

The Guide to Evaluating Social is nearing completion, but there is still time to help by reviewing the current content and sending in suggestions and ideas. You can download the current summary and draft of the guide.

The #IPASOCIALWORKS team would love more case studies, so if you have an interesting case where you can show ROI, in a way that can be peer reviewed, visit https://www.mrs.org.uk/mymrs/socialmedia.

More information on IPA Social Works can be found here.

Ray Poynter is the author of the Handbook of Online and Social Media Research and the soon to be published Handbook of Mobile Market Research. He is the founder of NewMR.org, director of Vision Critical’s knowledge sharing centre, and managing director of the Future Place.


6 years ago

You write that : his initiative is believed to be the first time such a wide collaboration across the industry has been formed, which reflects the urgent need to establish clarity on how to evaluate the business value of social." In fact more than three dozen major organizations including Procter & Gamble, McDonalds, GM and Southwest Airlines as well as thier agencies and more than half a dozen professional organizations have been meeting since 2011. In June 2013 we published comprehensive standards and best practices for social media measurement. They are published here: www.smmstandards.org. We meet quarterly and would welcome the addition of #!IPASocialworks to our group.

Like Report

6 years ago

Hi Katie Thanks for the heads up about the #SMMStandard group, really useful. Although there is clearly overlap, I feel that we are mostly talking about a different focus. Visiting your site it looks as though the core of your work relates to the public relations field and to establishing agreed standards for what the metrics mean? The #IPASOCIALWORKS project is focused on the advertising, marketing, and market research sectors, which overlaps but is not the same as the focus your team seems to have. The project I am writing about in the article is not looking at the question of what the metrics mean, or how the metrics are used. We are looking at issues like: Why measure? Why is measuring hard. And, the problems caused by issues such as: correlation and causality, influence and homophily, and the confounding nature of simultaneously using measures to evaluate and manage activities. I am sure we will benefit from drawing on the work you have done, just as I am sure you will benefit from the materials on the #IPASOCIALWORKS site – such as the peer reviewed case studies.

Like Report