Social media has made the world simultaneously bigger and smaller. Where once an ordinary person only had a limited circle of influence, now – as long as there’s an internet  connection – we can all share our reviews and recommendations of brands and products with the world, via Twitter, Instagram and Snapchat, among others. 

As a result, these platforms have become popular among marketers and market researchers alike, for providing access to ‘authentic’ conversations. But there are questions around the extent to which social media conversations reflect real-world ones. Can the two be reliably considered as reflections of one another? 

Not according to a recent study in the Journal of Advertising Research: ‘Why Online Word-of-Mouth Measures Cannot Predict Brand Outcomes Offline’. 

The study looked at four metrics – volume, sentiment, sharing and influence – to assess the potential correlations between online and offline conversations about brands. It found almost none. 

Brad Fay, chief commercial officer of Engagement Labs, who co-authored the study with his colleague Rick Larkin, vice-president of analytics at the company, explained that the study came about as a result of the company’s merger with Keller Fay Group around two years ago. 

“We’ve been tracking the brands people talk about using a survey research methodology. Then when we [Keller Fay] combined with Engagement Labs, the idea was that we would combine social media analytics with total conversation measurement to create something that we call ‘total social’. 

“Clearly, one of the first questions that we wanted to answer was – how do they compare to each other?” 

To answer this question, the researchers began combining offline and online consumer conversation data and online data for 500 US brands across 16 product and service categories.

Offline data for the brands came from the survey method, which asked respondents to record the categories and brands they talked about the day before taking the survey. Data from the whole of 2016 was included. 

Online social media data for the same time period came from an online listening service, which used Boolean keyword queries – a type of search that allows keywords to be combined with operators or modifiers such as ‘and’, ‘not’, and ‘or’ to produce more relevant results. This service identified mentions of the 500 brands across Twitter, online blogs and newsgroups. 

For the first test, of the volume of mentions, the researchers analysed all 500 brands collectively every month for a year. A relationship was found between online and offline, but not a determinative one. That is, many brands had similar scores in terms of conversation volume, both online and offline, but many had very different ones too. 

“There’s been so much interest in tracking social media, and behind that there’s been a bit of an assumption that it’s probably reflective of the broader conversation about brands,” says Fay. “While that is true for some brands sometimes, what we found is that, on average, it’s not true.”

few general patterns

The report suggested that where conversation is related to market penetration, purchase or consumption frequency, or advertising expenditure, brands might be ranked similarly in terms of conversation volume online and offline. 

However, there are factors that can work against such correlations.If a brand is fashionable or innovative, social media users may want to send signals about being in the know; while everyday products that provide value but aren’t ‘sexy’ have a tendency to perform better offline than online. 

“We went in thinking we’d probably find some categories where they correlate pretty well and others where they don’t,” says Fay. “What we found was that there were very few generalisable patterns. It seems to be quite unique for each brand.“

What’s more, Fay says, when the researchers looked at tracking brands over time, they found that one type of word of mouth (WOM) often goes up while the other goes down. 

There are “a long list of reasons” for this, says Fay. “To some degree, it’s because different people are doing the talking and they’re talking for different reasons. People’s response varies to different types of media and marketing. Also, in the social media sphere, whether the brand is deliberately making an effort is a bit more of an influence.”

The researchers also looked at correlations in sentiment and found “no meaningful correlation at all”, except in a few isolated cases – for example, with brands undergoing crisis situations, where sentiment declined on both sides. 

The final two metrics included in the analysis – brand sharing and influence – showed even less correlation between online and offline conversations. Online brand sharing is based on the frequency with which people share branded content via its social pages, while offline brand sharing is the percentage of the brand’s conversations that contain a mention of its advertising and marketing. Both are based, to some degree, on the effort made by the marketer. 

For that reason, there was some correlation on an overall basis observed during discrete periods. However, on a weekly trend basis throughout 2016, the correlation was ‘essentially zero’. The authors concluded that consumer engagement with brands’ marketing content works entirely independently online and offline. 

Finally, the report found that there was no correlation between online and offline influencers – based on online influencer score – and a slightly negative correlation over time. “It is fair to say that influencer-marketing strategies for brands need to be considered entirely independently from each other”, the authors said. 

Another study has revealed differences in trust levels across the two forms of WOM. Network Research looked at what channels consumers trust when planning holidays and visits to leisure attractions, and found a lack of trust in digital WOM. 

The study, based on a nationally representative survey of 600 UK residents, found that social media scored the lowest of all channels for trust. While 80% said they trusted family and friends for recommendations, 60% trusted TripAdvisor and 40% trusted newspaper articles, just 32% trusted Facebook, 20% trusted Instagram and 19% trusted Twitter. 

There was, however, some difference across age groups when it came to seeking advice from these channels. While 23% of all respondents said they would consult Facebook for holiday recommendations, followed by 11% for Twitter and 9% for Instagram, those in the 18-24 age group were more likely to do so: 40% of respondents in this age group said they would consult Facebook, 23% said they would consult Twitter and 28% Instagram. 

Reference:

Fay, B., & Larkin, R. ( 2017 ). Why Online Word-of-Mouth Measures Cannot Predict Brand Outcomes Offline. Journal of Advertising Research, 57( 2 ), 132-143.