Keeping up with the conversation

We are witnessing an explosion in the amount of user-generated content on the internet. While our industry can struggle to motivate a couple of hundred consumers to give their opinions, many thousands are tripping over themselves to give this feedback spontaneously via social media.

The rise of social media has provided market researchers with a valuable new source of data. At the same time it challenges us to rethink our research processes. What is the best way to tap into online conversations? How can user-generated content be analysed? How does it complement existing research?

When looking to maximise value from observing online conversations, we must start by keeping in mind that we are dealing with spontaneous outtakes, in that consumers are free to talk about whatever they choose. This is great, but by the same token it means that you may not find any buzz on the topics you are interested in. You can either take a bottom-up or top-down approach. Top-down, the starting point is one or more specific research questions, such as what is being said about a new product. In that case we would focus only on conversations that contain the product name or related keywords. In a bottom-up mode we let the data speak for itself. Instead of asking questions, the aim of the analysis is to distil conversations to a meaningful and manageable set of themes.

A rigorous process
In order to get to the insights, it is important to follow a rigorous research process when observing online conversations. Methods like social media netnography can help to bring some structure.

It all starts with sampling online conversations. Just as in traditional research, you need to ensure that you are sampling from a wide enough source. Sampling conversations is typically done via online catalogues that collect sets of conversations. But such catalogues do not cover the whole internet, so it is best to complete your sampling with additional proprietary web-scraping.

To collect only relevant conversations, you have to choose search terms carefully. It is crucial here to take consumer language into account. Marketers are often disappointed if they discover low volumes of buzz about their product, but this is sometimes simply because your consumer speaks about your product in a different way.

The second step is data collection. Through web-scraping technology and APIs (interfaces that allow you to create software that interacts with online social media services), information is collected in a structured database. It is important to clean your data carefully, getting rid of spam, duplication and extraneous information such as text from headers and footers. Information also needs to be restructured into a consistent format, which means watching out for things like the date format of posts, which can vary widely between websites.

A third step is framework detection. Based on the research questions you have in mind, you need to distinguish between useful elements of online conversations and those parts you can ignore. InSites recently evaluated a TV talent contest through social media netnography, focusing specifically on the contestants who featured in the show, so our framework distinguished between conversations about the contestants and about other topics.

The next step is reporting, which is done using a range of analytical techniques. The unit of analysis is text, which makes it very suitable for qualitative analysis, while the large amount of conversations creates the possibility to quantify certain topics.

The use of text analytics to help you translate the text into structured information is crucial. Using grammatical sentence diagramming, the software will propose a set of phrases that can be grouped into concepts. Typically the starting point for this is taxonomy detection, in which you build a model based on a predefined structure (for example, a category for all brand names that appear in conversations). This is followed by associative pattern detection. A major advantage of this approach is that it will also help you to cluster the cases bottom-up by looking at the extent to which phrases occur together. This can help you to find patterns in the text that would otherwise be difficult to spot.

The final step involves qualitative analysis: it is always important to read at least some of the comments to truly understand the nuances within a category. Through this process, it is important to realise that while text analytics software can help to deal with large quantities of unstructured information, the use of human judgement remains an important ingredient.

Putting data to work
One major application of consumer-generated content is insight generation. Consumers typically talk online about subjects they are engaged in – and it is often the contrast between the consumer perspective and the company view that yields the golden nuggets.

InSites recently undertook an insight generation study for the pharma company UCB, which specialises in epilepsy treatment. As part of its brand planning the firm wanted to listen to what patients and carers were saying online. When we set about looking for themes we found topics we expected like diagnoses, treatments and seizures, but also a lot of buzz about the impact of this condition on children and problems encountered at night time. This provided real eye-openers for the client. Themes detected in this way can serve as the basis for insight platforms that can be used
to foster innovation or other marketing decisions.

With almost a quarter of all online conversations mentioning a brand, a second major area of application is online brand reputation and competitive intelligence. Consumers consult social media more and more, and trust it as a source of information when making buying decisions. Especially in the case of high-involvement categories and durables, it is important to measure both how much your brand is discussed online and to gauge the sentiment of these conversations, to detect strengths and weaknesses of your brand and competitors. Even if you only identify a small number of conversations on your brand, digging into them can still be useful. Many companies have offline brand trackers in place. By bringing the consumer context to the fore, online conversations can help identify new values and consumption occasions that influence positioning.

Getting the message across
Finally, user-generated content can help you fine-tune your marketing mix. Online buzz activation is becoming an increasingly important KPI. Many companies set up initiatives on social media, and studying user-generated content can be a great starting point: by finding out what buzz already exists about the brand, you can facilitate existing conversations. A related area is post-testing of advertising campaigns based on user-generated content. As well as checking whether there has been any online buzz, one should also investigate how the message came across. Online conversations can even help shape the content of your communications – in our recent work on epilepsy, we studied the vocabulary patients use when describing their seizures, which informed the client’s subsequent communication.

Tactical studies are not limited to the area of communication. Social media content can be a great source of feedback on products and services. We recently conducted a study on the impact of ageing in which we collected 80,000 online conversations. One of the major themes among the elderly was eating, and one of the sub-themes in those discussions was ready-to-eat meals. We discovered that current offerings could be improved: in many case meals were not adapted to suit the palate or dietary needs of older consumers. And in cases where recipes were adapted the elderly had a hard time figuring out the packaging: too often nutritional information was not clearly indicated and printed much too small.

Netnography can prove invaluable in leveraging the potential of user-generated content. When used in the right way, online conversations can really bring the voice of the customer into the boardroom. Having said this, netnographic studies are most effective when combined with other types of research. For instance, online brand reputation should be accompanied with measuring offline KPIs. Given that the profiles of the people who have generated the online buzz are usually not known, validating the findings with a representative sample is crucial.

And since we are dealing with unprompted feedback, it is not always possible to find answers to all your research questions. As ever, the answer lies in selecting and blending approaches to deliver the results you require.