FEATURE21 January 2016
The reputation of qualitative research is under threat from researchers who underestimate online methods, says Tom Woodnutt of Feeling Mutual.
I am a long-term advocate of online qualitative research. What you can get from it in terms of depth and colour still blows me away. However, even though most agencies offer online and mobile methods, many researchers are yet to be convinced by it. I believe this is because online and mobile qualitative research is often poorly designed, under-resourced and under-funded.
This underestimation of online qual threatens the reputation of qualitative research in the future. If researchers fail to sell through effective online qual studies, then less experienced consultants will enter the industry and mop up client demand for online methods. If new entrants don't meet the high standards set by experienced researchers, then the reputation of qualitative research will suffer.
Too many researchers and clients underestimate online qual
The antipathy against online qual is partly due to researchers’ negative experiences of it. For example, it often takes longer than anticipated and so becomes less profitable, or it may not yield the right feedback. But this is because studies are often poorly designed and underfunded.
Clients who underestimate the time it takes to run online qual exacerbate the problem. In a recent survey with the ICG (Independent Consultants Group) created by Liveminds, 30 of the 43 qualitative researchers who responded found that most clients underestimate the time it takes to do online qualitative research. This helps explain why researchers feel disinclined to offer it and under pressure to under cost it.
To solve this, we need to quash three misconceptions:
1. Online qual is not necessarily cheaper or faster
Online qualitative projects are often under costed because of the false assumption that ‘online’ means ‘cheaper and faster’. While this logic might apply to insurance, taxis and even online quantitative projects, in the case of asynchronous qualitative research, it is often slower and more expensive. More conversation means more consultancy time. Clients and researchers need to recognise this and arrange bigger budgets.
2 . Online qual is not simply face-to-face qual done online
The other problem with design is to do with the false assumption that you can ask questions in the same way that you would in face-to-face research. In face-to-face you have a captive audience (although I did once have someone go to the toilet never to return – they may still be there), whereas in online you have to inspire them to want to take part to avoid drop out. This means framing the study to tap in to intrinsic motivations, for example, by making it fun, meaningful and creative.
Therefore in online, qualitative researchers have to act like online community managers and even content strategists so that they frame questions in interesting ways. Again, this requires charging a premium in order to have the time and resources in place to manage it properly.
3. Online qual is not more superficial than face-to-face
The truth is many still see online qual as more superficial than face-to-face. If they have experienced under-resourced projects I can forgive that. But when it is properly designed it’s anything but superficial.
Firstly, online qual gets you more feedback. To illustrate this, think about the two hours of input you pay participants for taking part in a group discussion. In face-to-face, only one person can talk at once. So over two hours you only get one eighth of that time per person. In asynchronous online qual everyone gives at least a full two hours feedback in parallel, so you get eight times more feedback.
It’s not just about quantity, but also about quality of feedback. People are often more willing to bare their soul online because they are less fazed by being in the same room as a bunch of strangers. Also people can give you feedback in a more natural environment, for example during their customer journey by mobile or in the comfort of their home. This means you can get more real world responses. Add to that how mobile video can tell such vivid customer stories and you have a strong argument for the quality of online qualitative.
If more online qualitative projects were properly funded, designed and managed, then more researchers would want to do them (and be able to do them well). In turn, this would protect the reputation of the qualitative research industry by ensuring that the highest standards are maintained (like avoiding leading questions, rotating stimuli and getting private responses). If online qualitative research continues to be under-funded and researchers turn their backs on it, then the future may not be so rosy.
Tom Woodnutt is founder of Feeling Mutual
The world's leading job site for research and insight
Resources Group
Associate Director – Finance / Auto
Up to £55,000 + Benefits
Resources Group
Research Manager – Environment, Education, Housing, Wellbeing – Social Research Agency
£38 – 48,000 + great benefits
Spalding Goobey Associates
Senior Research Executive – Qualitative –Strategy Consultancy
Up to £36,000 + benefits
-
Brought to you by:
©2024 The Market Research Society,
15 Northburgh Street, London EC1V 0JR
Tel: +44 (0)20 7490 4911
info@mrs.org.uk -
© Copyright 2024 Research Live
7 Comments
Dave Bradford
8 years ago
Excellent article Tom! When I started my research career in online qualitative 20 years ago, I couldn't imagine how far it would have come today. It is truly amazing that online qualitative even survived the early days of chat groups over dial up connections with no established best practices and limited online qualitative moderator training resources. Researchers today have such an incredible array of online techniques that they can use, numerous training options, and a much greater client acceptance of the value online qualitative research can deliver. It also has an even more important role in getting answers to the increase in content that needs more than ever to know the "why", especially as it relates to today's online and mobile users. At least we know a lot more now than we did even a year ago about the right way to conduct this research.
Like Reply Report
Tom Woodnutt
8 years ago
Thanks Dave - I've watched it emerge over the last 16 years and have been really surprised at how slowly it has been taken on - it's great to hear from a believer in it!
Like Reply Report
Ray Fischer
8 years ago
Tom, well said, great post. Your 3 points are spot on. Online qualitative is a unique insights generating method. There is a role for face to face and there is a distinct role for online. While there is some overlap there are unique benefits to each - when budgets allow it is awesome to do online and cherry pick the best respondents for groups or IDI's. Regarding cost, I agree that online should not necessarily be viewed as a more economical way to do research. While there are some economy driven online methods, they are not always the the perfect solution for your objectives. Well-funded online can yield a treasure trove of insights that you simply cannot get from 90 minute 6-8 person focus groups; and it takes time/money to analyze the fertile mountain of unstructured data online will yield. Thanks for sharing your thoughts.
Like Reply Report
Stephen Cribbett
8 years ago
I couldn't agree more Tom. My own experiences echo your thoughts in so much as researchers aren't giving online qual the time or attention during planning or execution that it requires. The level of engagement, richness of data and game-changing insight that I've seen come from those in the know when conducting online qual is, I would estimate, 3 or 4 times greater than those from what appears to be often junior researchers that haven't had the training and support to do it the right way and bring in inspiration. For that very reason, we've moved from providing technology to doing a lot more training, and it's highlighted the skill gaps that exist.
Like Reply Report
Caroline Hayter
8 years ago
Great article, Tom. It's surprising how under-utilised online qual still is. It can yield massively rich learnings, beyond what face to face can deliver. And of course, it's a great complement to face to face too. There doesn't have to be a binary distinction between the two methods, which is always the temptation when talking methodologies. Looking at an issue from more than one angle always leads to more powerful results and we've certainly seen this plenty of times with our clients. And for those who are less inclined to break out of traditional methods, a small online strand of work can be a great 'toe in the water'.
Like Reply Report
Maya Middlemiss
8 years ago
Excellent points. We love recruiting for online qual and it's good to see adoption spreading, but it's taking longer than we expected too. From the recruitment point of view it's important to understand that online does not mean cheaper, in terms of finding, screening, validating and onboarding the right participants, but it can mean savings on incentive payments. Most importantly though we are able to recruit people for online qual, who would never, for a wide variety of reasons, want or be able to attend a face-to-face qualitative session... That has got to be good for the quality of insight obtained by adding this important new dimension
Like Reply Report
John Holcombe
8 years ago
I like and use online qual. But your arguments do not ring true. First - "People are often more willing to bare their soul online because they are less fazed by being in the same room as a bunch of strangers." We often hear this claim but are never offered a shred of evidence to support it (its also a potential red herring, because when researching sensitive topics, you would likely conduct one-on-ones, not focus groups). What about the corollary to that argument? That like people on social media who create their "awesome selves," people involved in online qual may be inclined to be less than truthful - to inflate, exaggerate, etc. Second - your assertion that research that is poorly designed, under-resourced and underfunded tends to underwhelm holds true regardless of the platform used for data collection. Qualitative methods include: 1) participation in the setting, 2) direct observation, (3) in-depth interviews to gather data, and (4) analysis of documents and materials (content and cultural analysis). Because Online qual obviously handles #1 above with screens/the web as a go-between, it can obstruct the untalented moderator's ability to develop empathy. I love online qual when the objectives mesh. But like a virtual tour of the Grand Canyon, it is no substitute to being there.
Like Reply Report