OPINION1 June 2010

Online qual – always read the label

Opinion

Anna Thomas proposes an overhaul of product labelling for online qualitative research.

Qual researchers are a highly inventive bunch. At the Worldwide Conference on Qualitative Research in Prague last month, presenters showcased methodologies from film to collaging, sleepovers, photography and comic strips. There was virtual reality, and some companies were advocating putting CEOs on buses to better understand their audiences.

It seems like quallies will make use of any suitable approach to reach the answer. Which is why it seemed significant that, when it came to the presentations about online qual, there was a general sense of resistance, or even hostility. Some delegates commented that ‘quality qual’ is being diluted by online into a thinner, more superficial exercise. I was curious. Does online qual really represent a slip in quality? Is it more superficial? Or are qual researchers just being inflexible in the face of something new and different?

My thinking about online research starts from my own experience with ‘pop-up’ quant. I view it as a blunt tool with an edge on cost and speed – the research consultant understands and weighs up the risks of applying it. Why isn’t there the same pragmatism around online qual?

I think some of the negativity comes because the sellers are using the wrong language – they’re fundamentally insulting the qual research buyer. I recommend an overhaul of product labelling in the category. ‘Online qual’ is used for a whole bundle of web activities whose outputs are in the form of natural language. This clumsy focus on the form of the data, without regard for its meaning or quality, is what makes hackles rise. If you think quallies just gather opinions, you’ve got me mixed up with my tape recorder. If you think they just analyse information, you’re missing the difference between primary and secondary research sources.

At the Prague conference, ‘online qual’ was used to cover web-scraping and ‘Twitter-trawling’ – the research equivalent of eavesdropping at a nightclub. You might get the right answer, but when is it the best approach?

The term was also used to describe searching forums for brand mentions, or piggybacking existing forums. These activities can give a sense of what’s going on, but again, they’re secondary sources. Neither gives reliable understanding of who the respondent is, or when and why they might seek to control the external impression they present of themselves. Regardless of emoticons and exclamation marks, you cannot tell how strongly they feel.

Bundling in these ‘literature review’ tools with primary research methods generates profound irritation: they can be useful, but there’s a lot to watch out for. Why risk your reputation if you can do groups instead? Having said that, there are other types of online qual that researchers feel more comfortable with.

Firstly there are MROCs (market research online communities). These are purposely-created spaces where respondents are drawn together to work on tasks online. This is where researchers start to feel happier. You know the degree of risk the project can bear. The respondents are pre-screened. You build reflection time (which can significantly change responses) into the design. There’s a lot to do in managing an MROC, but essentially you’re able to confidently generate and test hypotheses with these people because you understand their general properties.

Secondly there are virtual focus groups – although for me, this label is problematic. Like MROCs these are recruited and screened beforehand, but unlike MROCs there’s no opportunity for reflection unless you reconvene. If you do it in text form, respondents can post visuals and write. If you do it by webcam, you get their body language too – a part of the data stream that can make all the difference in some projects. Quallies are not against distance methods per se.

Once you’ve made these distinctions, reframed your language to avoid offence, and accepted that all the tools have merit in certain situations, you realise that online qual and quant are blurring, and just how far the qualitative offer needs to flex. We need to be able to advise on the relative research merits of all these tools – primary or secondary, qual or quant. Or be left as a specialist data gatherer.

Resistance to ‘online qual’ reflects, I think, a proper concern that the people talking about it don’t always understand research. Quallies will embrace anything that gets them closer to where their clients need to be, and we’ve been waiting a long time for the technology. For me, it’s on the cusp of delivering and mixed methodology is where most of us feel comfortable – for the moment.

But the research landscape is shifting beneath our feet. E-commerce tools are appearing that allow customers to specify in great detail what they’re after, potentially making research into the ‘why’ part of the shopping decision redundant. And as I look at an ad offering ‘virtual focus facilities’, I see clearly the USP of online research – I could one day do it from the comfort of a tropical beach. This raises a different angle on the economic argument – what happens when the client pits a Beijing-centred international study against a Boston-centred one? Qual or quant, we may need to be more flexible than we can imagine.

Anna Thomas is research director at Define Insight

Anna Thomas

Opinion

6 Comments

Jasper Lim

14 years ago

I fully agree that the definition of online qual needs some re-thinking. Most of the time, I am confused. Authors in news articles often do not define the qualitative methodology and the data collection method when they talk about online qual or social media research. Some label "Social Media Research" as doing qualitative research using social media data. Others label "Social media research" as using social media tools to collect qualitative data. I personally think that the concept - online qual or social media research need to be conceptualise properly. I suggest that all authors define their terms in their articles before going further into their stories. Kind Regards, Jasper Lim Merlien Institute http://www.merlien.org

Like Report

Ken Parker

14 years ago

Having been in Prague myself, I recognise Anna's comments completely. Our own view is that online can enhance the overall breadth and quality of the qualitative researcher's toolbox. All too often we limit ourselves to short data collection sessions with respondents. A good online tool that has been specifically designed for purposes engages respondents for much longer. Using such approaches, we can now set tasks allowing us to understand the home and environmental context before traditional data collection methods are used, and we can continue dialogue with respondents after the events. In addition, we can set up communities to ensure long term relationships are continued. Online qualitative tools do not just mean online groups. They mean the imagination can be stretched to offer our clients better and more thorough research, cost effectively. Quallies are brilliant at adapting themselves to ensure creativity is maximise. Our experience has shown they are already responding to this exciting opportunity. Kind Regards Ken Parker The Thinking Shed ken@thethinkingshed.com

Like Report

Michael Hollon

14 years ago

I've kept part of Anna's distinctions clear by considering Social Media research much different than all types of online qualitative research. Social media research has the power of knowing that somebody has something to say and they want to share it with anybody who will listen. This is different than any form of traditional market research method because it is not dependant on a researcher asking a respondent to provide feedback. Instead with social media, the "respondent" is not actually responding to the researcher. They are saying something more powerful. They are saying something already on their mind. And as far as online focus groups, I'm still waiting to be convinced that respondents on the other end of the computer connection are actually giving their attention and engaged as much as they would if they were in the same room as the researcher. But I'm optimistic that day will come.

Like Report

Phil McCormick

14 years ago

Maybe it is that the clear distinction between qual and quant doesn't really fit today's world but as it is hard to let go definitions we grew up with we somewhat lazily fall back on labelling everything qual or quant. Agree we need to be much clearer with definitions.

Like Report

Jeroen Verheggen

14 years ago

I think you have to understand that online qual tools are research tools that form part of the social media landscape. Take Facebook for example this is nothing more than a public online forum, but its purpose is not market research. Online qual tools should be build for research purposes specifically and allow you to engage with stakeholders (and not just participants!). That means the researcher can use his or her skills to answer a client's question, by studying behavior, attitudes etc. As to terminology for the moment I would argue that the following are the main online qual research tools - MROCs - Online focus groups - Online Bulletin Boards/Forums - Online diaries - Online web usability tools You ought to choose the tool that fits the purpose. For example online focus groups are pretty good in getting "gut reactions" useful in concept testing for instance. Online forums can be good for exploration of an area or group while diaries can be used for immersive research. Also researchers should mix offline qual with online qual, they are not mutually exclusive! I think the resistance and the (in my eyes wrong) assumption that online qual is somehow less “quality research” will fade. We saw similar behavior when online quant arose and I bet there are still people who think telephone interviewing is a better surveying method than online surveys. It does not matter, reality is that things like speed, costs, reach etc. is just more important to the end clients. The only question we have to continue to answer is what online methodologies can we come up with, build and make robust that caters for brilliant research in a world that continuously changes! In this we (technology providers) need constant feedback from researchers, participants, clients and peers. Jeroen nqual.com

Like Report

Andreiko

14 years ago

Online is a new kind of natural environment that people are spending increasing amounts of time in - for work and for pleasure. This environment is becoming richer, more interconnected and more interactive. We're all learning to communicate and interact in this rapidly evolving environment in increasingly profound ways. Online tools, both qual and quant, are nothing new - they have been around for many years now to allow research to happen in an easier way in this growing environment, which has now reached a critical mass. Nearly half a billion people are on social networks worldwide - that is one hell of a panel - and in most developed countries, 76% of people have online access - 84% of those people aged 18-54 use social media regularly. As qualitative researchers, it makes perfect sense to immerse ourselves in an environment where so many people interact both with each other, and with brands and services in almost every sector. Of course it's natural that new technologies and methodologies have necessarily evolved in our approach to implementing research in this environment. There is no doubt that we will need to continue to innovate in order to stay relevant and ahead of the game - it's an exciting time to be a researcher! I think that a lot of the resistance to online qual comes from the fact that an online approach to qualitative research is textbook 'disruptive'. It disrupts an established part of an existing value chain and the deep relationships within it. If you own a focus group facility, would you be singing the praises of online qual research to the moderators that regularly use your facilities, and risk losing some or all of the business they are giving you? I think not. I agree with Jeroen that online and offline methods are not mutually exclusive. The debate always flares up when online is looked upon as a complete replacement to offline. It's the same debate that flared up years ago about online quant - and we know which side won that one. To those that really know, Online is not just another tool in the toolbox, it's a whole new toolbox that enhances the quality and scope of your research. Andreiko Kerdemelidis CEO Visionslive.com

Like Report

@RESEARCH LIVE