Forming connections: How to find the right research mode

x Sponsored content on Research Live and in Impact magazine is editorially independent.
Find out more about advertising and sponsorship.

There are at least three things the market research industry can agree on after surviving more than a year working through a global pandemic. One: researchers have become more nimble and agile than ever to ensure different (virtual) research modes create high-quality data and insight. Two: online qual has become a distinct research mode in its own right, with new rules and boundaries. And three: “You’re on mute, Karen” has become a daily reality for many online moderators, illuminating the connection between how people interact with research and the quality of the resulting insight.

The way in which people interact with different research modes was an issue way before Covid-19 stepped onto the world stage, but the pandemic has unquestionably become a catalyst for accelerating digital transformation.

“This whole situation has seen the industry challenging boundaries with an appetite for change,” says Christian Dubreuil, senior vice-president, client solutions, Schlesinger Group. “But we should always innovate in the ways we gather evidence. Yes, there are tried and tested methods, but we should be challenging briefs and looking at more innovative methodologies to solve client issues. It’s about learning what each mode is best for – you start with the old world, but then you need to evolve.”

Whether they liked it or not, researchers rapidly accelerated their understanding and effectiveness of new virtual research modes over the past year.

“Qualitative research was already experiencing a digital revolution,” says Thomas Prosser, research director, qualitative and innovation at Kantar. “But I’d equate this with what’s happened with online retail and e-commerce during Covid-19 – suddenly, we’ve leapfrogged in terms of uptake and openness.”

Shifting modes

When the pandemic took hold in spring 2020, researchers had to shift modes swiftly. Many important tools in a qual researcher’s toolkit – large co-creation workshops, escorted shopping trips, face-to-face focus groups, meeting people in their own homes, and literal observation – were impossible due to social distancing restrictions. Similarly, many face-to-face interviews, and many telephone quant surveys, could no longer take place.

Martin Lee, co-founder, Acacia Avenue, says: “When the pandemic first struck, all our commissioned projects that would have involved being in a viewing facility were affected. At a strike, they all had to move online at great speed. With one client, we shifted modes from a large focus group to pairs and triads. With another job that involved workshops, we shifted to one-to-one interviews.”

The Ipsos Creative Labs team faced a similar challenge of having to shift modes, says Adam Tornabene, vice-president, early stage operations and logistics at Ipsos. “Historically, we were a face-to-face quant/qual method/team. When the pandemic first hit, we pivoted using a standard online survey with separate qualitative groups. While this method worked, it was lacking our founding principles: collaboration, speed and learning quality, so we knew we had to keep innovating. Where we ended up was a virtual replication of our face-to-face tool, called a Virtual Lab, where we conduct a live online quant survey with a focus group directly following with the same respondents,” he says.

While technology has become a saviour for many projects, for those insight briefs looking to better understand audiences who don’t have access to tech, shifting to virtual modes was a challenge that required reshaping methodologies.

In rural India, research consultancy Basis set up a video screen in a Covid-safe venue, to which participants could travel to be interviewed by a remote moderator, with one other member of the team operating the laptop. “We normally would have gone in person,” says Charlotte Smith, head of qualitative research. “That’s been a real challenge – how do we talk to people without access to mobile internet? We had an elaborate set-up in rural India, but it worked really well. It taught us that, sometimes, we have to take a leap of faith.”

Similarly, in the Philippines, Basis ran telephone interviews and, in Ghana, moved away from online focus groups to one-to-one interviews. “I couldn’t watch every single interview,” says Smith, “so we analysed the transcripts. Without the physical implement of people in our memories, we had to use structured analysis grids to cross-compare each interview.”

Challenges – from old to new

Simply shifting focus groups online can damage the quality of data. Instead, researchers needed to view virtual focus groups as a new research mode, with fresh rules of engagement to optimise interaction with participants.

“Qualitative research is such a skilled discipline,” says Dubreuil. “Researchers who normally use all five senses are having to hone two in the online world. It creates a whole new bunch of challenges. It’s tough to orchestrate and tough to moderate.”

Researchers can no longer read a room – or, indeed, read individuals’ whole body language on screen. Learning how to make people feel comfortable, and prevent participants from talking over one another, are new moderating skills with online focus groups.
“There was a little trial and error at first,” says Prosser. “Everyone had to learn how to engage in digital conversation, but people have become comfortable with the interface. Our biggest adaptation was in terms of moderation.”

Fitting the old modality into a new approach doesn’t always cut it online. In the early days, before adapting virtual groups, one researcher noticed that a five-person online focus group wasn’t generating the same quality of conversation that they’d have in person.
“The moderator has to say stuff they would never have to say – things like ‘during this conversation I’ll have to ask you to speak, and I’ll be inviting you to take turns’. It’s hard for participants, because it’s not as fluent. Some of the natural stuff they might do in a room, such as laugh or make a small interjection – for example, ‘yes I agree’ – can’t really happen online.”

Focus groups need to be much smaller in the virtual environment, with most agencies suggesting figures of around three or four people maximum. “It is definitely harder with more people in a group, as it becomes a series of siloed individual responses,” says Pauline McGowan, researcher at The Nursery.

Different approaches

The chat box is taking on a new role within virtual focus groups, encouraging people to build on their ideas and form connections with other participants. It’s also helping researchers to discover more about participants’ inner thoughts than they might reveal in person. “Interrupting, [waiting for] permission to speak, and voice lag are always awkward on video calls, so we encourage people to use chat boxes and text streams to indicate agreement, or express a new thought we can pick up on,” says McGowan.

Dubreuil believes that bulletin boards, where participants build on thoughts and responses over a longer time period, are helping to fill the gap of real-world focus groups. This is the space where participants might have shared a joke, agreed with one another, or completed someone else’s sentence.

“Bulletin boards have always been there – when we were looking at a product test over a few weeks, for example, or a longitudinal study – but now we’re using them for more of a deep dive.”

It’s also critical for the industry to bring contemporary use of modes into research, such as social media, argues Dubreuil. “We need to reflect real life and make it a more authentic experience for participants,” he says.

He cites social media listening for segmentation and measuring the impact of a new product launch as a mode that’s growing in significance. “It’s about looking at social sentiment and using algorithms to group what it really means around engagement and key words. It’s about using insights that are already there and running analytics to get a wider appreciation of the subject.”

Similarly, Basis discovered that deep insight could be generated by tapping into the WhatsApp dynamic, where people already feel at ease. During the pandemic, Basis trialled WhatsApp as a qual platform for a digital-only study of women in Nigeria. It believes it learned more via WhatsApp than it would have done in a face-to-face group, because the women were already using the platform for commercial and social purposes, and felt relaxed in this environment.

Discourse analysis has also been key for researchers looking to put transcripts under the microscope for deeper analysis and insight. “Discourse analysis looks at the way people subconsciously use metaphor to describe their relationship to the world, and these can change subtly in times of flux. It’s helped us to understand issues such as fear, anxiety, wish fulfilment or desire in the pandemic,” adds Acacia Avenue’s Lee.

Fluid and nimble

Mixing research methods – focus groups with ethnography, or semiotics and one-to-one interviews – is nothing new, but everything is more nimble in a virtual world.

Kantar’s Prosser likes the fact that there is more fluidity during the qual process than there used to be. “When you’re setting up an old-school viewing facility, the stakes are higher. But online, if it’s not quite right, it’s easier to recover,” he says. “If we do an online community, we can get back in touch with those people. It’s less about the quality being awry and more about how we can keep building on different learnings and methodologies to extend our insight. This is how we should work anyway.”

Many one-to-one video interviews are now naturally falling into ethnography, with people grabbing props to illustrate a point. Basis’s Smith recently conducted a project for a financial client researching super-high net worth individuals in the UAE, Singapore and China. “Normally, we would have visited in person,” she says. “But that audience is very mobile and hard to get hold of. This time, we could speak on a video call in a relaxed environment.”

A shift towards online quant, away from telephone and face-to-face interviews, has inevitably sped up over the past year. Unilever in China, for example, accelerated its process to make all its brand tracking digital. However, moving from face-to-face to online surveys can bring its own set of challenges, especially for tracking data – and especially when a bonus for a CEO is based on a Net Promoter Score score and it changes.

lt’s well documented that people answer online surveys differently, as Pete Cape, global knowledge director, Dynata, explains: “People answer questions differently online. It provides completely different data. If I’m being interviewed by you, I see you and I have to look good to you. I start to think about giving you the answer you want to hear. It’s not people lying; it’s a human bias, a desire to please. No-one really concerned themselves with social desirability bias in the past because we couldn’t account for it. Now, it’s well known. We can’t account for it, or change it, but we interpret answers in a different way. It’s a different new truth to deal with.”

Hence, some more personal topics, such as financial habits and sexuality, tend to work well in online quant. These are issues about which people tend to be more guarded and the anonymous environment of an online survey can make it easier for them to respond more honestly. Similarly, a written verbatim response in an online survey might be more crafted – and sometimes more poignant – than that which might be elicited in a verbal conversation, suggests Dubreuil.

Additionally, an interviewer cannot help people completing online surveys with a prompt, as they might do on the phone or face to face. There tend to be more “don’t knows” in response to questions online.

Cape notes that people react differently to scales in an online survey, and no-one quite knows why. “If you move from an interview to an online survey, your scores tend to move towards the centre of the scale. This might be related to an honesty factor; if I ask a question, you might just have a go... it results in different data.”

Quant interviewers are highly skilled in using humour and other techniques to hold attention, but attention spans for online surveys are much shorter. Lou Horner, research director, Acacia Avenue, noticed that the quality and quantity of open-ended response in quant research markedly improved in the first lockdown, but this enthusiasm tailed off as fatigue set in, “to the point that we are having to put a forensic lens on data quality”, she says. “This is on the questionnaire writer, too, though. It’s more important than ever that surveys manage people’s energy levels by giving them engaging formats, conjuring real-life environments and decision-making processes.”

It’s a lot faster to complete a survey yourself than to listen to someone else. Quant experts reflect on whether this is reading time or thinking time. How much thinking is being done? But speed can also be an advantage in this new reality.

“The mobile phone has become the epicentre of research activity,” says Jon Puleston, vice-president, innovation, Kantar Profiles. “Before lockdown, I would expect a 24-hour turnaround for 500 people. Now, I can send it out, make a cup of tea, and come back to 500 completes. I have to include quotas to ensure that I’m also hearing from people who are too busy during the day.”

Cape, who is involved in the Market Research Society’s work with panel companies on mobile survey optimisation, points out that mobile surveys have a higher dropout rate than those on desktop. “We’re getting better at this, but we need to redesign questionnaires to make them easier to fill in on a tiny screen while maintaining comparability with previous surveys,” he says.

Puleston adds that emerging technologies in this space will grow in importance, such as linguistic taxonomy mapping, where researchers will be able to analyse open-ended answers en masse, using tech analytics.

He also predicts a rise in voice-in-quant online surveys – where people might be able to listen to a digital survey and then respond by speaking. He thinks the chance to listen and speak in response to a question, rather than read and type, will increasingly be offered to quant participants. It’s likely to improve the richness and quality of data in two ways: by offering participants a choice in how to respond, and because talking might create more data.

Voice technologies such as Phebi (winner of best technology innovation at the MRS Awards 2020 ) use artificial intelligence to analyse speech from participants. This approach could lead to richer data from online surveys, as talking can lead to longer answers, and may solve the current issue of mobile surveys not working as well as they should. Here, the mobile platform becomes fit for purpose; Phebi points out that talking is faster – and easier – than typing. While typing on a mobile with one finger generates (on average) 29 words per minute, talking at a relaxed pace generates 125 words per minute.

There’s a bigger issue at the heart of quant research in the Covid-19 pandemic, mentioned by a number of practitioners, and it’s about human behaviour and people’s world view. “The bigger question is, what will happen to my data if people aren’t behaving the same way? Should I still be asking the same questions?” says Cape.

Kantar’s Puleston is trying to understand whether people’s personalities have changed during the lockdown. “People are appearing more extrovert on paper, but that’s because the questions are centred on things such as ‘I like my own company’, which has a different nuance in lockdown. So, it’s the question that has to change. But we’re also seeing genuinely fundamental changes in people’s world view – they’re more communitarian in outlook, for example,” he says.

The future looks...

The research industry, not through choice, has moved 10 years in just one in terms of digital transformation and experimentation. Practitioners should return to their roots as curious investigators, striving to transform these new virtual research modes into powerful tools, and ensuring they select the right modes for each brief.

For example, while virtual focus groups are here to stay as a new methodology in qual researchers’ toolkits, they don’t sound the death knell for face-to-face research modes. “Is the qual toolkit as good as it was before? No, but it’s good enough. Tools such as large workshops and meeting people in their homes were devised for a reason. I can’t honestly say it is as good as it ever was; it’s only as good as it can be at the moment,” says Acacia Avenue’s Lee.

For Dubreuil, it’s important for the industry to look at how it might approach client problems in a different way. “The client wants to get from A to B, and our job as researcher is to make sure the engine’s working well. We need to look at the way we collect our data, and the benefits and drawbacks of each approach, so we can better understand the biases and strengths of each research mode.”

Keeping calm and carrying on

• Expect the unexpected
• Don’t immediately jump to a data quality conclusion if surprised. Take a step back. Has the pandemic affected what I’m looking at? What are the big societal forces that might be affecting it? Do I need to change my thinking in the way I analyse this data?
• Don’t be tempted to think that, just because circumstances have changed so much, people have. Human wants and needs don’t change anything like as quickly as circumstance does.
• Embrace the fluidity of this brave new world. Lean into the speed, the flexibility, the freedom to evolve stimulus. Encourage participants to share glimpses of their home life.
• Make the virtual world work for you. Evolve methodologies and techniques. Mix things up. Enjoy the opportunities it brings.
• Beware Zoom/screen fatigue.
• Now is the time to experiment. Don’t fall into the habit of force-fitting traditional research modes into a virtual world. Consider every client brief in context to get the best-quality data.
• Enjoy the wider client audience and the democratisation of your research in this new reality, where online ‘meeting rooms’ can be as big as you want them to be.

this article was first published in the april 2021 issue of impact.