How can researchers get the most out of AI?

AI, particularly generative AI, has been on everyone’s mind. Since the launch of ChatGPT in November 2022, businesses in the market research sector have been experimenting with the technology to get more out of squeezed budgets and augment insight gathering. 

At the MRS AI: Powering up consumer insights conference, speakers discussed how they have approached artificial intelligence in their research, 

Asking good questions

Ajay Bangia, global scale lead, Ipsos qualitative, outlined the results of a pilot study Ipsos conducted in the US, China, Mexico, Denmark and Thailand, to assess the quality of generative AI’s transcripts, translation, time saving and thematic analysis.

The study began with a process of “reverse engineering” reports Ipsos had put together for studies in the five countries with the use of generative AI. Researchers evaluated them in a blind format, with a human generated report and an AI generated report given to a set of researchers in the countries who had not worked on the original study. 

Bangia said: “The AI did a pretty decent job when it came to summarisation but when it came to elevating insights, this is where the human researcher is absolutely essential. What AI can do is a decent job when it comes to transcribing and translation, developing observations, but the human is needed to elevate those insights and activate the findings.”

Speaking during the same session, Katherine Jameson Armstrong, head of media qualitative UK, Ipsos, said: “There has been an explosion of interest and conversations around AI this year. AI is not new, we’ve all been using analytical AI in our everyday lives and in our jobs, it’s all part of our day to day working life. This new evolution of AI – generative AI – has completely reached the public consciousness and conversations around AI are evolving around this.” 

Ipsos has been working with clients, according to Jameson Armstrong, on how they present AI to consumers and partners. She said: “This instinctively feels to people that this is a radical shift versus other tech innovations we’ve seen. What generative AI does is seen as unique to human intelligence and creativity as qual researchers. However, we need to be mindful of the risks and limitations here – the accuracy is not perfect and we need to be aware of AI hallucinations, so putting a fact checking process in place is really important. We also need to be really mindful of the use of data.”

Bangia outlined a ‘CREATE’ model for qualitative researchers drafting prompts for large language models:

• Character – What persona should the AI take?
• Request – What exactly is your task?
• Example – Provide examples of the type of output wanted
• Audience – Specify to the AI who the output is for, e.g. marketing director
• Type of output – What kind of tone should the output have; formal, informal, etc? Should it be written as a paragraph, a table, or as a song, for example?
• Extras – for example, techniques including flipped interaction prompts e.g. ‘why don’t you ask me a question?’

Researchers should also think of questions that are less obvious to extract more, take an iterative approach, and teach AI models. Bangia said: “As every qualitative researcher knows, asking a direct question doesn’t always lead to insightful answers. Change the perspective: ‘How might Confucius think of this category?’, for example.

“Teaching AI is also important because it is not really trained in market research tasks. But what you can do, by prompt tuning, is help it to understand your framework, your IP, to help it to keep these aspects in mind while it is analysing data and generating the insight.” 

Identifying insights beyond the human eye 

Discussing research Mars Wrigley conducted with Radius and PSA Consultants to understand the quick commerce space – including online takeaway intermediaries and grocery delivery services – Betsy Fitzgibbons, shopper insights lead, e-commerce and new transactions at Mars Wrigley, said: “What we found with this research, is that we do have to go after human first. We put the human at the centre and now AI is going to become part of it. We were able to identify new insights that the human eye couldn’t spot. We did the qual research but we couldn’t have learned everything if we hadn’t applied AI to that.”

The research involved four phases: initial qualitative research with communities; data mining of publicly available data sources; AI data mining of the original qualitative findings, and a fourth phase – using generative AI to suggest creative communication themes for the campaign. 

For the data mining phase of the research, researchers examined publicly available data from wherever people were talking about confectionary in quick commerce, mined from sources including social media applications, app reviews and store reviews. This employed an approach that collected metadata about why people were using certain terms, in addition to the keywords.

The study also used a technique to understand why and how quick commerce apps were displaying products to users. Prerit Souda, director, data science and strategic insights, PSA Consultants, said: “We tried to understand what the machine is trying to do – what’s the logic in their display, and how does that impact what the consumer is seeing? We created agents that went into the quick commerce apps and try to understand how they are showing products in different categories.”

Matt Blacknell, global d-comm customer and shopper insights, Mars Wrigley, said: “The qualitative research gave us loads of rich insight and allowed us to immerse ourselves within the lives and behaviours of the consumers, however we knew that stakeholders like the confidence that numbers and quant analysis provide. For example we are able to quantify the usage, the moments and occasions where people choose to use these services.”

The last phase of the study used generative AI models “to add a little fun and inspiration,” said Souda. Through this, AI suggested potential campaign communication themes based on behavioural research responses, including AI-generated product imagery and marketing slogans, to inspire stakeholders. 

The team at Mars Wrigley plans to integrate AI into its consumer insights and shopper insights briefs going forward, and is trialling new techniques in this area. Fitzgibbons said: “One of the benefits of this research is that it has shown us the importance of adding AI as a tool in our toolkit to improve our research in the future. The watch-out is: don’t try to do one without the other, and it doesn’t mean everything is just going to speed up.”

Can AI really be qual?

The Firefish Group discussed a study it conducted to evaluate whether a project that had been conducted in a world prior to ChatGPT could be done “cheaper, quicker and better”. 

The agency developed three approaches within the experiment: feeding a brief into an AI platform that returned a simple summary; another version with more human involvement (researchers turning what the AI had summarised into an insight debrief); and a third, longer form approach largely led by researchers, using AI to drive efficiencies in the research process.

While all three were cheaper than traditional research, the third approach was cumbersome and still quite traditional so not deemed easier, while the second approach – AI with human involvement – had the most potential, according to Firefish. 

Richard Owen, head of innovation, Firefish, said: “Basic AI was so simple and superficial and had an inability to get under the hood of what was really going on. It cast doubt on veracity of outcomes and felt risky. Human and AI got us much closer to something we would be proud to deliver. There is a real opportunity here, where we take the best of what AI does and teach it how to do what humans do.”

Firefish chief executive Jem Fawcus said: “AI clearly changes the qual game, but too much AI in your approach delivers something that doesn’t really feel like qual and is quite superficial. It raises the question: is this qual at all? Can AI techniques replace the human crafts that people have built over many decades? Is any of this qual? I think it is and it can be. Done in the right way, AI can offer the depth of human insight that good qual research does.

“The secret is to remember what qual is about – humanity. Much of what makes us human is the same as what makes generative AI and large language models seem human.”

Researchers can apply more ‘humanity’ to AI models by training AI with the right prompts to develop human-like briefings, and using bespoke large language models trained on frameworks to feed into tailored prompt engineerings. Fawcus said: “This is where we can really elevate the outputs beyond superficial summarisations.” 

Fawcus has changed his mind about the possibilities of AI: “I’m a qual moderator, and up until recently I would have thought absolutely no way tech would ever be able to replace that. I have almost totally changed my mind. The latest generation of chatbots can do basic moderation extremely well.”

The full on-demand conference is available to ticket-holders on the MRS website here.