FEATURE22 April 2024

Mastering the wave: How to future-proof the insight sector

x Sponsored content on Research Live and in Impact magazine is editorially independent.
Find out more about advertising and sponsorship.

AI Asia Pacific Europe FMCG Features Impact Latin America Middle East and Africa North America Technology Trends UK

The influence of artificial intelligence is rising. Katie Jacobs reports on the interplay of researchers and technology, and how practitioners can avoid the drift and become masters of the new insights paradigm.

person surfing on wave

For Clare Michelmore, UK insights lead at FMCG giant Kimberly-Clark, it all started when someone showed her how ChatGPT could be used to write silly poems. Since then, she’s gone from limericks to complex research applications, using generative artificial intelligence (AI) tools to transform the way she works – and she’s excited about the future potential of such technology.

“Insights is one of the best industries [for these tools] because there are so many good, interesting applications,” she enthuses, name-checking techniques such as eye tracking (for which AI has a close correlation with humans, she notes), qualitative probing (where generative AI follows up on open-ended questions with research participants, generating in-depth answers) and assistance with question writing (taking away some of the mental load) as particularly useful in her experimentation journey so far.

As Michelmore’s story demonstrates, generative AI is already having a profound impact on how many insight professionals are working. The sector is ripe for disruption, with the potential of these tools to speed up data analysis, automate insights generation, produce compelling data visualisations, and even recommend campaigns based on its insights. No profession or industry is immune, however. According to a report from Bloomberg Intelligence, the generative AI sector could grow to $1.3tn over the next 10 years, from a market size of $40bn in 2022. ‘AI’, the abbreviation of artificial intelligence, was named the 2023 word of the year by Collins English Dictionary, while Cambridge Dictionary chose ‘hallucinate’ and updated its definition to include AI programs providing (or, rather, making up) misleading or incorrect information.

In the research sector, much discussion has centred on the potential of these tools to transform the work itself. Less has been said about the skills required to future-proof the industry and the people working within it at a time of growing reliance on technology. With the possibilities that technology offers advancing at an intense, almost breathless, rate, how can researchers ensure that the future is one of humans and machines working in tandem, rather than machines going it alone?

First, though, it’s worth re-establishing what generative AI does well – infinitely faster, if not better, than humans – and how researchers can benefit. Ben Davis, head of data, AI and research tech at Opinium, explains that AI can help tackle complex research problems, such as how to extract meaningful insights from large volumes of unstructured data, at speed and scale.

“Insights workers will be empowered to perform technically complex tasks, such as advanced statistical analysis and iterative design of open-text code frames,” he says. Research from Harvard Business School and Boston Consulting Group in 2023 found that consultants using AI produced 40% higher quality results, and completed tasks 25% faster, compared with a control group. “AI tools not only help us automate tasks and complete work faster,” adds Davis, “but they also help us deliver higher-quality study design and research analysis to our stakeholders and clients.”

Phil Sutcliffe, managing partner at Nexxt Intelligence, describes the technology as having game-changing potential compared with tools that have come before. “There’s been a mantra for the past 10 years about using technology to be faster, cheaper, better – but mostly it’s been faster and cheaper, sometimes with detriment to quality,” he says. “But generative AI allows us to understand people better at scale.” This, he adds, frees up researchers to focus on more impactful – some would say rewarding – work, such as designing research and analysing and communicating it in a way that adds value and leads to better business decisions.

“This disruption is fundamentally a good thing for the research business, because it forces us to connect with what we are great at,” agrees Zoe Fenn, leadership consultant and founder of You Burn Bright. “Those strengths might have been underused for a long time – we used to spend so much time churning through data to get to what we love to do. This shift will allow those brilliant minds to get back to what they are great at, but haven’t had much time to put their energies into.”

wave crashing

The benefits of researchers engaging with this technology is clear, and the research sector is well placed to take a lead on embracing new tools. Betsy Fitzgibbons, shopper insights lead, e-commerce and new transactions at Mars Wrigley, believes the insights function is the one “best suited to pilot, test, kick the tyres and embrace AI technology for the good of the business, identifying where it can add strength, power and effectiveness”. But, she adds, it is incumbent on insights teams to “recognise and call out AI’s limitations and pitfalls, as well as build guidelines”. (More on that later.)

So, what are the specific ‘technical’ skills insights professionals need to engage meaningfully with this technology? After all, despite their impact and speed of development, many of these tools are in their infancy.

According to a 2023 survey by Microsoft, industry leaders believe it is essential that employees from all professions and sectors learn when to leverage AI, and how to write great prompts, evaluate creative work and check for bias. Fitzgibbons flags data analytics as an essential skill for insights professionals and one that won’t be disappearing, even as machine capability increases. Opinium’s Davis highlights a culture of continuous learning, with an emphasis on data and digital literacy.

As Microsoft’s survey found, prompt engineering – the process of structuring text instructions so they can be understood and interpreted effectively by a generative AI model – is increasingly important. “To be most useful, generative AI needs clear objectives and context,” explains Harley McKee, business intelligence manager at Opinium. “Better outputs and ideas come from better reasoning and thoughtful instructions. Similar to how the best managers inspire great work by defining roles, removing ambiguity and setting clear goals, researchers must communicate with large language models (LLMs) as if they were a person to whom you were delegating important work.” When it comes to learning how to do this effectively, he recommends “getting your hands dirty”, jumping in and experimenting with the technology.

This is something Michelmore has experienced. “The first time I briefed a bot to do an end-to-end piece of research, I realised it hadn’t understood my brief when I got the research back,” she recalls. “If I’d been briefing a human, they would have asked questions. But my business questions remained open – which was fine, as it was a pilot, so I was in a safe space to learn. It was a great tool; what messed it up was my inexperience.”

Michelmore advocates piloting and creating safe spaces to learn and fail. A portion of her budget is ring-fenced for doing just that, and the team has quarterly development days to focus on learning new skills. “The most important factor is the willingness to try something new,” she says. Curiosity about the possibilities of new tools and an appetite for experimentation – with the necessary guardrails in place – is a critical mindset for researchers to adopt.

wave crashing

Steering the ship

The client picture: promoting adaptability and learning agility

Nick Rich, a global insights leader who has held roles at Carlsberg Group and IHG, is the author of a forthcoming MRS paper on the skills required in a modern client-side insights team. Impact spoke to him about what AI means for skills…

“Insights leaders should recognise the transformative potential of these tools and embrace the benefits. That means fostering a culture of learning and adoption within teams, investing in upskilling and continuous learning, encouraging people to explore training and certifications, and promoting creative experimentation within a supportive environment.

When it comes to recruitment, leaders should prioritise data science skills, including programming and machine learning. Be sure to encourage collaboration between data scientists and insights professionals, to foster mutual learning. But exercise caution before rushing to recruit ‘prompt engineers’ – instead, use AI to empower insights teams to augment their capabilities.

Beyond technical skills, we need to emphasise cognitive abilities, such as critical thinking and problem-solving skills. In recruitment, set problem-solving challenges and observe how candidates use the tools available, including AI.

We should promote adaptability and learning agility: a willingness to learn and stay up to date with emerging technologies. Leaders should foster a growth mindset in teams, encouraging them to embrace change as an opportunity for growth. Change is happening so fast, you want people who will go with it and learn as they go.

While AI will bring transformation, complete redundancy is unlikely. It’s just a tool, not a replacement for human expertise. We need to learn to handle it well, using it to enhance our capabilities rather than fearing obsolescence.”

For those who are less technically minded, Sutcliffe has a reassuring message. He believes insights professionals, on the whole, do not need to understand what is inside the ‘black box’: it’s about knowing how to leverage the technology and mitigate risks. “AI will increasingly become about buying applications,” he says. “Not everyone is going to need to be an expert in how AI works; you are going to need to be expert in how you can use it to your advantage.”

While there is a benefit to understanding prompt engineering, Sutcliffe doesn’t think it is essential for all researchers, given that solutions will increasingly have the AI built in. “The job of a researcher is how to address the client problem, find the insight and communicate that effectively,” he explains.

In other words, it is more important to know how to leverage some of those uniquely human (for now, anyway) skills. For Sutcliffe, the most essential are what he calls the ‘four Cs’: connection, critical thinking, creativity and communication. Connection is about empathy – the ability to connect with people on a human level, something machines cannot yet achieve. Insights professionals need to “add a human layer to data to bring it to life”, says Fitzgibbons, “as well as be the human conduit to maintain that empathy component that is necessary… to connect with consumers”. They must also be “the voice and champion of the different voices of the consumers, [ensuring] businesses don’t just go after numbers”, she adds.

Critical thinking is perhaps the most essential skill when it comes to mitigating risks associated with AI. Assuming tools are infallible is not just ill-advised – it’s dangerous. There are plenty of examples of generative AI ‘hallucinating’ inaccurate or misleading information that could be damaging to consumers and organisations. It frequently hallucinates false legal information, for instance, and AI-generated imagery has been found to be biased, amplifying gender, racial and other stereotypes.

Insights professionals must be aware of such risks and ensure that a human is always in the loop to watch out for errors and inappropriate suggestions. “Automation does not relieve us from validation,” says Nick Rich, until recently vice-president, insights and analytics at Carlsberg Group. “We have an obligation to ensure [AI-generated outputs] represent a true perspective of consumer behaviours, needs and desires.”

The issue to recognise, says Shari Aaron, executive vice-president of growth and innovation at Radius, which has worked with Mars Wrigley on AI-based projects, is that we cannot keep tabs on the millions, even billions, of data points from which AI is drawing. “Given the scope and large-scale abilities of the LLMs to take in data, we have to be uber-cautious about the validity of the information included before we trust the outputs from these models,” she warns. “There is no way to know the accuracy… LLMs are so opaque.” Anderson offers a note of caution around inputting proprietary or sensitive information into open-access tools such as Bard or ChatGPT: “Security of confidential information is our first priority.”

McKee agrees on the need for caution, advising: “Researchers must be careful not to ‘fall asleep at the wheel’, letting the AIs do everything for them or copying outputs without critically reviewing them. We need to grow as strategic thinkers. While AI tools can help us run our metaphorical ships, we always want to make sure we’re the ones who make the final decision about where to steer.”

While generative AI can produce ‘creative’ work (see image generators such as DALL-E), true creativity remains in the realm of things humans can do better. The consensus is that using AI frees up more time and brainpower for humans to focus on creative and strategic tasks. But that doesn’t mean AI can’t provide a useful starting point for creativity, whether by analysing reams of insight to suggest concept angles and taglines, or suggesting survey questions that a tired and overworked researcher may have missed.

“Generative AI tools help jumpstart creativity and offer a sounding board for our insights consultants to sharpen their research reasoning, which, they say, makes tackling the most complex research challenges feel more manageable – and seriously fun, too,” says James Endersby, chief executive at Opinium and chair of MRS.

Once you have developed ideas and analysis, communication remains, for many, the final hurdle. As Sutcliffe says: “Having insight is one thing, but persuading organisations to take action on those insights is another: it requires great storytelling and communication skills.” These are not a given, and communication is an area in which many commentators feel insights professionals must up their game.

“We still have a long way to go to develop those skills,” says Fenn, about the importance of communicating effectively with colleagues, stakeholders and clients. “We’ve been working in silos. You don’t come to an insight on your own.”

That enhances the importance of what Fenn refers to as social capital: the connections between people. To briefly skirt another hot debate, the increasing use of AI could mean there is more value in spending time together in a workspace, especially for early-career professionals who benefit from mentoring (formal or informal) and observing and collaborating with more experienced colleagues.

bird's eye view of surfers

Remembering strong foundations

Communication, creativity, empathy, critical thinking… these are all transferable essential skills. They are sometimes described as soft skills (rather unfairly, as many leaders know they can be some of the hardest to demonstrate), but they are the skills a machine will find most challenging to replicate. It would still be a mistake, however, for insights professionals and employers to neglect the foundation of market research.

When musing on in-demand skills, Sinead Hasson, chief executive at specialist recruitment agency Hasson Associates, points out that companies will always need classic research skills. “The ability to be a great qualitative or quantitative researcher will always be valuable,” she says, noting as equally useful those skills around open source and social media data analysis, people, communication and leadership, and business nous.

Rich also highlights business acumen, pointing out that client-side leaders in particular need to approach AI from a ‘business problem first’ perspective rather than getting over-excited about the technology itself. “Most important is to seek to define and solve the business problems identified with AI in a methodical way,” he advises. There is also a need to remain strategic when it comes to investment. “I’ve seen a lot of businesses talking about AI this, AI that – and what we’re doing is potentially making the same mistakes we made with big data,” says Rich. “It’s really important to understand what the business problems are that you have, and then work out which AI solutions offer you a better, faster, maybe cheaper way to solve that problem.”

Michelmore says: “Don’t stop learning traditional research skills. AI is like swimming; I can swim faster with flippers, but the flippers can’t swim without me. AI means we’re going to be able to do more things faster, but I don’t think that you can take humans out of it anytime soon.”
With the risks of relying too unquestionably on the outputs of AI, researchers must have a solid knowledge base to be able to question and evaluate outcomes. Deciding on what skills to develop is a challenge for leaders in the sector. Soft skills are transferable and should be invested in, as these will help people work cross-functionally and are not going to be replaced by machines. However, people with those skills might not have gone into market research traditionally, instead choosing marketing or consultancy-type roles. As Sutcliffe points out: “The sector is going to have to be more focused on recruiting intelligent, capable, curious people and helping them develop those skills.”

A final rule to live by when it comes to working in tandem with machines, says Preriit Souda, of PSA Consultants, (see boxout, below0 ), is not to try to compete with the machine. “Machines come and go; it’s the ability to think that matters,” he says. “Use these tools to enhance your ability, not compete with what they do faster and better.”

A 2023 IBM report found that leaders expect 40% of their workforce will have to retrain or reskill in this new world, and included a powerful quote: “AI will not replace people… But people who use AI will replace people who don’t.” Asked for his advice to researchers looking to future-proof their careers, McKee says: “Be curious and never stop learning.”

It’s not about letting technology take over, but considering how you can use it to do your job better. Don’t fear the machines, but don’t trust them implicitly and uncritically either. Future-proofing is about making sure you are one of the people who is aware of and curious about how to work in partnership with them.

wave crashing

Mars Wrigley: Technology as a springboard 

Mars turned to generative AI when looking for impactful behavioural insights around consumer use of quick commerce retail including online takeaway intermediaries and grocery delivery services. Betsy Fitzgibbons and Matt Blacknell partnered with Radius and PSA Consultants to carry out qualitative research across social media communities in five countries, and use behavioural data science to analyse 85m public data points. Mars used AI to generate ideas and imagery as a springboard for future marketing activity.


The project led to improved sales and targeted promotions, and it’s given those involved a greater understanding of the power and drawbacks of AI.

“The speed with which you can generate thought-provoking and inspiring images, taglines and concepts, crafted from research results, to ideate with teams in workshops is amazing,” says Fitzgibbons. “The watchout is [these concepts] are not fit to be launched as is – for now. That will change as generative AI improves.”

This means the insights team needs to be central to building guidelines around AI use, and ensuring human elements, such as empathy, remain front of mind. “I heard someone say that ‘AI is like having a million interns’,” Fitzgibbons adds. “It is great to churn through reams of data, but you must check it. You need the research basics under your belt so you can identify when the AI outputs seem off kilter.”
Core to developing AI skills are a willingness to try new things and a psychologically safe environment. “Build a culture where failures are not ‘death to your career’ but rather a way to learn,” advises Fitzgibbons. “When piloting anything new, build in a post-mortem period to evaluate what worked and what didn’t.”

Shari Aaron and Shayna Beckwith, of Radius, class “higher-level thinking, critical thinking and the ability to analyse data across a variety of sources to extract the most meaningful and impactful information” as skills that remain uniquely human – as is the ability to apply good judgement in a world where misinformation can be rife.



You've got this!

Many in our industry are feeling a little overwhelmed by the tidal wave of opportunities, benefits and complexities that generative AI presents. Many wonder how their roles might change, or if they will even disappear. Most are asking what skills they need to develop to not just stay relevant, but also to thrive.


Generative AI is an invaluable resource and delivers efficiencies in data discovery, ideation, content creation, and technical programming. This, in turn, allows us to operate at speed and scale, freeing up consultants, for instance, to spend more time on creative solutions to client challenges.

But what if you’re in a client-side team or, like my team at Opinium, you’re at a mid-sized people-focused insight consultancy and now suddenly need to know how to get the most out of this incredible new ResTech? What skills do (human) researchers need to work on to make the most of these benefits?

The good news is that we have many of these skills already. High on the list must be our communication skills and a willingness to experiment.
You (usually) wouldn’t expect a team member to read your mind about strategic project priorities or deliverables formatting. These technologies can make assumptions and infer what you want, but the best results come from spending the extra effort to clearly define your problem and outline the specific steps or formatting you want back.

No training or webinars can fully substitute for experimenting with these tools to understand where they excel and what types of tasks you need to steer away from or bring more scepticism to the outputs. Remember, machines will never be able to replace collaboration, empathy and a deep understanding of research (and human) needs. See, you’ve got this.

Client-side researchers, their stakeholders, and agency insight consultants will still need to communicate with each other, to deeply understand what we all want to achieve through the research, why it matters, and how best we can use it to drive forward.

Humans are still best at understanding cultural nuances, which can affect how we write a specific survey question, or inform how we approach a sensitive research topic in an interview, for example.

Businesses must foster a culture of continuous learning to drive efficiency, creativity and innovation.

Data-driven storytelling skills, cross-disciplinary collaboration and effective communication need to be at the core. You’ve got this!

James Endersby, chief executive, Opinium

0 Comments