Analysing AI outputs 

Generative AI is reshaping how research is being done. However, a key skill is understanding where AI can be actually be helpful, and to have the subject-area knowledge to understand whether the information gleaned from it rings true. AI is not infallible, and researchers need to be able to spot abnormalities, hallucinations and missteps in its outputs. 

“Researchers need to become highly effective users of AI while retaining the expertise and critical judgement required to challenge it,” says Ray Poynter, chair of Esomar’s professional standards committee and managing director at The Future Place.  

“The technology is developing so quickly that this demands continuous learning, experimentation and, at times, unlearning. AI can produce impressive work, but it can also make basic or plausible-sounding mistakes. 

“Researchers who understand both its capabilities and its limitations, and who can work with it iteratively rather than simply accepting its outputs, will be increasingly valuable. In the future, the differentiator will not be access to AI, because everyone will have that, but the ability to use it intelligently, critically and responsibly.” 

Rebecca Cole, chief executive at Cobalt Blue and president of the Market Research Society, adds: “Everyone talks about learning to prompt AI tools well. Fair enough, but it’s not really the skill that matters most.  

“What matters more is critical evaluation of AI-generated outputs: synthetic data that doesn’t hold up, an AI-moderated interview that missed the point, a summary of verbatims that’s smoothed over something important. Clients often won’t notice this. Researchers need to.” 

Experimentation 

New technologies bring new opportunities but also risks in their use. Future researchers will need to be willing to trial new tools and learn as much through iteration as by rote. 

Kelly McKnight, executive director at Verve explains: “Traditional research training revolves around method – designing surveys, conducting interviews and analysing transcripts to understand behaviour. The AI-focused roles learn through experimentation.  

“AI-ready researchers test prompts, compare model responses and run simulations to see how simulated audiences behave. They run A/B tests on prompts, personas or knowledge bases – changing one variable and observing how the system responds. As one described it, the process often involves ‘trying things, seeing what happens, and refining it’.” 

Cole says it isn't necessarily about individual technologies or approaches: “Whatever tool or method is cutting-edge right now won’t be in two years. What actually sets people apart isn’t what they know today, it’s how quickly they can pick up something new and work out if it’s any good.  

“That’s harder to put on a CV than a list of tools you know – but it’s what employers should actually be looking for.” 

Knowing client needs 

Research partners will need to double down on anticipating the requirements of clients and stakeholders and using research to propose useful solutions to problems the organisation faces. Researchers should be proactive in helping get to the central issues brands face. 

“Understanding the real business need behind a research request has always been important, but it will become even more central to a researcher’s career,” says Poynter.  “As AI automates more of the process of collecting, analysing and presenting information, the greater value will lie in defining the right problem, identifying what has been left unsaid and ensuring that the work leads to a useful decision or action.  

“Researchers who can connect evidence to business needs will remain relevant because they are not simply delivering projects; they are helping organisations make better choices.” 

With AI able to provide easy answers, in-depth knowledge of a subject area and the desire to properly understand clients’ business will prove invaluable. 

Paul Griffiths, founder at Client Advocates, says: “Where market researchers are really going to have to step up is on commercial thinking. In that context, doing your research around a particular brand or around a particular customer issue or around a particular client need. 

“What the client is still going to need is somebody to sit there and go, ‘right, this is what you need to do as a result of that’. That comes back to commercial knowledge, and understanding and uncovering what those commercial needs are, what those commercial requirements are and understanding the individual you’re talking to.” 

Storytelling 

Successful researchers should already be adept at drawing out the key insights from information. However, the ability to craft narratives will prove crucial to helping to win over senior staff, both internally and externally, and prompt action. 

“One of the most important considerations is influence,” says Grant Feller, former journalist and founder of Every Rung. “Researchers need to be more influential in their organisations; their teams need to be closer to decision-making.  

“Too often, leadership doesn’t appreciate the value of the research and that’s partly because of how it’s distributed and accessed. It sits in PowerPoint decks or dashboards and lacks compelling reasons to engage with it. Storytelling is not about story arcs or tighter sentences or innovative design – it’s about influence and confidence.” 

Critical thinking and judgement 

With AI automating many research processes, researchers need to be adept at deciding what the key trends are and how the company should respond. This means having both the curiosity to explore ideas, including those outside the box, and the judgement to take the right approach.  

“As AI takes over more of the mechanical work, the real value shifts to the things humans do best: framing a better question, applying judgement and knowing which answers to trust,” says Nicki Morley, global managing director, innovation solutions, at Kantar.  

Morley warns that there is a risk of believing AI is right simply because it sounds so confident. She says: “The researchers who’ll stand out are the ones with enough expertise, curiosity and scepticism to keep questioning the answer. Learning how to use AI is important, but learning how to challenge it, coach it and know when not to accept a first answer may be even more important.” 

McKnight says: “AI systems are powerful, but they are also prone to bias, distortion and overconfidence. Much of the work involves spotting when something feels wrong – questioning assumptions, adjusting prompts and ensuring AI personas behave in ways that feel recognisably human.  

“AI outputs are often technically correct but strategically absurd: internally coherent, yet disconnected from how real people actually behave in culture and in markets. In other words, the task is not simply to accept the output – but to judge it. Both analysis and judgement are forms of research thinking, but as AI takes on more of the analytical workload, human judgement – and comfort with ambiguity – becomes more valuable, not less.” 

World-building 

Synthetic consumers have grown in prominence in recent years. Researchers will have to understand when synthetic data can be turned to and how to deploy it  in such a way that the work can get the very best results. Researchers will also need to know the strengths and weaknesses of using synthetic personas and digital twins, and which projects and methodologies naturally suit the approach. 

McKnight says: “The goal is to create worlds where AI can behave like recognisable human audiences: consumers, experts or communities whose perspectives can be explored and tested.  

“Traditionally, early research roles looked different. Much of the craft involved learning how to structure insight from evidence – charting data, summarising interviews and gradually building logical arguments that lead to a clear conclusion. Both approaches produce insight. But one focuses on building the argument, while the other focuses on building the world that generates it.” 

Coding 

Traditionally, this has not been in the realm of the researcher. However, just as understanding a foreign language is useful when working overseas, knowing coding languages can help researchers converse with those in the organisation who are developing and maintaining the technology on which the future workplace could become dependent.  

Cole says researchers should seek to develop their experience in “basic scripting (Python/R), understanding of how AI models are trained and where their blind spots are, and being comfortable with data visualisation tools”, adding that the motivation is “not to become data scientists, but to hold intelligent conversations with the people who are”. 

Understanding your value 

Ultimately, researchers who successfully adapt to a changed industry are going to have to know where they fit into the new landscape and make their niche profitable. 

Griffiths says: “We have, as an industry, been so involved and so focused on what we do that we’ve quite often lost sight of the value we create and the outcomes we generate, and therefore the value that we should be charging.  

“The point is to tell the client: 'I think this is a good way of fixing your problem’.”