In the study, LLMs reproduce human purchase intent via semantic similarity elicitation of Likert ratings, based on research by PyMC Labs researchers based in Tallinn, Estonia, and Colgate-Palmolive in New York, the research team focused on semantic similarity rating (SSR), a method that gains textual responses from LLMs and maps these to Likert distributions – measures which help visualise the frequency of responses to a question.

The experiment tested a dataset comprising 57 personal care product surveys carried out by an unnamed leading personal care brand, with 9,300 human responses analysed overall for the project and used to compare to results from synthetic respondents generated by a LLM.

The study found that SSR achieved 90% of human test-retest reliability, while also maintaining realistic response distributions.

The synthetic respondents were deemed by the research team to have provided rich qualitative feedback, with response behaviour regarding age and income level, in particular, mirroring actual human response behaviour to a good level.

The study concluded that when paired with appropriate elicitation methods, LLMs could act as valid synthetic consumers for concept testing.

Speaking to Research Live, Nina Rismal, senior researcher at PyMC Labs, said that the findings “represent both a disruption and a major opportunity”.

Rismal said: “We are still in the early stages of integrating AI into market research. At the moment, many applications simply replicate traditional methods – what used to be human surveys are now ‘synthetic surveys’ conducted with LLMs. It’s true that this trend could reduce the need for human data collection in some areas.

“However, the real potential lies beyond imitation. AI enables the simulation of entire virtual societies where new products, campaigns, or ideas can be tested dynamically. In such environments, researchers could go beyond gathering opinions to observing behaviour and emergent dynamics – seeing not only what people (or models) say, but what they do.”

Rismal said that market researchers need to understand the technology and re-imagine their role, while retaining existing expertise in psychology, sociology and consumer insight.

In addition, LLMs should continue to be analysed for the extent to which they can reproduce human-like behaviour across domains, Rismal said, adding: “Our results suggest optimism. Under specific conditions, and within well-defined contexts, LLMs can indeed act in ways that mirror human decisions.

“This does not mean they think like humans or possess human-like cognition. But it does demonstrate that their outputs can align with human behavioural patterns – a promising sign for both applied and theoretical research.”