The researchers fielded a survey on political opinion and consumer insights to a politically representative online sample of 996 US participants and supplemented it with AI-moderated qualitative interviews.

They then attempted to reproduce the human data with LLM-simulated respondents at increasing levels of persona fidelity, from no information at all, to demographic personas, to demographics plus verbatim idiographic interview transcripts.

The study found that individual-level simulation with demographic personas – a commonly used method for generating synthetic data – tripled distributional error compared with asking the model for an aggregate distribution.

Andrew Gordon, head of research sciences at Prolific and the paper’s first author, wrote in a LinkedIn post: “Simulating 996 individuals should beat a single zero-context guess.... but it didn't. It roughly tripled the distributional error, with personas collapsing onto stereotyped modal answers. Interview transcripts helped, halving the variance collapse, but not enough to beat the zero-context baseline.

“The bigger fix was in changing the prompting method. Eliciting a probability distribution per persona rather than one forced answer cut error by 35-48% while keeping individual-level data. But post-stratification (MRP) couldn't touch what remained, and no method matched the accuracy of a second human sample.”

Gordon added: “Our worst method at reproducing the distribution was among the best at calling the plurality winner, so judging synthetic data on one metric rewards exactly the wrong methods.”

• Gordon, Andrew and Petrova, Nora and Burden, John and Bosch, Oriol and Ding, Ning, Do Richer Personas Improve LLM Survey Simulation? A Fidelity Paradox (August 10, 2026 ).