AI synthetic audience matches human polling in study

A study led by researchers at the London School of Economics and New York University found that an AI-based synthetic audience produced survey results closely matching those from human polling panels. The research compared one synthetic platform with seven online sampling services using responses from 7,755 UK participants.

The paper examined whether the source of survey respondents changes the answers researchers get, with implications for polling, market research and behavioural studies. It compared two opt-in panels, Prolific, two multi-source aggregators and two river samples recruited through Facebook and Instagram with a synthetic benchmark generated by Electric Twin.

All groups received the same questionnaire, covering financial outlook, grocery shopping habits, social media use and views on UK military intervention in Ukraine. Researchers applied the same recruitment targets and weighting approach across the samples, using age, gender, education and voting history to improve comparability.

Low-quality responses, including bots, duplicate accounts and failed attention checks, were removed before analysis. Polling firm Stack Data Strategy handled the human fieldwork.

The results showed that Electric Twin's synthetic audience fell within the range of outcomes produced by human samples across most questions. According to the authors, this was the first study of its kind at this scale to assess a synthetic audience platform alongside traditional online sampling methods.

The study also found marked variation between human sampling platforms even when survey design and quotas were held constant. Voting intention differed by as much as 20 percentage points between platforms, while statistically significant cross-platform differences appeared on all seven substantive questions.

Platform differences

River samples recorded the weakest data quality, with nearly twice the rate of failed attention checks and duplicate accounts seen in Prolific and traditional panels. Respondent profiles also differed: river samples skewed older, while Prolific leaned towards younger, more highly educated participants.

No single human platform consistently performed best when demographic representativeness, response variability and data quality were considered together. After demographic weighting was applied, differences in responses across platforms fell by more than 80%, indicating that sample composition, rather than response behaviour, drove much of the divergence.

That finding adds to broader concerns about the reliability of online opt-in polling and raises questions about how far results from a single panel can be generalised to the wider population. For commercial researchers, the study suggests that the choice of recruitment source alone can materially affect the strategic conclusions drawn from a survey.

The authors also presented synthetic audiences as a possible tool for exploratory work before commissioning larger human studies. The paper said the AI benchmark produced results close to human samples while avoiding some of the problems associated with online panels.

Michael Muthukrishna, Professor of Economic Psychology at NYU and LSE, commented on the findings. "This study offers some of the first solid, large-scale evidence that a synthetic audience can stand in for a human panel and get you to the same place. Across most of the questions we tested, Electric Twin's simulated respondents landed within the same range as the responses of real people. That's a significant finding for anyone doing research under time, budget or practical pressure. Synthetic respondents don't fatigue, don't forget and don't give in to social bias. This is an impactful moment because studies of this scale almost never get academic funding. This is one of the largest studies to stress-test the day-to-day tools of market and behavioural researchers alongside the next generation of AI tools that enable us to talk to sections of the population that might otherwise be hard to reach," Muthukrishna said.

The study was pre-registered, and the paper disclosed that it was funded by Electric Twin. It also disclosed that both authors hold shares in the company and serve in advisory or founding roles.

Commercial context

Synthetic research has drawn growing attention in marketing and audience analysis as companies look for faster ways to test messages, products and public attitudes. The study enters that debate with evidence that AI-generated respondents may replicate broad patterns in human polling, while also highlighting the instability that already exists across conventional online samples.

Electric Twin was used in the study as the synthetic benchmark. The company was founded by Alex Cooper and Ben Warner. It says organisations including News UK and Lebara use its system.

Ben Warner, Visiting Senior Fellow at LSE and Co-Founder of Electric Twin, said: "This study shows that at real scale, with independent oversight, a synthetic audience produced results that were consistent with human panels across most of what we tested. That's a strong indicator that this approach can be trusted for exploratory and iterative audience research. It also puts synthetic audiences in useful company. This study did not set out to proclaim one research method as the best - it set out to test methods properly, side by side, at a scale the industry rarely gets to see. Synthetic audiences earn their place in that conversation by being testable, repeatable, and inspectable in ways traditional sampling cannot match. This study isn't about crowning a winner, but about an industry looking to better understand its methods because that is the only honest path to better understanding the world."