Walr publishes study linking respondent experience with research quality

US – The wording of a survey question has a measurable effect on data quality, according to a study by data collection company Walr.

woman using laptop at desk

Walr conducted a survey with 2,000 nationally representative respondents in the US, exploring attitudes towards AI and autonomous driving.

Half of the respondents were issued with a survey containing standard, neutral wording and scales, while the other half used a more conversational survey design with natural language. Both groups passed identical fraud and quality controls.

The A/B test found that the experimental conversational survey’s responses scored higher for data quality.

For example, one question, focused on trust in autonomous vehicles, saw ‘junk’ responses fall from 19.0% in the standard survey design to 8.4% in the conversational design, while mean length of interview increased for the experimental design, according to the paper.

Additionally, 53% of respondents to the conversational survey said they would take another survey, compared with 46% of respondents to the standard version.

Staci Kinney, vice-president of data consultancy at Walr, said: "We've spent years getting better at one question: is this a real, engaged human? This research asks a second one. Once we know we're talking to a real person, how hard are we working to talk to them like one?"

We hope you enjoyed this article.
Research Live is published by MRS.

The Market Research Society (MRS) exists to promote and protect the research sector, showcasing how research delivers impact for businesses and government.

Members of MRS enjoy many benefits including tailoured policy guidance, discounts on training and conferences, and access to member-only content.

For example, there's an archive of winning case studies from over a decade of MRS Awards.

Find out more about the benefits of joining MRS here.

0 Comments


Display name

Email

Join the discussion

Newsletter
Stay connected with the latest insights and trends...
Sign Up
Latest From MRS

Our latest training courses

Our new 2025 training programme is now launched as part of the development offered within the MRS Global Insight Academy

See all training

Specialist conferences

Our one-day conferences cover topics including CX and UX, Semiotics, B2B, Finance, AI and Leaders' Forums.

See all conferences

MRS reports on AI

MRS has published a three-part series on how generative AI is impacting the research sector, including synthetic respondents and challenges to adoption.

See the reports

Progress faster...
with MRS 
membership

Mentoring

CPD/recognition

Webinars

Codeline

Discounts