University researchers warn of response substitution in surveys
David Gal and Derek Rucker’s study, ‘Answering the unasked question’, unearthed what they call ‘response substitution’ – a situation where consumers misrepresent their opinions in surveys because they want to express opinions that were not asked of them.
For example, a consumer completing a survey about a restaurant where they received great food but poor service would use the survey as an opportunity to express the dislike they felt toward the organisation, even if the survey focused on the quality of the meal.
Gal (pictured) said: “Since the survey does not ask about the service at the restaurant, the only means for this unhappy customer to express his general disapproval of the service is by being negative about the food. Basically, respondents have the potential to project their negative attitudes onto the wrong aspect of the business so they can share their opinions. This bias can be a major source of misdirection for a company.”
The professors tested their theory by conducting a survey about confectionery where participants were asked about the quality of sweets produced by a company with “immoral business practices”. The researchers found that respondents who viewed a company’s moral behaviour as an important factor in their purchasing decisions rated the sweets poorly compared to those who did not take a company’s practices into consideration.
Gal and and Rucker concluded that those respondents felt strongly about the manufacturer’s business practices were using the quality rating to vent their disapproval, rather than focusing on the standard of the sweets.
The answer, the researchers claim, is for survey authors to explicitly provide a place for people to express their opinions on issues, rather than letting their bias slip out via their answers to unrelated questions.
We hope you enjoyed this article.
Research Live is published by MRS.
The Market Research Society (MRS) exists to promote and protect the research sector, showcasing how research delivers impact for businesses and government.
Members of MRS enjoy many benefits including tailoured policy guidance, discounts on training and conferences, and access to member-only content.
For example, there's an archive of winning case studies from over a decade of MRS Awards.
Find out more about the benefits of joining MRS here.
3 Comments
Annie Pettit
15 years ago
I have seen and done this myself. When respondents are given the opportunity to provide "additional" information, they are more than happy to share opinions about topics not in a survey. And whether survey responders realize it, there is always the unconscious spill of secondary opinions onto primary questions. Yet another fascinating case of the manifestation of psychological behaviors in marketing research.
Like Reply Report
Gary Halpern
15 years ago
I am not surprised to learn this. In fact, we believe that the occurrence of "venting" responses are as, if not more common then the occurrence of socially acceptable responses. At SBR we have developed customer satisfaction techniques designed to positively utilize customers' desire to vent, eliminate the negative effects of this behavior, and provide our clients with a more strategic and tactical customer satisfaction tool then is commonly available.
Like Reply Report
Philip DERHAM
15 years ago
Good to have this issue explored and reported. Failing to give respondents (in any survey method) opportunities to tell what is important to them, as well as what is important to the researchers and clients, is to miss a proper understanding of the consumer's behavior that can help the client better meet consumer needs. It is not difficult in paper or online surveys particularly, to allow room for open ended response, and so opportunities for respondents to tell what was important to them. Researchers need to test their surveys for this as much as testing skips and question logic. The requisite approach is to remember survey response is always voluntary, respondents choose to participate and that willing engagement should be respected, and so seeking what is important to respondents as well as to the researchers, will ensure this engagement. This paper is a useful reminder of the respect need in marketing research.
Like Reply Report