University researchers warn of response substitution in surveys

David Gal and Derek Rucker’s study, ‘Answering the unasked question’, unearthed what they call ‘response substitution’ – a situation where consumers misrepresent their opinions in surveys because they want to express opinions that were not asked of them.

For example, a consumer completing a survey about a restaurant where they received great food but poor service would use the survey as an opportunity to express the dislike they felt toward the organisation, even if the survey focused on the quality of the meal.

Gal (pictured) said: “Since the survey does not ask about the service at the restaurant, the only means for this unhappy customer to express his general disapproval of the service is by being negative about the food. Basically, respondents have the potential to project their negative attitudes onto the wrong aspect of the business so they can share their opinions. This bias can be a major source of misdirection for a company.”

The professors tested their theory by conducting a survey about confectionery where participants were asked about the quality of sweets produced by a company with “immoral business practices”. The researchers found that respondents who viewed a company’s moral behaviour as an important factor in their purchasing decisions rated the sweets poorly compared to those who did not take a company’s practices into consideration.

Gal and and Rucker concluded that those respondents felt strongly about the manufacturer’s business practices were using the quality rating to vent their disapproval, rather than focusing on the standard of the sweets.

The answer, the researchers claim, is for survey authors to explicitly provide a place for people to express their opinions on issues, rather than letting their bias slip out via their answers to unrelated questions.