Doing what you say: Online behaviour and research

x Sponsored content on Research Live and in Impact magazine is editorially independent.
Find out more about advertising and sponsorship.

What did you do yesterday? Or the day before that? Can you recall your actions on 11 February 2021? If not, why would you assume a consumer can?

Survey data in market research can sometimes fail to capture the information that companies desire, looking at people’s faulty recollections of past behaviours rather than observed actions, and using that as the basis of future analysis. The ‘say-do gap’ has been observed in numerous settings, in the physical retail world and in online shopping – but can it be measured?

Customer experience firm Disqo set out to measure the say-do gap across three shopping categories: automobiles, travel and groceries. The result was the report Mind the Gap, published earlier this year. The three categories were chosen because of the variety in typical purchase cost and frequency, which the company expected to impact the accuracy of respondent recall when asked about their purchasing behaviour.

The research used Disqo’s consumer insights platform to survey a nationally representative sample of 53,749 consumers about their shopping behaviours, and then compared the answers with their actual digital behaviours collected over a 30-day period.

Only ‘active shoppers’ were included in the study, namely those who recorded five or more ‘digital events’ – such as keyword searches, website visits or mobile app launches – across the month.

“As traditional sources of digital behaviour data such as cookies, mobile IDs and IP addresses are getting harder to work with, because they are constantly being depreciated, some clients are turning towards asking consumers about their behaviours, rather than using observed behavioural signals,” says Anne Hunter, vice-president of product marketing at Disqo.

“We wanted to see if that was an accurate substitution, or whether there were still requirements to have passive data collection.”

The research found that an average of 49% of respondents were “highly confident” in their ability to recall their actions correctly, while 3% were deemed “not confident”. However, 38% of respondents’ self-reported shopping activities contradicted data on their real, observed shopping behaviours, highlighting the prevalence of the say-do gap. The say-do gap was recognised in all three product categories included in the study.

The biggest difference from the total respondent average was seen among active shoppers who said they had not shopped online in the past 30 days, even though online behavioural data suggested they had. Only 38% said they were highly confident in their recollection, a full 11 points below the respondent average; 12% of this group said they were not confident in their memories.

“The most shocking finding for me was the inverse relationship between people’s confidence in remembering what they do versus their accuracy in remembering what they do,” says Hunter. “The people who are most confident were also the most likely to be wrong. When researchers are setting up surveys, often the people who say ‘I don’t know’ might be the most accurate, whereas the people who give an answer might be overconfident. Surveys need to be used to understand how people feel, as they are immediate, measurable reactions.”

Take this example: if a survey asks people if they eat healthy food such as quinoa regularly, the respondents might not give the survey good information. But you could survey people who are actually observed buying it, and then ask them why they did so. That gives you the ability to market the product to gain more customers. Hunter maintains that behaviourally targeted surveys for people who already engage in the habit “gets you ahead of the trend”.

Hunter uses the example of the environment, which is a top concern for many people who do not follow through their intentions when purchasing household items, for example, or choosing a car or holiday abroad. Convenience and money remain primary factors in why we make purchasing decisions, despite most people’s concerns over climate change.

“Wants and beliefs are not always in line with our actions,” Hunter says. “For better or worse, often our desires and hopes get ahead of our actions, which can inspire those actions over time.”

This article is from the July 2022 edition of Impact magazine.