People are creatures of habit, buying the same brands week after week. While loyalty works in a company’s favour for a chosen brand, for new products – those trying to build market share or break into a new category – it’s a force against them.

But scientists at University College London (UCL) and customer science company dunnhumby have created a new model that can predict when people are most likely to try different brands or products. 

The researchers – led by Professor Brad Love of the UCL’s Department of Experimental Psychology and The Alan Turing Institute – drew on the Tesco Clubcard database to look at people’s purchasing habits.

They looked at six categories: beer, bread, coffee, toilet paper, washing detergents and yogurt, and examined the choices of more than 280,000 anonymised individuals in supermarkets over several years. 

People with at least 50 purchases within a specific product category were selected to get sufficient data for individual analysis and to retrieve customers likely to use their Clubcard with every visit. Each product category gave between 39,105 and 79,988 datasets. 

The categories were chosen because all involved frequent purchasing, which meant they produced sufficient data. But other factors in the category choice were the diversity of products within the category, and that the competitor products were obvious. 

“For some products it’s not always clear who the competitors are, so we tried to choose those where it’s really clear what the alternate brands are,” says Love. 

The study wanted to identify when shoppers were in ‘exploiting mode’ (taken from when honeybees, foraging for nectar, continue to exploit their current location – i.e. people are buying the same product) and when they were in ‘exploring mode’ (ready to explore new locations for nectar, or, in this case, buy new brands/products).

“We looked at how people explore and exploit studies in the laboratory, where objective rewards are used like money and where people treat choice almost like information. So the longer it’s been since one has explored, the more likely one is to exploit. With objective rewards, people treat choices as an opportunity to gain information about competing options, as if they become more curious and drawn to options that are less familiar or well known to them.

“But people don’t choose products that way – it’s exactly the opposite; the more one buys something, the less likely one is to explore,” explains Love. “In these cases, it’s as if people are trying to align their preferences and choices by changing their preferences to match their recent choices.

“It’s not as if people get curious about the alternatives – if someone drives a BMW they become a BMW person – it’s not that they were a BMW person and went out and got a BMW. So, it’s relatively arbitrary what ends up in someone’s shopping basket, but once they start buying it, their tendency to keep buying it strengthens.

“However, once you break that cycle, it seems to reset; it’s not that we keep buying the same stuff again and again and again – people’s exploration rates that we looked at were the same in the first 2.5 years as the last 2.5 years,” he says. “I don’t really know why people eventually explore – maybe it’s that the product isn’t on the shelf  – but, when they finally do, it opens them up to more exploration.”

The research identified a clear pattern; that people are less likely to explore the longer they’ve been exploiting.

“People vary in how strongly they manifest that pattern, and if someone strongly shows that pattern for one product category, they’ll show it for another,” says Love. “Some people strongly show the coherency maximising approach – the more they exploit, the less likely they are to explore. There are a few people – about 10% – who become more likely to explore the longer they’ve been exploiting.” 

The model built by the team excluded many of the factors normally associated with retail such as price, time of day, location. “It’s a really simple logistic regression model. [It considers] one thing that predicts whether people explore or not: how many times in a row they’ve exploited,” says Love.

“The model is ridiculously simple and, because it’s so simple, you can fit it to individual shoppers’ data, so we can predict what an individual wants to do. There’s an issue with predictive models that the more complex the model is, the more data you need to use it properly. Simpler models are limited in many ways but they can be more robust. What’s nice about this data is that we could probably predict something about a new product that we don’t yet have data on, because what people do for one product class seems to spill over to another.”

They tested the theory with two coupon studies – one using existing data and another that involved running a new campaign to try to replicate the results. They sent coupons to thousands of people and used the model to predict who would
use them. 

From the first experiment, the researchers just had data on those coupons that had been redeemed, so they could see the successes, not the failures. “We only had the number of days, not the redemption rate, so our thinking was, if something was a really exciting coupon and really important to you, you would redeem it more quickly. What we found was that when someone received an ‘exploit’ coupon – for instance, a coupon for something they were already buying – customers were faster to redeem it when they were on a long exploit streak. We found the exact opposite when someone received an ‘explore’ coupon for a novel product,” says Love.

The findings indicated that shoppers who had recently switched brands were twice as likely to use the coupons to try a new product.

So what can marketers learn from this model? “From a marketing perspective, I don’t think it can be used as a strategy; it suggests that rather than try to convince someone to switch to a particular product, the marketer should wait for the person to enter a period of exploration. When people enter these periods they are more open to just about anything,” says Love. 

“It’s not really worth putting in the resources to change people’s minds when they’ve embarked on a long exploitation period. It goes the other way too; if I had a loyal customer and, all of a sudden, they faltered and purchased other products, I’d come back at them with the most amazing offers to try to get them back into the fold. These tendencies seem to self-reinforce so, once they drift away, it gets harder and harder to bring them back.”

Whether trying to poach customers, retain them, or change people’s behaviours to be more social or eat other foods, the time to strike is not when people are entering the exploitation period but recently exiting it. 

Coherency-Maximising Exploration in the Supermarket by Peter Riefer, Rosie Prior, Nicholas Blair, Giles Pavey and Bradley Love was published in Nature Human Behaviour.