In 2001, my wife and I did something very strange. We sold our flat in London and moved to a flat near Sevenoaks.

The only strange word in that sentence is the second occurrence of the word ‘flat’. Most people who move out of London use it as an opportunity to buy a house.

As it happens, we found the place in a copy of The Week. There was a two-page feature on apartments for sale in listed buildings and we took a look. We were hooked.

But had we gone about searching on the internet, not in a magazine, there is almost no chance we would have learned of it. Instead, on arriving at Rightmove.com, we would have been asked ‘house or flat’. We would have clicked on ‘house’ and been shown no further properties that weren’t houses.

Since you are reading Impact, you already know about ‘order’ effects – that the order in which questions are asked affects people’s answers. But there is also strong path dependency in choice. 

Suppose you are trying to buy a house, find a partner, book a flight or choose a toaster online. The first thing you may try to do is reduce the choice set to a manageable number – ‘elimination by attribute’. For example, you search for toasters under £50. What you never discover is that, for £55 there is a glass-sided toaster which lets you see your bread as it browns. There aren’t many innovations in toaster-world, but this is one worth paying a few extra pounds for. But you never learn about it.

Railway and travel websites are worse still – they don’t even allow you to choose what’s important to you, but assume that you only care about minimising end-to-end journey time. If you want to get almost anywhere in western France from London by train, the trick is to change at Lille rather than Paris. But the algorithm never shows you that option because, though far easier – no changing stations in Paris – it takes longer. 

Similarly, if you wish to get from London to Bath, Bristol or Exeter, there are trains from Waterloo as well as Paddington. These trains are often insanely cheap, and first class advance tickets are a bargain. But unless you stipulate ‘travel via Salisbury’ you will never learn these trains even exist: put in Waterloo to Bristol and the National Rail website will tell you to take the tube to Paddington first.

This seems to me morally dubious – by hiding a cheaper option – not to mention extraordinarily inefficient, since it will lead to one line being overused and another barely used at all.

Now, if you are in a long-term relationship, ask yourself how many of your existing partner’s definable attributes you would have stipulated in advance. Yes, we all have a prior conception of what we want in a partner (or a toaster), but in reality we fall in love with someone or something and then post-rationalise our choice – known as adaptive preference formation: we don’t know why we want something until we want it.

Would internet searches actually be better if they incorporated a degree of randomness? So that they injected unasked for wild-card results alongside the requested results? My view is yes. 

In many ways, markets, like termite hills, are the product of emergence – where a collection of people performing simple tasks collectively produce something remarkable. It seems to me that one of the reasons why markets possess such remarkable collective intelligence is that the individual decisions taken within them are messy. 

People may choose to buy the same thing for entirely different reasons. In choosing a car, performance, fuel economy, safety and environmental considerations will be deal breakers for some people and irrelevant for others. The net effect is that most cars are pretty good across a range of measures, precisely because they aggregate different people with different decision-making styles. 

So, by imposing a regimented uniformity on consumer decision-making, online interfaces may be making markets worse rather than better. In a complex system, you do not necessarily optimise the whole by optimising each of its constituent parts.

This article was first published in Issue 23 of Impact.