FEATURE1 June 2011

Keep calm and embrace failure


Companies need to experiment to survive, says Tim Harford in his new book. But they must accept that not all their efforts will result in success.


Biologists have a word for the way in which solutions emerge from failure: evolution. Often summarised as survival of the fittest, evolution is a process driven by the failure of the less fit. Disconcertingly, given our instinctive belief that complex problems require expertly designed solutions, it is also completely unplanned. Astounding complexity emerges in response to a simple process: try out a few variants on what you already have, weed out the failures, copy the successes – and repeat. Variation, and selection, again and again.

In a market economy, variation and selection are also at work. New ideas are created by scientists and engineers, meticulous middle managers in large corporations or daring entrepreneurs. Failures are culled because bad ideas do not survive long in the market place: to succeed, you have to make a product that customers wish to buy at a price that covers costs and beats obvious competitors.

Many ideas fail these tests, and if they are not shut down by management they will eventually be shut down by a bankruptcy court. Good ideas spread because they are copied by competitors, because staff leave to set up their own businesses, or because the company with the good ideas grows. With these elements of variation and selection in place, the stage is set for an evolutionary process; or, to put it more crudely, solving problems through trial and error.

“Trial and error is a tremendously powerful process for solving problems in a complex world. We will have to make an uncomfortable number of mistakes, and learn from them”

Try, try again
Whether we like it or not, trial and error is a tremendously powerful process for solving problems in a complex world. We will have to make an uncomfortable number of mistakes, and learn from them, rather than cover them up or deny they happened, even to ourselves. This is not the way we are used to getting things done.

Variation is difficult because of two natural tendencies in organisations. One is grandiosity: politicians and corporate bosses both like large projects – anything from the reorganisation of a country’s entire healthcare system to a gigantic merger – because they win attention and show that the leader is a person who gets things done.

The other tendency emerges because we rarely like the idea of standards that are inconsistent and uneven from place to place. It seems neater and fairer to provide a consistent standard for everything, whether it’s education, the road network or the coffee at Starbucks.

If we are to take the ‘variation’ part of ‘variation and selection’ seriously, uniformly high standards are not only impossible but undesirable. When a problem is unsolved or continually changing, the best way to tackle it is to experiment with many different approaches.

If nobody tries anything different, we will struggle to figure out new and better ways to do anything. But if we are to accept variation, we must also accept that some of these new approaches will not work well. That is not a tempting proposition for a politician or chief executive to try to sell.

See what sticks
If any company can be said to embrace trying new things in the expectation that many will fail, it is Google. Vice president Marissa Mayer says that 80 per cent of Google’s products will fail – but that doesn’t matter, because people will remember the ones that stick.

Fair enough: Google’s image seems to be untarnished by the indifferent performances of Knol, a Google service vaguely similar to Wikipedia which didn’t seem to catch on; or SearchMash, a testbed for alternative Google search products which was labelled ‘Google’s Worst Ever Product’ by one search expert and has now been discontinued.

According to the influential TechRepublic website, two of the five worst technology products of 2009 came from Google – and they were major Google products at that, Google Wave and the Android 1.0 operating system for mobile phones.

Yet most internet users know and rely on Google’s Search, Google Maps and Image Search, while many others swear by Gmail, Google Reader and Blogger. As long as the company doesn’t pour too much money into failing products, the few big successes seem to justify the many experiments.

“If any company can be said to embrace trying new things in the expectation that many will fail, it is Google. 80 per cent of Google’s products will fail – but that doesn’t matter, because people will remember the ones that stick”

The experimental method
Some years ago, a craft and fabric chain called Jo-Ann Fabrics offered its customers a surprising deal. It wasn’t surprisingly creative or surprisingly generous. In fact, it was surprisingly lame: buy one sewing machine, get a second one at 20 per cent off. Who on earth wants to buy two sewing machines? But the deal was also surprisingly successful.

Customers found that the prospect of saving 10 per cent per sewing machine was tempting enough to make it worth having, so they went hunting for friends who might also want to buy a sewing machine. In short, the quirky offer turned out to be an unexpected way to recruit amateur sales staff.

More interesting than the special offer was how it was discovered: Jo-Ann Fabrics was using its website, JoAnn.com, as a laboratory. Different customers would automatically be shown different website designs and different offers, the particular combination chosen at random by a computer.

The deal for bulk-buyers of sewing machines was one of the unlikely successes that this random process discovered, and the use of randomised experiments on the website more than tripled revenue per visitor.

As Ian Ayres explains in his book Super Crunchers, stories like that of Jo-Ann Fabrics are becoming more and more commonplace. Credit card providers have long used combinatorial experiments in their junk mail – these experiments turbocharge the randomised trial method by layering multiple randomisations on top of each other to generate a very rich set of data. The results are all used to refine the mailshot and to hook more customers.

But whereas these experiments once required statistical experts and cutting-edge computer technology, they are now very easy to run online. Anyone can buy two or more advertisements on Google AdWords and see which one works best. (Ian Ayres did just that, which is why his book is called Super Crunchers and not his own favourite, The End of Intuition.) Or for bigger projects, professional help is available to unleash the full power of combinatorial experiments.

Such experiments aren’t limited to the web. Supermarkets can and do randomise their price offerings, their shelf placement, the vouchers they send to customers with loyalty cards, or the design of the advertisements they place in local newspapers.

Fast-moving consumer goods companies play with the packaging of key brands. Publishers sometimes offer several different covers to a magazine or a book and see what sells.

Experiments have been going on in corporations behind the scenes for over a century. Thomas Edison may have been known as the Wizard of Menlo Park, but his experimentation hit a systematic, industrial scale in 1887 after he built large laboratories a few miles north in West Orange, New Jersey.

Famous for saying, “If I find 10,000 ways something won’t work, I haven’t failed. I am not discouraged, because every wrong attempt discarded is just one more step forward,” he also commented more directly on the industrialisation of the trial-and-error process: “The real measure of success is the number of experiments that can be crowded into 24 hours.”

Embrace failure
Experimenting can be a frightening process. We are constantly making mistakes, not knowing whether we are on the right lines. Kathryn Schulz, in her elegant book Being Wrong, describes the state of profound uncertainty that comes with feeling wrong about some fundamental belief. She compares it to being a toddler lost in the heart of Manhattan.

But experimenting doesn’t have to be like that. On the very same day on which I read Schulz’s words, my three-year-old daughter was lost in the centre of London – on the South Bank, a car-free space that is otherwise just as bewildering as Times Square. And it didn’t bother her in the slightest: she bolted out of the door of a café and began to play hide and seek. Witnesses told her increasingly frantic family that she had sauntered along the bank of the Thames, playing on the street furniture, ducking behind benches, dancing around and exploring a space she found delightful. For the ten minutes during which she was lost, it seems that she felt absolutely secure that she would find her family or that her family would find her.

The ability to adapt requires this sense of security, an inner confidence that the cost of failure is a cost we will be able to bear. Sometimes that takes real courage; at other times all that is needed is the happy self-delusion of a lost three-year-old. Whatever its source, we need that willingness to risk failure. Without it, we will never truly succeed.

Copyright © Tim Harford 2011. Extracted from Adapt, published by Little, Brown, £20.00

Reader offer
Research readers can buy Adapt for £18, including postage and packaging. Call 01832 737525 quoting reference LB 140