Transforming to Survive

x Sponsored content on Research Live and in Impact magazine is editorially independent.
Find out more about advertising and sponsorship.

It’s unfortunate that the two most famous quotes that damn market research are attributed to two of history’s most innovative entrepreneurs. “Some people say, ‘give the customers what they want’, but that’s not my approach,” said Steve Jobs. “Our job is to figure out what they’re going to want before they do… People don’t know what they want until you show it to them. That’s why I never rely on market research.” 

Henry Ford – whom Jobs referenced in his quote – also famously said that, if he had asked people what they wanted, they would have said a faster horse. 

But Apple does perform market research and Ford – to the best of our knowledge – never actually said anything about the public’s taste for faster horses. Indeed, his quote doesn’t exist in print until 2002; even The Henry Ford museum can’t locate the source. And Ford – who innovated with the Model T and then froze the design while he optimised his production line – might have profited from listening more to his customers between 1921 and 1926, when his market share halved as General Motors used customer research to out-innovate him. 

For those who believe innovation springs from flashes of genius rather than a collaborative process, Ford’s pithy accusation instinctively feels right – if only because we know people’s first response to innovative products is often negative, and the radical new design respondents ask for isn’t always popular when they are given it. 

There are certainly examples of this ‘faster horse’ theory. In his book Blink, Malcolm Gladwell tells the story of how the best-selling office chair in history, the Aeron, tested badly when it was shown to customers in 1994. It took two years to catch on because it didn’t look like people expected an office chair to look. 

So a ‘faster horse’ theory would probably argue that disruptive innovation can’t be researched. But perhaps it is more accurate to say that radical innovation can’t be researched using the same methods, the same respondents, and the same teams that we use to evaluate incremental innovation. 

This thought was first captured by a PhD candidate in the early 1990s. Clay Christensen went back to college after the ceramics firm he had taken to initial public offering (IPO) suddenly failed when market conditions changed. He wanted to find out, he recalled, “why smart companies fail”. His research was first published in the Harvard Business Review in 1995, in an article called Disruptive Technologies: Catching the Wave, which introduced the concept of a disruptive innovation. This was expanded upon a year later in The Innovator’s Dilemma, one of the books that helped shape the dotcom boom. 

Christensen’s influential idea was that mature, successful companies become unable to innovate precisely because they are so good at their existing jobs. They become experts at knowing their customers and in using them to identify the most profitable moves to make. Because of that, they fail to identify the next wave. Even if they do, they are ill-equipped to act on it, because the most radical innovations are initially not characterised by high purchase intent. This implies that if we apply research techniques that are used to evaluate the viability of incremental innovation to radical change, we will fail to spot the ‘next big thing’ until it’s too late. “Smart companies fail,” Christensen said later, “because they do everything right.” 

In which case, how can research catch Christensen’s wave? 

If you want to be a fundamental part of a client’s innovation process, argues Andrew Dalglish, director at Circle Research, you first need to be the type of agency that can drive that process. Traditionally, this has been something a management consultancy does, but there are limits to a consultancy’s ability to guide innovation – not least because a large firm has fixed models and processes that don’t always fit with a client in the middle of a messy transformation. However, the skills of a consultant may be useful in helping to define, as well as progress, an innovation process – and, as a result, Circle recruits heavily from that background. 

“If research is going to drive transformation, you can’t just say ‘you need to change this’. You need to help work out how change will happen in a cross-functional working group,” Dalglish says. “You need to challenge the client at the beginning and ask what the real problem is that we are trying to solve. Often, the problem they think they are solving is a symptom of a much deeper issue.” 

Circle frequently recruits staff from a non-research background and teaches them the skills. “The reason research often doesn’t drive action is because it’s getting superficial answers. We say, you hired us to do this, but we have found something different, and this type of conversation is far beyond the natural skill set of many researchers.” For example, Circle was asked by a distributor to analyse loyalty among its customers, so it could target its added-value initiatives better. However, the research revealed that the fastest-growing new entrants to a market were gaining share by selling on value, with more commoditised service: a classic disruptive tactic. As a result, what seemed like a simple customer satisfaction programme morphed into a bigger change-management project to redesign the distributor’s business around value rather than service.  

Dalglish offers four principles for capturing the innovation process in research. First, be involved early and often, especially if the client is responding to disruption. Structure the research around quick results, and next steps. 

Second, be iterative – break the project into steps, with each one dependent on the results from the previous iteration. Many agencies claim to be agile but iteration requires the researcher to be embedded in the design process, Dalglish argues. 

Third, don’t restrict the research to quant, qual, or even quant-and-qual. “This process demands that you take multiple sources and integrate them. For example, in BSI’s innovation project, Circle analysed internal company data and integrated desk research about the structure of potential markets. 

Finally, Dalglish argues that it is the researcher’s job to identify the people within the client organisation who can be agents of change – who will help create change and help develop their role. 

Early-stage value 

It’s often hard to structure a project if the client does not know exactly where it is heading – but it is at this early stage that the researcher may be most valuable. 

“Is it increasingly common for clients to want to innovate in a non-linear, disruptive way? Yes, especially when the category for the product that they are looking to launch isn’t mature yet,” says Chris Thompson, director of innovation at FreshMinds, who has a background in consultancy that he draws on to help clients innovate.  

“Until the mid-1990s, it seemed that all clients had to do to grow was phone a management consultancy and they would tell you which company to buy, or which emerging market to expand into. If you threw enough money at a strategy, you could get your company to grow just by taking your services to an ever-increasing audience. All the easy moves have been made now. Once you take those options off the table, innovation equals growth. It has become a core business function.”

Thompson agrees that, to be part of the innovation process, the research function has to be involved at the creative step. “Is market research the best way to generate innovation? I think we need to turn it on its head a bit,” he says. “We need to say an innovation programme is the best way to generate innovation – it’s just that market research absolutely underpins the programme.”

One of the ways in which Thompson generates insight for non-linear innovation is by solving the first part of Christensen’s dilemma – helping to scope the project by finding the atypical early adopters, or unusual users, who can visualise the world that the client wants to create – as FreshMinds did when it worked with O2 on its connected home project.  

To make disruptive innovation possible, Christensen recommends seeking out those odd, marginal customers who set the agenda, who probably don’t show up strongly in existing customer data because they are not very interested in what you are offering. “We call it seeking inspiration from the edge, rather than your general core consumer, so you need to look at extreme-use cases,” says Thompson. 

“If you’re going to launch a security product for the home, go and speak to a prison guard, who is steeped in an extreme-use case for security. They’ll be able to bring that inspiration.” 

Those extreme users are a bespoke recruitment task. Existing panels, even carefully filtered, are unlikely to get you the edge users you need, argues Thompson. They want engaged, articulate, forward-looking subjects who “can paint a vision themselves of a future they’d like to live in”. Most don’t make the grade. 

Work on the wild side 

Having embedded your agency in the creative process, and before you can locate the right ‘edge’ customers, it’s important to decide what type of project you are helping to create. However, innovation – especially technological innovation – is usually iterative, so it’s not always an easy question to answer. 

“Recently, a consumer electronics company client we worked with said: ‘We really want to design some future-forward concepts’,” says Guy White, CEO at Catalyx. “We asked who for, and the client replied: ‘We don’t know.’”

The project that followed has, in White’s words, been “quite wild”. His team have been working almost backwards; instead of finding an innovation for an audience, they have been looking to see if an audience exists for a technology. As part of the development, both evolve, and the project proceeds in stages of innovation, towards a goal that they are sure is out there, but which wasn’t defined at the outset. 

Unlike FreshMinds’ technique of building small, highly specific communities, Catalyx focuses on crowd-sourcing and co-creating, using communities of 100 people or more over periods of between one and six weeks. But this has something in common with Thompson’s approach – the research helps to co-create the product, rather than just testing something that already exists. 

“We’re basically trying to integrate a crowd of relevant people into a creative process. You’ve got creativity, validation, understanding and insight all happening at the same time.” White says. 

With interplay between innovators, the consumer and other experts, the crowd-sourcing process is an input to early-stage innovation. The weakness of traditional research, White says, is that a focus group or a survey generates insight, but this is then taken out of the hands of the insight team and mediated behind closed doors by a design team. “We’re trying to kind 

of open those doors. It is powerful because traditional methods really don’t get that very often.” 

Even for small, non-linear innovations, this can take the client to areas where they didn’t expect to go, or generate counter-intuitive ideas that need to be followed through with confidence. Recently, medical device company Helen of Troy contacted Catalyx with a problem: its in-ear thermometer was under-performing, but it was designed to the highest standards. “It came to us and just said: ‘This isn’t selling and we don’t know why’,” White explains. 

By asking the crowd to generate ideas, the design team didn’t just take the ‘faster horse’ option; instead, the crowd rejected all of the orthodox prototypes for a new component and generated its own ideas. The trick was to design a thermometer that looked less like a thermometer. Luckily, the client was flexible enough to listen, and start again from square one. 

Other agencies rely heavily on agile methods, themselves borrowed from software development. The lesson, says White, is that any research-led innovation process can’t use the research simply to validate an idea. It has to have the potential to put everything up for grabs: design, positioning, price. 

“We were able to shine a light on the process, which helped really open its eyes to the solution it thought it was going to get to,” he says, “which, basically, was completely different from where it ended up.” 

Targeting innovation 

According to Andy Goll, associate partner at Sparkler, being open to new ideas does not mean anything goes; it is quite appropriate to use research as a basis for radical innovation if it is based in robust science. An innovative, forward-looking approach is inherently risky, so one way to reduce risk is to think carefully about how to find out about the changes in a market or in preferences, so the client can be precise about where to target the innovation that results. 

Goll recently completed a project with Microsoft Advertising, which tried to bring innovative thinking to the received wisdom that people’s attention spans are getting shorter. This assumption has had a dramatic effect on online advertising, but Microsoft wanted to find out if it was universally true, or whether it could find smarter ways to reach smaller segments who behaved differently. 

Investigating the link between digital devices and attention isn’t easy, says Goll. “We’re getting these sort of tough briefs, where the answer isn’t obvious,” he adds. “We can’t use standard techniques, so we need to find different ways to look at this type of problem.” 

Self-reporting of attention wouldn’t work, and the tests that would identify the nature of attention – though established and validated – were restricted to laboratory environments. Sparkler’s response was not to identify a particular type of respondent, but to do a standard online survey of 2,000 respondents by getting them to play games that would measure sustained attention (concentrating for a long time), selective attention (concentrating when distracted) and alternating attention (being able to move efficiently between tasks). This was mixed with a smaller sample of ethnographic research. As expected, high-volume media consumption was found to reduce sustained and selective attention – but the surprising insight was that it improves alternating attention. 

The research also identified the ways in which attention manifests itself in different environments and for different age ranges. We train our brains to absorb information depending on the environment in which we spend our time, so attention may work differently in old and young people, gamers and television viewers – often in ways that are initially counter-intuitive and that challenge the received opinion of advertisers such as Unilever, Nestlé, Coca-Cola and P&G, which all requested presentations based on Sparkler’s work. For example, a disruptive programme of cross-platform advertising for a second screen might be extremely effective for an emerging category whose alternating attention is better than suspected. 

“We’re not creating cutting-edge science,” Goll says. “These models were developed in the 1950s and 1960s. But it’s about applying it in ways that are relevant. 

“Conventionally, we think of reduced attention span as losing something, but – creatively – we can think about it as adapting to our environment. Advertisers are increasingly trying to shout their message, but the innovation is in thinking carefully about why people give attention, and what they give it to.” 

Because we see innovation through the filter of the companies that survived, it’s possible to underestimate how traumatic it is to the people who have to innovate. While the researcher – and the sponsor of the research – may focus on the creative part of Schumpeter’s creative destruction, the employees are thinking about what innovation will destroy. 

Both present problems: the first is a Pollyanna-ish acceptance that all innovation is good – “sometimes, in our experience, the best idea is to do nothing”, Dalglish says. On the other hand, if doing something is the better option, the researcher needs to help bring the organisation along with them. 

“Often the client doesn’t want to change,” Dalglish warns. “The external factors are forcing change. That’s not a job for a researcher, but it is for a research-led consultancy, which is how we like to think of our work.” 

At FreshMinds, Thompson likes to use a reverse brainstorm to identify and solve problems with innovative ideas. Pioneered by the psychologist Gary Klein, the theory is that the group is free to imagine and express doubts and problems, rather than succumb to group-think. 

Klein calls it a “pre-mortem”, and describes it like this –“before a project starts, we should say: ‘We’re looking in a crystal ball, and this project has failed; it’s a fiasco. Now, everybody, take two minutes and write down all the reasons why you think the project failed.’” 

“You’re not asking people to come up with ‘how do we grow sales of soft drinks?’,” Thompson says. “We ask them a question such as ‘how would you destroy the sales of this particular type of soft drink?’ People are great at coming up with all these negative things, but then – at the end – you flip the answers. You flip them to the positives and then you end up with really bold solutions.”

It helps, Thompson says, to identify the internal people who are able to inspire. “If you can tell a story, you’ll have a bold ambition and paint ideas visually – and take people with you and bring it to life for them; then that’s when they start to buy in.”

Not everyone in an organisation can inspire radical or disruptive change, and not every researcher can play a part in making it happen. Research is definitely a component, but one that has to adapt to a process that is not ‘business as usual’, and work with the client through a stressful and risky moment in its existence. 

It might be the most important project a researcher ever undertakes, because Schumpeter’s creative destruction is real, even for the largest companies. 

Sixty years after the Fortune 500 was created, in 1955, 88% of the original companies on the list no longer existed. “Chaos is not a pit,” says Petyr Baelish in that other popular guide to modern management techniques, Game of Thrones. “Chaos is a ladder.” Research, done well, is still an essential tool for the climb. 

What is disruptive innovation?

In 1942, economist Joseph Schumpeter – inspired by the work of Karl Marx – declared that ‘creative destruction’ was the essential fact about capitalism. The idea that innovation “incessantly revolutionises the economic structure from within, incessantly destroying the old one, incessantly creating a new one” has been inspiring entrepreneurs ever since. But who drives this process of destruction? 

Clay Christensen’s great insight in 1995 was that success often made firms more vulnerable to the creative disruption of upstart rivals. “Leading companies succumb to one of the most popular, and valuable, management dogmas. They stay close to their customers,” wrote Joseph Bower and Christensen in Disruptive technologies: Catching the wave. Staying close meant asking customers what they thought of innovations. “But what happens when customers reject a new technology, product concept, or way of doing business because it does not address their needs as effectively as a company’s current approach?” The answer is that the innovation is ignored by a dominant business. 

But, Bower and Christensen argued, the ‘next big thing’ would often be thrown out by this process. The reason: these innovations will have a different set of performance attributes that only a few customers value highly, and under-perform in those attributes that most customers say they want (think about the portability, but the tinny sound, of early transistor radios). When the under-performing attributes improve quickly, the new – often lower-cost – technology invades established markets. At this point, a larger set of customers realise the value of the idea, and research reveals that this is where dominant companies should be innovating. Also at this point, market leaders are racing to catch up, and may ultimately struggle to copy an innovation that cannibalises their business. 

In his influential follow-up article, Meeting the challenge of disruptive change – published in the Harvard Business Review in 2000 – Christensen, with Michael Overdorf, argued that a “company’s disabilities become more sharply defined even as its core capabilities grow”. This, they claimed, is because a successful business has established processes, implicit values and a culture, all of which are valued by customers. These are designed not to change, so companies are slow to overturn them – with the unintended consequence that creative destruction may be impossible. Christensen and Overdorf wrote. “One of the bittersweet results of success is that, as companies become large, they lose the ability to enter small, emerging markets.”

Agile research

‘Agile’ has become one of those buzzword attributes that it’s hard to argue against, with many research providers claiming to offer agile methodologies. It has its origins in the agile movement in software development, created in the 1970s.  

Agile software development does not just mean ‘fast’ or ‘flexible’. Development uses ‘scrums’ – small, cross-functional, self-managed teams, that work together in a series of two-week innovation ‘sprints’. At the end of each sprint, they use empirical feedback as a basis to negotiate the goals of the next sprint with the project owner. There are daily checks, and one person (the ‘ScrumMaster’) has to ensure that the natural conservatism of the organisation isn’t holding back the project. 

Denver-based GutCheck is an online research agency, operating in 26 countries, that focuses on agile research, often to test early-stage concepts or copy. Its clients include Google and Logitech. We asked GutCheck about its approach:

What is agile research?

Its principles and ideas are not new; it’s just that, previously, researchers found it difficult to gain faster insights without making trade-offs to do it. Now that agile research has gained traction in the past three years, there is a structure and framework in place so companies can build processes around it.

Compared with more traditional research methods, what are the advantages of adding agile research to the toolkit?

Agile allows for more dynamic learning; you can refine questions and answers and ask follow-up questions in real time, so there is more flexibility for trying new things. There’s a narrower project scope, and it encourages more regular communication among all the teams involved. Agile research also produces shorter, sharper reports – like executive summaries.

What value would agile research add to a company?

Last autumn, our research team did a study of in-house researchers from various industries, company sizes, and levels of experience. They all liked the flexibility, the ability to respond to change, the accelerated time to market, and iteration. Agile also includes the voice of the customer in everything the team does.

What problems does agile research help solve?

It’s important to understand that agile research is not always the best solution. Projects focused on product or concept testing are the most popular fit, especially when speed is the main driver. Our clients use it for exploratory, innovation research, quick answers and disaster checks – even deciding on a name.  

O2 and home automation

When O2 decided it wanted to enter the connected home market, it knew it would be launching into an immature segment that few British consumers understood, with complex standards and technology problems, and in which it had no direct brand profile. With existing domestic tech manufacturers struggling to create viable connected home products, it was also the perfect place to launch a disruptive innovation. But which one? 

Working with FreshMinds, O2 initially went to the ‘edge’, asking technologists, bloggers and product designers where they saw room for innovation. It then built two online communities of early adopters and target customers; each either had to have a propensity to spend on the connected home, or own products already. 

The groups were given self-ethnographic tasks to show how their homes functioned. They were then invited to a pop-up home to experience connected technology first-hand. Instead of imagining an idealistic sci-fi home, however, O2 took over a London house. “It was how people naturally live,” says Harriet Boyd, consultant at FreshMinds, who managed the project. 

Concepts that were inspired by the first stage of research were tested in a larger quant study, so respondents who were not connected-home users could understand how the technology related to their lives. “It gave us confidence about progressing this,” says Charlie Oliver, O2’s research and innovation manager. 

In 2015, O2 went back to the pop-up home to test early-stage propositions. “I was able to see how respondents were reacting to the technology in life,” Oliver explains. “They could see that opening a front door would turn on a lamp… that kind of thing.” 

Launched in June 2016, O2 Home has a full set of connected home platforms informed by the research. So what did it learn about disruptive innovation? First, that the average customer is the wrong place to start. Early adopters adopt because they are the first to see a practical use for innovations. “We shouldn’t be trying to create use cases around specific propositions; we should create platforms and allow people to customise their homes,” Oliver says. 

Second, you can use the research process to get internal stakeholders to understand radical innovation. Oliver joined the research team after a number of roles at O2 and is aware that any new project is competing for finite resources and attention. 

Finally, customers can imagine the future for you, if you work with the right ones in the right way. “We live in the real world, so we need to test in the real world,” Boyd says. “We weren’t asking people to fill in a survey; we were allowing them to test these products and then to say, ‘I can see that working in my home’.”
 

Keeping up standards

You may be familiar with ISO 20252: Market, opinion and social research – Vocabulary and service requirements, but not so familiar with the body that creates and oversees standards in the UK, the British Standards Institution (BSI).  

Founded in 1901, it was awarded a Royal Charter in 1929 and, in 1942, the UK government officially recognised BSI as the UK’s national standard body (NSB). The group has operations in 150 countries, revenues of more than £300m, and issues 35,000 active standards covering everything from anti-bribery to social responsibility. 

In 2015, BSI faced a challenge: to deliver standards that could be integrated into a firm’s modern, digital workflow. “BSI wanted its standards to become truly embedded in the day-to-day operation of businesses for years to come,” says Graeme Cade, client director at Circle Research, which was asked to work with the BSI on the scheme. “This wasn’t just a digital transformation project – it has the potential to change the nature of how standards are seen and used by hundreds of thousands of organisations around the world.” 

But BSI’s customer base is every business, which makes knowing where to start far from easy. BSI needed to develop a robust, commercially sound and universally supported evidence base for one of the most radical changes in its 115-year history. 

Circle Research assigned a member of its staff to the working group that BSI established. The group’s first task was to make a business case for the change – creating criteria to identify the sectors most suitable for the new offer. It also had to pick out the key people inside BSI who could best inform and drive forward this change.  

The initial stage combined desk research, the mining of internal data, and interviews with 35 senior BSI employees. BSI and Circle Research decided the first implementation would be the standards for automotive manufacturing, an industry with a highly developed ecosystem of suppliers, good integration and automation, and a culture of best practice and compliance. 

External interviews with customers were completed to decide what the innovation would look like. The brief was to keep any solution ahead of the game, which meant being open to innovation. “BSI wanted to challenge the status quo, rather than simply iterate existing ways of doing things,” Cade says. “So the team pitched the discussion in terms of ‘best practice’ rather than ‘standards’. We also focused on techniques to help respondents think laterally.” 

Today, BSI has a dedicated team developing a new product concept for how to deliver automotive standards. The innovation project is changing the way BSI operates, but the research that drove it has also changed how BSI thinks of insight as a tool for radical innovation.

“This project has generated insights that extend our experience and understanding of our key markets, giving us the confidence to make bold decisions,” says Christina Jackson, lead new product development manager, BSI. “It has been a catalyst for change; it has enabled us to forge ahead with a clear action plan, and rallied the entire team around recommendations and reflections that we have taken ownership of.”

Catalyst for Change by Andrew Dalglish, director, Circle Research

Research and business transformation should go hand in hand. Research can provide an early-warning system, flagging shifting needs and landscapes so that organisations can adapt proactively. It can provide the foundations of innovation, revealing the fundamental motivations and desires to design against – and it can make a powerful case for change, using evidence to create a compelling argument for doing things differently.

That’s the theory – but, in reality, research often fails to live up to its potential for three core reasons. 

Sometimes it’s myopic. Often, research is conducted when a specific issue needs to be explored, a set problem needs to be solved, or a particular decision needs to be made. That’s a critical role, but the side effect is that the insight stream can become blinkered. With the agenda always being set by the organisation, there’s a danger that it may miss critical developments not on the radar. 

So, to pick up on new ideas, emerging trends and nascent issues, research that is designed to ‘ask’ needs to be complemented with research designed to ‘listen’ – an open forum where customers define the agenda, and feedback bubbles up spontaneously.

Sometimes it misses the point. A common criticism of research is that it is poorly communicated and, as a result, is either ignored or ends up paralysing the decision-making process. So rather than overwhelm the audience with information and unnecessary minutiae, there should be a laser-sharp focus on the critical insights. 

Rather than dump information and leave the audience to make sense of it, a clear story should be told that teases out the implications of the research and identifies what needs to be done. Also, rather than talking ‘research-ese’, it should use the language of business, providing commercially astute recommendations and a solid business case for action.

The biggest failing though is to forget about the human dimension. As researchers, there’s a tendency to assume that if a logical case for change is made, it will happen. But that’s not always the case because, for change to happen, people need to make it happen. 

So to become a catalyst for change, research must not be kept in a silo, but instead needs to be shared with those who have the remit, skills and influence to drive change. To rally people around the cause, key stakeholders who can help or block change should be engaged from the outset, so that they buy into the research and it equips them to act.