This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Find out more here

FEATURE1 September 2007

In search of lost insight


New research isn't always the best way to new insight. Robert Bain goes on the hunt for the valuable information that's already within your reach – as long as you know where to look

Fictional treasure hunters have it easy. Once they've located it, the booty is hard to miss: it's the wooden chest overflowing with gleaming gold and jewels. For real-life archaeologists, that's only half the hunt: you still have to consider context and connections before you'll know if what you've uncovered is a gem or just more junk.

In an age where companies have more data than they know what to do with, finding the nuggets of precious insight buried among the junk is a daunting task – even when you know they're in there. So how do they end up getting lost in the first place?

The first reason is that the people carrying out research never expect anyone to go looking for it. Research is usually commissioned for a particular purpose, and once that purpose is served, it's carefully stored and forgotten about while everyone gets on with their jobs. Teams are so busy ticking boxes and fulfilling KPIs that they don't get a chance to tease any broader insights out of what they've done.

The second reason is simply that no one knows the data is there. Why would you, if the work was done in a different department, or before you joined the firm, or is stored in a format you can't use, or – most likely of all – you just don't have time to look.

For a frazzled and under-resourced clientside researcher, it's often much more tempting to commission new research than to embark on a time-consuming and potentially fruitless trawl through existing data. "The default is to get a fix now rather than to delve into history," says Leslie Sopp, head of research at Age Concern. Freshminds Research MD Jeannie Arthur, an ex-management consultant, says: "There are definitely instances where clients work with suppliers, everybody gets very excited and before they know it they're off commissioning something before they've looked at what they've got already."

And if people can't face grappling with the data in their own department, they're going to get even less far with information in other parts of the organisation. Paul Nola, account director at KAE: Marketing Intelligence, says: "The amount of good quality insight that sits around in an organisation and doesn't deliver much value because it's not communicated, doesn't bear thinking about."

Ann Calcara spent twelve years at P&G before moving agencyside with Seek Research. She says one of the trickier reasons data gets wasted in client organisations is the prevalence of 'Not Invented Here' syndrome – a condition brought on by the unrelenting pressure for innovation. For sufferers of the syndrome, referring to research that has already been done is tantamount to an admission of failure. It's a fast-moving world, we keep telling ourselves, and nobody wants to be the one caught leafing through dusty old data.

All these matters aren't helped by the myriad formats that the data might exist in – a situation that Mike Page, a former Synovate director who now runs his own consultancy, blames on a "lack of maturity" in the MR software arena.

On the face of it, these problems shouldn't be that hard to solve – they're little more than bad habits that individuals, organisations ( and perhaps to an extent an entire industry ) have fallen into. Paul Nola of KAE recently led a review of a major telecoms client's existing research, seeking new insight about the 'customer journey'. He says: "Research managers need to think of themselves as consultants, and any consultant worth their salt won't do anything without looking back and seeing what's been done." It's hard to disagree, but then it's hard to disagree with someone telling you to stop smoking or biting your nails – it doesn't mean you'll stop.

Calcara says: "The golden nugget is that if you do take the time to trawl through the old research, your new research is going to be so much better and more effective." Again, few would say she was wrong. But how do you break the habit?

Mike Page is positioning his startup agency, Cognicient, as a 'systems integrator', helping clients to bring research and other business data together to unleash its predictive value. In a 2003 paper he compared the parallel data streams of business intelligence and market research as "two swift flowing-rivers that never meet".

Page says: "You have to look and see what aspects of the research you conduct are strategic and high value – i.e. what aspects are predictive of the outcomes in your business. Anything outside of that, you should question why you continue to collect the information." In this way, a concerted effort in the short term could reap considerable long-term benefits for the business, as well as making sure that future research budgets are better spent.

Software firm SPSS is making the most of these trends. CEO Jack Noonan says: "Everyone is trying to figure out how to integrate pieces of data from silos across their organisation to get this unified view of the customer. Integrating customer feedback with your business data is driving significant change in understanding the behaviour of customers."

But ultimately this is a cultural problem, not a technological one, and it needs to be addressed from the top down. "You can get hung up on technology," says Nola. "The most important strategy you can have is to make sure senior management understand the needs of insight to the business. It should be inherent in all the strategic marketing and consumer marketing people in the business."

Often it takes an outside perspective to make a business realise this, and more and more agencies are filling the gap. Andy Dexter's new consultancy Truth lists 'insight maximisation' as a core part of its work. "There are often lots of insights buried and data lurking around an organisation," says Dexter. "But it requires a cultural shift to say that reinspecting some of that could give you some of the answers. My observation is that many organisations on the clientside are looking at this quite specifically."

Page says: "Agencies are now being asked by their clients,'You've done all this work for us, is there more you can tell us from it?' There's a generally wider expectation of what data can do and tell you."

The pressures that cause insight to go to waste aren't going to go away – finding better ways to deal with them is a matter of will power. Calcara believes the cure for the damaging 'Not Invented Here' syndrome is to reward reapplication as much as, if not more than, innovation. "If you're always innovating from scratch, how many times are you going to come up with something that actually already exists, you just didn't know it did? Reapplication can be the route to truly breakthrough innovation."

Efficient Insight – Questions To Ask Yourself

• Before you get whipped up in the excitement of commissioning a shiny new study – stop, look and listen. What insights are there already in your organisation that you can use as a starting point?

• What are relations like between the end client, the insight team, the agency, and anyone else involved? Is everyone clear about what they're trying to learn? Are discussions staying open as the project rolls on?

• How easy is it to access data in your organisation? Can old data be manipulated in new ways?

• How integrated are the various sources of information and insight in your business? Do people in different parts of the organisation talk to each other? Does the data exist in a format that can be shared and understood across the business?

• What sort of behaviour does your organisation's culture encourage when it comes to research? Is maximising insight really a priority, or are you just paying lip service?

• When the final presentation is over, who in the organisation knows and cares about your research? Do the right people have the chance to join the discussion, and reap the benefits of what you've learned?

Research Forum Views
On rediscovering the 'new' value of 'old' research

"Research can be seen as old news too quickly. It's always easier to try and sell the new thing. We researchers need to be better and more confident about advocating what we know we know, and focusing efforts of new work on what we don't."

Chris Mundy, BBC

"So often we are told, 'Yes, we did research this a few years ago, but I wasn't here then…' For agencies to become 'conductors of knowledge' rather than simply providers requires us to get closer to clients, so that they trust us to manage all their knowledge while their staff come and go."

Darren Noyce, Skopos

"Surveys often end up scattered across different research agencies' portals, or in a disorganised heap on a shared drive. The work and cost involved in creating a living, accessible research library is beyond the reach of many insight teams' pressured resources."

Tim Macer, Meaning

"There is an inevitable bias towards 'new data'. People want to generate data to show how skilled they are or point out the new results they have found. 'Old data' is less glamorous, and it can be easier to run a new project than get the old data in a comprehensible form. Standardising how data is held and catalogued is a first step to finding buried treasure."

Andrew Jeavons, Nebu

"The right people generally can't find or use the data they need. Technical solutions get implemented to solve this for operational data, but these usually don't have the flexibility to store research data without diluting it. The good stuff gets lost, either literally or through inappropriate accessibility."

Sam Winstanley, ForgetData

"Often it's deadline pressures and a lack of early involvement for the research teams. There's no time for a relevant trawl, and research departments rarely have the resources to do speculative work. Management summaries – the accessible face of past projects – focus on key conclusions and action points, whereas it's the context for these that may be of more durable and wider relevance."

Kate Hamilton, Gravity Planning & Research

"When I started in research in the 1970s the deliverable was a written report, which often had a shelf life of several years, because the printed word was capable of being accessed. Databases ought to be more accessible. We need a Google-type product for research results."

Ray Poynter, The Future Place

"There's not enough manipulation of the databases that hold the data. Query builders, and even linking data with other software can provide surprising new insight, or a new look to old analytical queries."

George Mathioudakis, GfK NOP

Sign up for the Research Magazine Facebook Forum at

September | 2007