OPINION4 March 2010

From gatherers to analysts – the new insight value chain?

For the research and insight industry, the implications of an explosion in the amount of data available are profound. In a world of scarce information, those who could somehow gather data and shed light on the truth were valued and feted. In a world of superabundant information, those who offer a teaspoonful of insight are less likely to be noticed. as the real stars will be those who can make sense of, and draw insight from, vast amounts of data quickly and reliably.

This week’s edition of The Economist focusses on what it calls ‘The Data Deluge’, a discussion of how the rapidly accelerating volume of information available about all aspects of modern life is transforming the way we do business.

We are moving from a world where information was scarce to one where it is becoming superabundant, a world where a megabyte ( 220 bytes) used to be a worryingly large amount of information to analyse but where we are now starting to worry about dealing with Yottabytes – that 280bytes!

For the research and insight industry, the implications of an explosion in the amount of data available are profound. In a world of scarce information, those who could somehow gather data and shed light on the truth were valued and feted. In a world of superabundant information, those who offer a teaspoonful of insight are less likely to be noticed. as the real stars will be those who can make sense of, and draw insight from, vast amounts of data quickly and reliably.

To remain relevant to data hungry and fast moving industries, we have to move from being an industry where value was derived from providing scarce information, to one where value is derived from connecting and interpreting the vast amounts of infomation available, to help clients make better business decisions faster.

Martin Hayward

@RESEARCH LIVE

3 Comments

Dan Rockwell

14 years ago

I agree. There is a ton of data out there and while its a great time for miners it should be an even greater time for researchers to really comb through all that data and help piece together the patterns and help communicate the value and actionable result back to the stakeholders. But data can be overwhelming, most of this data, just in sense of data from digital word of mouth via social networks, by far the most interesting stream of data out there is unstructured. We can attempt to dig for context and break out our big book of assumptions on what it all means but it should primarily be used as discovery data to kick off a real thought out research initiative. There's also plenty of examples of success out there in the word where gut and instinct made for a good product or service, in some ways I wonder if the data deluge will just overwhelm us with potential futures and not inform us. Researchers are aware of analysis paralysis but joe public here is getting in on the mass data collection game- they don't know the process really and its a dangerous reactionary data firestorm they could be walking into.

Like Report

JamesG

14 years ago

Wow. This gives me pause considering how research is becoming commoditized (yep, US) enough as it is. I am concerned that the “superabundance” of data creates so many dots for the connecting that more and more organizations will craft new constellations by seeing what they want to see. However, it is not as if we could or would want to staunch the firehose flow of the incoming data. Interesting times ahead, to be sure.

Like Report

Tiina Raikko

14 years ago

It's also time for companies to stand back and refocus on what they really need. There is A LOT of fabulous information out there and data with a lot of potential but who is going to do all the work on it? There is not the time or money to do it internally in many cases or even to outsource. Time to think long and hard about what really matters in terms of key data you need to remain focussed as a busness and deliver on your KPI's. What is really interesting and has potential but what really helps you drive your business forward? It's terrible to see the potential slide by but economic reality is what it is. The answer will often be investing in be less but better managed and utilised data.

Like Report