Are researchers gambling with the truth?

With simultaneous use of mobiles, tablets and televisions vying for our attention at home; mobile and web analytics giving us a continuous stream of big data; social media driving news headlines (and libel cases); do we know what the truth is anymore when it comes to media consumption and audience behaviour data?

The conference speakers and panellists embraced the theme and took some big topics to task, not least those relating to our role as researchers. I took out from the conference the following ‘home truths’ for all media and market researchers working in our industry today.

1. Social media metrics need a reality check

Let’s face it – social media platforms such as Twitter and Facebook are sexy for advertisers. Brands are jumping on the bandwagon to reach out to the ad-attractive users of social media.

An analysis of retweets and likes, or volume and sentiment, and it appears on face value that engagement metrics will supersede good old-fashioned face-to-face interviewing, as the recent tweet below about waiting for the new Radio 1 breakfast presenter’s audience figures on the official radio audience survey (RAJAR) highlights:

“With RAJAR u have to wait till Jan to find out how @grimmers first show went. With social media you get feedback that day”

So, is random sampling the past and ‘fandom’ sampling the future? The reality check is that analysis such as tweets and posts are simply qualitative research on a large scale, but without a moderator. Whilst this may yield some rich opinion (one MRG panellist referenced the value in searching brands alongside #fail as a health check), it is hardly representative of the population or indeed of brand users.

Only around 15% of the population regularly use social media and we need to remind ourselves of this. The data at our disposal is not necessarily the whole truth. We have to search for the truth from within the mass of data and information.

2. Big data is just data

According to Google, there were 350 million YouTube videos shared on Twitter last year.  Furthermore, 16% of Internet searches on a daily basis are new – i.e. words being typed into Google not seen before.  These are just two examples of what we call “big data” that is, data collected via web or mobile analytics and not from traditional survey research. 

The two worlds of survey and server data make uneasy bedfellows.  Whilst media owners are fed their staple diet of survey or panel-derived audience measurement data, often on a quarterly basis, this is now being served with extra helpings of behavioural digital data on a continuous stream. 

There is a growing perception that survey data is slow and out of date, whereas server data is real time and true.  Putting it simply, survey data counts people; server data counts machines. Whatever happens, we still need research to measure people.

I personally hate the term ‘big data’, but I have come to accept that the buzzword is here to stay. Essentially, it is still just data and we, as researchers, deal with data (big and small) all the time. The trick is to separate the wheat from the chaff, or the truth from the lies, in our world of ‘infobesity’. We must stop quoting big numbers for the sake of it (“number-wanging”) and start quoting some big thinking instead.

3. Stop being London-centric

We need to remind ourselves of the home truth that the UK is made up of far more geography and demography than just London. Media consumption varies by region and not everyone owns for example a tablet – actually it’s only 11% of us albeit for now. Interestingly, an argument was presented for segmenting audiences based on where they work rather than where they live, which is particularly relevant for London.

4. More haste less speed

Access to fast data, such as web analytics or social media sentiment, means the expectations have increased over the ability to make fast decisions (see tweet referenced earlier). However, in this age of big data and very quick data, I firmly believe that the timeframe of judgement on performance is artificially shortening.  Companies can get quick data to make quick decisions; but quick data does not necessarily make good data and therefore good decisions.  It really is ‘more haste less speed’.  

5. We need specialists, not generalists

The truth is that market research will be around for a while to come. We still need survey methodologists and storytellers but what is starting to change is the need for more diverse skills within the team. We need to be data scientists, data fusion experts, data quality controllers and data visualisation designers. The emphasis on data in the team is not accidental. The onus on research and insight teams across our industry is still very much to identify (and verify) the truth from data.

John Carroll is senior director in Ipsos MediaCT and chairman of the Media Research Group (follow John on Twitter @MediaCarroll)