FEATURE23 August 2011

Time to rewrite the rules of research?

Features News

In their efforts to protect research from regulatory straitjackets and public opprobrium, are research associations leading the industry up a dead end?

Res_4005880_laptop_magnifying_glass_search_iStock_000010540776XSmall

In their efforts to protect research from regulatory straitjackets and public opprobrium, are research associations leading the industry up a dead end?

That was the question debated yesterday by representatives of research associations and companies engaged in social media research, in a webinar hosted by the GreenBook blog and chaired by Andrew Jeavons.

It all relates to how researchers deal with information they find in blogs, communities, Twitter, Facebook and so on. Must we still respect the anonymity of those involved? Do we need their consent?

New guidelines from Esomar, draft guidelines from Casro and a discussion paper from the MRS seem to suggest we do. The associations have sought to apply to social media the principles of anonymity, informed consent and the separation of research from sales and marketing. Their efforts have met with an impassioned response.

Michalis Michael of DigitalMR has said Esomar and the MRS are behind the times, Ray Poynter has urged research to “join the 21st century”, while Annie Pettit of Conversition imagined a eulogy for a defunct trade organisation that failed to adapt to a new world. Michael and Poynter were among those discussing the matter with representatives of the trade bodies yesterday.

“In an attempt to stay ever purer, organisations are coming up with a narrower and narrower definition of market research”

Ray Poynter

Barry Ryan of the MRS argued that, rather than lagging behind, researchers were way ahead of the times in realising so long ago the potential for data to be misused, and the importance of protecting it. The codes and guidelines which have been developed over the years should still apply to social media research because “in nearly all respects they are simply restatements of the applicable law”, he said.

Adam Phillips of Esomar said research is built on people’s trust, and that the new guidelines were “a good starting point” for maintaining this trust in the social media age.

But Ray Poynter of Vision Critical said that “in an attempt to stay ever purer, organisations such as the MRS, in particular, are coming up with a narrower and narrower definition of market research”. If we’re not careful, most of what researchers do will soon be classed as ‘not research’, he said.

A frustrated Michael of DigitalMR said that if these guidelines take hold, “we’ll probably go out of business in favour of software development companies that do not abide by our ethical codes”. He pointed to the persistent use of the word ‘respondent’ which has little meaning in the context of social media monitoring.

Tom Anderson of Anderson Analytics was even more dismissive, saying the guidelines represented “a naive attempt by some trade organisations to remain relevant”, which was more likely to hinder companies than help them. Such guidelines are not needed and most practitioners won’t pay them much attention, Anderson said, because companies in this area are “already looking away from the traditional market research space and competing with a totally new set of companies”. Survey researchers know “next to nothing” about social media research, he said, and should not be looked to for guidance.

“A solution you devise yourself is going to be far superior to a solution somebody imposes upon you”

Peter Milla

Applying the principles of anonymity and informed consent to social media is, at best, awkward. Esomar says that if consent can’t be obtained to use someone’s online comments – either explicitly or through a site’s terms of use – then researchers must conceal their names and “mask” their comments to prevent them being found through a search engine. But this attempt to respect established rules goes against the conventions of social media, which hold that you should credit your sources. Naming (and most likely linking) your source is seen as an honour, not an imposition, and failing to do so is a shortcut to being shunned. In any case, the MRS’s paper calls masking “rather unsatisfactory” from a methodological point of view.

Michael argued that we should set out from the assumption that somebody who posts online “wants his opinion to be heard” and knows what the implications are. If they don’t, “tough luck”.

But the trade associations say they are simply upholding long-established principles and interpreting the relevant laws in order to anticipate any legal problems. In the UK, for example, the Information Commissioner’s Office has the power to issue fines of up to half a million pounds for data breaches.

Peter Milla of Casro’s social media task force spoke in favour of self-regulation, saying “a solution you devise yourself is going to be far superior to a solution somebody imposes upon you”. Poynter disagreed, saying that self-regulation had “had its day”. He highlighted the exemption of market research from do-not-call lists as an example of rules drawn up by “the chattering classes” rather than by the people whom the rules are supposedly there to protect. The public, Poynter said, “hate the fact” that researchers aren’t subject to the rules.

In two quick polls at the end of the debate, a majority of listeners agreed that research codes should be applied to social media research – but there was uncertainty about whether current proposals are “heading in the right direction”, with 41% saying ‘don’t know’ and only 34% saying yes.

What seems rather lacking from the debate at the moment is data on how internet users themselves actually feel about these issues. Last year Vovici conducted a study suggesting that stated views on online tracking have little influence on people’s behaviour online, while ad targeting firm Specific Media has claimed (perhaps not surprisingly) that its research shows consumers are less concerned about privacy than industry bodies. Tom Anderson said yesterday that research by his firm suggested consumers understood that comments posted online could be read and used by companies, and weren’t troubled by it.

If the research industry wants to keep people’s trust, then it would be good to see more research conducted into what that means in the online world and what steps can be taken to achieve it.

6 Comments

13 years ago

We did a bit of research about consumer opinions towards being contacted via social media. It's a slightly different agenda item but it's still relevant. Just because people write things online does not mean they are giving us free license to do whatever we want. This is why I'm so adamant that we need to respect privacy by masking verbatims. Unless you know that this specific individual wants to quoted verbatim, then you shouldn't do it. 26% of people say brands should never contact them http://www.conversition.com/social-media-research-privacy-survey/ 59% of people didn't like being contacted by a brand http://www.conversition.com/comment-survey-social-media-research-engagement-privacy/

Like Report

13 years ago

Great summary Robert, thank you for a masterful job in cutting to the heart of the matter. I agree wholeheartedly that perhaps we need to stop navel gazing and undertake an effort to truly understand how consumers feel about these issues as well as research professionals; perhaps we can develop a coalition to tackle that? For anyone who missed the debate you can access the recording here: http://www.greenbookblog.org/wp-content/uploads/2011/08/Privacy_Debate.mp3

Like Report

13 years ago

Robert – thank you too for summarising the debate and thanks also to Lenny etc for hosting/organising it. As someone who was closely involved in helping formulate the ESOMAR guidelines, I have some comments to add which may be useful. Firstly over the lack of data on what users think, there are now a number of studies that have been published. Probably the most quoted have been conducted by SSI this year which very much supports the view that many people are not at all happy about their quotes being used for other purposes and for someone else’s profit. For example from their US data, about 60% of social media users disagreed that ‘companies should be allowed to collect information from .. public forums’. Their data from other countries showed a similar result. There is marked age difference as you’d expect, with only the 18-24’s showing less than 50% disagreeing. Given this, the arguments of Michael’s that we should assume that somebody who posts online wants his opinion heard and knows the implications are just wrong. I’m sure the woman recently sacked because people published her private tweets would also agree. Having said that, as one of the first practitioners of social media research in the UK, I have a lot of sympathy for some of Michael’s and Tom’s views. Indeed that is why I was happy to be part of the ESOMAR guideline group. I think the guideline that our group created is probably the best that could be currently achieved given the current state of legislation. It does a pretty good job in balancing the need of ESOMAR members to uphold their professional standards whilst at the same time not stopping them conducting what I think is an ever more important sector of MR ie social media research. I personally however think the MRS standards board position has gone too far to the other extreme. For those criticising the standards, please read both. Their stance is quite different. For example, the ESOMAR guidelines talk about ‘users’. The MRS discussion paper still talks about ‘respondents’. The ESOMAR guidelines talk about practical ways to conduct social media research. The MRS paper comes across like a legal case why the industry should never conduct social media research. Like many said in the debate, my view is that the MRS’s stance is untenable given the way the world is heading. We have to work out a way to work to utilise social media research or the industry may face the prospect of disintermediation in the near future.

Like Report

13 years ago

It makes little sense to keep discussing whether or not people are fine with market researchers using (what many people perceive to be their) web content without their explicit consent. Some are - some are not. However interesting that distribution might be, it’s fairly irrelevant. It’s not a majority vote issue. The core question is whether we should respect people’s expectations of privacy or merely abide by the relevant laws (data protection acts, terms of services etc.). While I’m all for guidelines and a healthy set of ethics, I share the view that we must be careful not to cripple ourselves needlessly in a growing and increasingly competitive "new" market.

Like Report

13 years ago

Thank you Robert for a good summary. I agree wholeheartedly with Pete and Annies comments. For what it is worth we have just completed some research with under 18s; not specifically about social media or social media research, but only 15% were actively happy that organisations would access and read their posts about products or services they had consumed; massively lower than than the equivalent figure for friends or family. This means that consumers themselves do not readily accept that social media posts are there for all to use as they wish. We need to be careful here as eventually the law and data protection will eventually catch up and swing in line with public opinion. As early front runners in this field, we must not be caught on the wrong side of the debate, or law, by not listening to consumer views. We must do what we are supposed to be good at; understanding consumer views through research. And that is telling us to respect people's data.

Like Report

13 years ago

I couldn’t agree more with Brain Tarran, a level of agreement that was encouraged by the views of Michalis Michael and Ray Pointer. We are seeing the terrible effect of dropping standards when we went along with convincing some of the more gullible clients that access panels were a good source of cheap interviewing. The con was made easier in the UK by some access panel companies correctly forecasting a 2 horse race in the Mayoral elections. What Michael and Pointer want to do is continue this decay even to the point of blurring the distinctions between research and selling. How often do we find new media exponents moan that traditional (i.e professional and reliable) researchers don’t understand their new age art form and that the MRS is not qualified to attempt to control their amateur behaviour. We are being seduced by the idea that we could get rid of proper sampling and interviewers thereby reducing the cost of research and making research more available to organisations that couldn’t afford to do surveys properly. What is actually happening is that the survey is constantly being devalued as a profession to that point that companies like Toluna, are arguing that you don’t need researcher at all, just buy the software, access the 1% of people that complete surveys for 75p and the PR/marketing department can do the research themselves. Once we convinced everyone that the concept of that devil incarnate, the random sample, could be killed off, we can just interview anyone irrespective of how they are selected, what research they have done before and their motives for doing it at all. A lot of the social network data resource can be quite useful and the internet is probably the best way to do interviews (properly sampled). However, we are ruining these new methodologies by ignoring the fundamentals of what makes a properly designed survey. The reality is we need all media, CAPI, CATI, CAWI, Postal, Facebook Google etc etc. We use them all and each has its place. So come on MRS, persuade new age or nothing mob to go off and set up some hybrid cross between marketing and rubbish research in the hope that we can reduce what is left of what I considered to be a profession before they ruin it completely. Over the past 30 years we have seen endless new dawns that would render conventional methodologies obsolete. I remember John Goodyear, about 400 years ago trying to persuade people that they don’t need to quantify surveys at all because some good qualitative research will do the job just as well. The trouble is, he was joking unlike the latest batch of new age drivellers who actually believe that their new ways of collecting data are the most accurate and want to assign to assign traditional research to the dustbin.

Like Report