FEATURE16 November 2011

Are online panels finished?

Features

Are social networks the only sample source researchers need? Pulse Group’s Bob Chua and uSamp’s Gregg Lavin square up.

Res_4006405_Boxing_glove

YES
Bob Chua
Chief executive
Pulse Group

Online panels have had a significant impact on the market research industry over the past decade. That said, social media is changing the game. Sites like Facebook and Linkedin have seen massive adoption and their stickiness – the amount of time consumers spend on them – will continue to grow.

These sites, therefore, are prime hunting ground for marketers and market researchers looking to gather consumer opinion. You fish where the fish are.

The sheer size of Facebook and Linkedin’s user bases provide massive reach and make sample recruitment both feasible and cost-efficient. You can target users based on their profile data and their behavioural data. You can reach respondents on multiple devices, like mobile phones and tablets, through a single platform. Clearly, social networks are the holy grail the industry has been waiting for.

Think about how online panels are built today: how the majority of recruitment is performed via traditional banner advertising, newsletter marketing and database acquisition. Multi-panel sourcing for surveys is also increasingly commonplace.

“The size of Facebook and Linkedin’s user bases provide massive reach. These sites are prime hunting ground for marketers and market researchers looking to gather consumer opinion”

With Facebook we are recruiting sample from a single source, which allows us to have much more control over who we invite to our surveys. Better still, we are engaging with people in an environment where they like spending time and have a high emotional attachment.

We use apps on both Facebook and Linkedin to find the right people, however the respondent still has to opt in and go through the necessary screening to participate and complete a survey. The profile data we use to target survey invites could be extremely rich or extremely vague depending on the type of user we’re seeking to reach. A light Facebook user, say, simply provides basic profile information, while power users provide detailed profile information with various forms of demographic and psychographic details. But profiling could also take account of the user’s experiences, conversations, and paths to purchase – a very valuable source of information for marketers. Tools at our disposal allow for sentiment analysis and fan page profiling.

Some might question the representativeness of the social media audience as a source of survey sample, but representativeness is becoming a moot point. The recent Esomar Congress heard arguments from some major buyers of research in favour of panels that are fit for purpose rather than truly representative.

And what is more fit for purpose as a source of sample than Facebook or Linkedin? Their combined membership base is approaching a billion consumers. People spend significant chunks of the waking day on them. Their smartphone apps are massively popular, providing a means to reach people with survey invites when they are away from their PCs. All this adds up to one inescapable conclusion: the days are numbered for online panels.

NO
Gregg Lavin
President
uSamp

Social networks are a very compelling media, and definitely have their place in qualitative and quantitative research. However, when it comes to understanding detailed information and demographics of users, today’s sample providers are in a league of their own.

Where Facebook is limited to a few key demographics collected at registration, panel providers in many cases obtain hundreds of data points on a user. If an agency were looking to survey 500 iPad owners that use the iPad in a certain way who also have a specific occupation and income, I would argue that sampling from Facebook could not deliver. To screen through thousands of people would be highly inefficient, not to mention a really lousy user experience. But because of the rich, progressive profiling capabilities of panel providers, the hard-to-reach audience remains very accessible. And this capability is critical as marketers increasingly seek insights from niche audiences.

Online panels are a broadly sourced collection of engaged survey-takers, who are motivated and willing to participate in market research studies. Panellists are vetted, profiled and screened for attitudes and behaviours. Contrast this to Facebook data points, which are limited to very basic information either collected at sign-up or through real-time screening. Given this disparity, the breadth of practical research applications delivered on social networks narrows quickly to high-incidence, general population sampling. While solid for some things, social media sampling is simply incapable of more targeted insights.

“The breadth of practical research applications delivered on social networks narrows quickly to high-incidence, gen pop sampling. social media is incapable of more targeted insights”

Another thing to note is that online panel companies are not restricted by the size of their panels. They can utilise highly profiled panels when targeting is needed, while also having real-time river sampling to reach the broadest audience. River sampling allows recruitment across the web for in-the-moment surveys.

Upon agreement to participate, users are screened and electronically validated using a host of technologies and tools. Routing technologies then direct these respondents to appropriate surveys.

River sampling has been around for over a decade. It provides an advantage in engaging fresh survey takers who may not be willing to join a panel but are interested in sharing their views nonetheless. Frequently real-time river sampling draws respondents from social networks via advertising or apps, which helps broaden the sampling frame – creating highly representative online samples. At times, panel and river are used together to deliver high-quality sample that leverages the strengths of both. So this is the best of both worlds.

Panels provide a great deal of value, and are a requirement for some research. Social networks are an excellent place in which to engage potential respondents, but they are not a substitute or a panacea for problems facing panels. Smart sample providers simply need to pull the right tool from the toolbox.

  • Who delivered the knock-out blow? Add your thoughts in the comments thread below.

14 Comments

13 years ago

Gregg Lavin won with a huge upper cut. Predictable and reliable sampling through social media is still a ways off. Online panels provide the most predictable and reliable results based on the purpose-built nature of these assets v. social media's come one, come all approach.

Like Report

13 years ago

The Yes argument here is misguided. Any marketer will tell you that on the surface Social network sites seem to be the "holy grail" to everything from increased sales to survey taking. But in practice, Social Network users are on those sites for a specific purpose. In the case of Facebook, it is to brag about their lives and to keep up with the positive things going on in the lives of others they know. In the case of LinkedIn, it is to keep up with happenings within an expanded professional network. Therefore, to solicit these users for traditional online survey participation is a far stretch since the activity strays so far from their purpose of being on the site. The best place Social media sites have in research is data mining that may result in some emotional trending analysis. Furthermore, with the focus in the MR industry on ensuring the individual participating in research is who they say they are is further complicated with use of Social media for research. Issues on not being able to identify unique accounts or in other words duplicate accounts from the same individual also compromises quality. The argument for No is spot on, for custom detailed research the world of online panels will continue to exist. That said, traditional methods of panel development are in dire needs of an overhaul. The fact is users are using social sites and mobile apps more and more, so the exposure via traditional banners, Google ads, and affiliate marketers is the lowest it has ever been in the modern web era. Overall, a nice article with great points on both sides!

Like Report

13 years ago

Seems to me that the value of panels will always be there when you're researching something that's truly new - new products, new product features, new brands and brand names, or entirely new markets. But when it comes to established brands, goods and services, the vector is away from consumer engagement altogether. Brands with access to the Twitter or Facebook fire hose and the right algorithm can answer damn near any qualitative question without having to ask a single human being. Companies pioneering this kind of technology and service are going to explode. Or already have.

Like Report

13 years ago

Good article, and very much in keeping with the themes of the recent ESOMAR conference, too. But as pointed out above, in reality, with the sheer volume of quantitative research being done, panels are still necessary: it's like the difference between farming cows for meat and trying to catch them in the wild. The latter is fine up to a certain level, but once you reach a certain tipping point of meat (surveys!) required, farming is the only viable solution. Having said that, even this analogy raises again the point of trained panellists - and how representative they and their views are of the wider population.

Like Report

13 years ago

Social networks aren't the only source we need, but it doesn't mean they're not valuable. A decade or so ago, we were having similar debates about whether migrating from pen & paper to online research would skew our samples. We got around initial biases towards younger, more affluent consumers by using quota sampling and, over time, we are seeing demographic disparities between online users and non-users diminish. We now need to bear in mind the limitations of recruiting via social media sites, while embracing the fact that they will allow us to reach larger (more robust) samples more easily, the incremental cost of each interview being very low. They provide us with one more way to conduct quicker, cheaper research in times when panel response rates are dropping.

Like Report

12 years ago

Good points by both sides, (glad to see panels won the poll) and I agree with most of these comments, as well. I believe the best approach going forward will be sample primarily from panels with some amount of blending from river sources. To me, the sourcing of respondents is not the important issue. The statistical validity of the sampling method -- the way routed sample is manipulated and the way that panel sample and river sample are blended -- is the key driver for research quality. The real winners will be the providers who figure out how to best blend sample to meet the quality demands cost effectively. I just posted a blog with more on this topic at www.lightspeedresearchblog.com.

Like Report

12 years ago

Both ways have limits. These 2 ways are compelmentary. But what about a more deep issue we are facing for online surveys and surveys at all : decrease of return rates, decrease of quality in collected datas ? The key issue now for our job is : are people still interested in sharing their opinion by online classic surveys ? how to increase and sustain panelist involvment (incentives are not enough, incentive is quantitive solution to reach a number of completes but don't ensure the quality of datas collected) ? We need to adapt our surveys to people lifestyle : make surveys a friendly user experience (and not only by few pictures on layout), stop asking demographics questions that are known ( gender, age at least for panelists, licence driving or not?, ...) in order to avoid as much as possible panelist frustration, decrease surveys lenght !

Like Report

12 years ago

Nice topic and comments. It will be very early to say panels are vanishing. However it may loose its market size in future. As pen and paper is still alive though online is preferred, similarly the sampling methods will have their places. The number of people responding, commenting twitting is increasing day by day and are proved to be powerful media to drive people sentiments. the tool is though not aggressively used to measure people sentiments towards any product or brand. There are research agencies that are pioneering in social media research and are providing new edge to research. it is at data mining and analytic stage and can be termed as "Proactive approach". these are expressions of people without any desire or hidden agenda. On the other hand panelists join survey with some benefits like incentives or reward points and responses are "Reactive". If social media panels can still provide proactive responses after recruiting will be a benefit to companies or it will become similar to online panels one day.

Like Report

12 years ago

Social Networks has 2 limits as well, that could be solved by OnLine Panel : Each Social Network is specialized and no one could cover as much datas on respondents as Online Panel what are (age gender localization) basic demographics . Depending on kind of social network you'll manage to have some more informations (Linkedin = professional informations; FB : very rarely you could find marital status, relationship with family members but this is really poorly filled in by users). Other limit that is also a great opportunity : representativeness of these users : on the one hand, you have likers ( intersting if the subject of the survey is to focusing on brand fans) on the other hand, this likers are not representative of the markets ( non-purchasers, competitors products purchasers /consumers, non purchasers/ consumers of this kind of product, ...) To conclude : The question " Are online panels finished ?" is really is too reductive, the 2 ways have weaknesses but they cover this with strenghtes of the other. This is complementarity. Social networks are a leverage for Online research.

Like Report

12 years ago

I could be wrong but I thought LinkedIn abandoned their full survey business model. If this is still the case then it makes the YES argument less plausible.

Like Report

Results 1 to 10 of 14