Social sampling: a reliable alternative?

Is there a faster or cheaper way to get a good quality representative sample? With so many potential respondents already engaged on social platforms, from social media to gaming sites, would it be viable for market researchers to take samples from these sources and these sources alone, instead of using online panels?

At Lab42 we wanted to find out whether getting sample from these alternative sources makes a smart choice for online research. We wanted to see not only if social media sources were faster, but most importantly, whether the data collected via these sources matches traditional panels when it comes to behavioural and attitudinal representativeness.

For the purpose of the study, we generated samples via our proprietary social media methodology (where we obtain respondents for online surveys via social networks, games and applications) and from a traditional panel provider, both representative of the US Census based on gender, age, region, household income levels and ethnicity. We also set up quotas for each category in order to ensure populations were identical to each other, and the quota groups were held constant for both sample sources.

To compare the samples, the questionnaire focused on determining the attitudes and behaviours of respondents with specific questions related to media consumption, motivations behind completing online surveys, purchasing and shopping habits, and personal beliefs.

How do speed, data quality, attitudes and behaviours compare?

Right from the start, we saw that social media sampling brings a benefit in the speed of filling quotas for certain demographics. By using social media, e-commerce and gaming sites to fill the sample, we were able to get younger (under 35 ) respondents more quickly. The rate of filling the 35 – 55 year old age groups was similar with both methodologies. In addition, the data integrity measures were stronger when using respondents acquired through social media.

In the majority of cases the answers from the two samples were virtually identical, and in almost all cases, the order rankings were identical, indicating the two samples are attitudinally and behaviourally comparable. Both groups’ opinions about survey taking were similar and they were both as motivated to take part and to receive the same types of incentives.

Interestingly, we also found that social media and technology use by these two samples was identical. About half of the respondents from both sample sources considered themselves to be early adopters of technology. Purchasing patterns of electronics and groceries as well as media consumption were also similar. With electronics, this includes where items were purchased as well as which items were purchased. With grocery shopping, the top five staple items bought by the two groups were the same.

The survey also found that the differences in attitudes across a wide variety of topics ranging from volunteering, donations, and organic food consumption to travel and political affiliation were negligible.

So, social media sample is as good, if not better?

As expected there were some differences in the outputs of the two samples. Social media sourced panels are less frequent survey takers. More than half of the traditional panel respondents are likely to report being on panels vs. 26% of the social media respondents. Traditional panel respondents, are more likely to be on multiple panels and to have taken multiple surveys in the past month. Social media sample has a better reach among younger generations (millennials, generation Z) and older respondents ( 55+) and can fill surveys faster than traditional sample providers.

These differences can provide significant value depending on the subject of the survey. For instance, we found that this type of sample sourcing provides significant additional value for brands in consumer electronics, apparel and gaming. Over half of the social media sample shopped for electronics and 72% for clothing/apparel during the past month compared, with 39% of the traditional panellists shopping for electronics and 60% shopping for clothing/apparel. Looking at grocery shopping, the traditional panellists shop more from grocery chains than from mass merchandisers — 43% and 29% respectively — while the social media sample shop from these channels more equally.

We could conclude that this new sampling approach would support key business decisions just as solidly as other types of online research. Despite the differences in respondent recruitment, a social media sample can be used in place of more traditional online-based panels without sacrificing research accuracy or validity.

Jonathan Pirc is co-founder and VP of Product at Lab42