USamp founder calls for change in sample pricing model

Great article Brian and some of Matt’s thoughts and action points are bound to resonate with other suppliers of online sample. Hoping to encourage wider and further discussion, I’ll leave my comments to Matt here rather than on uSamp’s blog. @ Matt Dusig: Thank you for an intriguing read. I fully agree that survey tool companies should enable panel providers to read quotas, as this is clearly in the interest of all parties (from end client to panelist). I also like the idea of moving from a CPI- to a CPF-based pricing model. But this is not only wishful thinking, it also seems entirely unnecessary given that quotas would be passed along? Surely survey routing technology - as has been introduced by most access panel companies by now - should then pretty much solve this entire problem? (Even if it creates one too by introducing bias… but that’s another discussion). I may well be wrong, but I was under the impression that one of the reasons that uSamp acquired DMS Insight last year was precisely because of their survey router know-how? The last two action points in your CPF argument leave me a little troubled. What’s stopping anyone from rewarding every panelist that responds to a survey invitation? Why aren’t panel companies already doing this? - and why aren’t you/they charging more for ”blind targeting” panelists (to the extent that you still do this in spite of any survey routing technology), when you know that it generates churn and thereby has a real cost on your side? While it’s always easy to point the finger at the competition, it’s anyway a little hard to find sympathy for challenges which access panel companies have largely brought onto themselves. For years, online sample providers have been competing on price rather than quality and now they/we are feeling the consequences. If the access panel companies had been more strategic about their sample products rather than ”burning” their panels for short term profit by over sampling, -surveying and by allowing survey drones to fly across their panels without a pilot - we would have been in a different place today. Sure, there’s a ”changing times”-issue, but most panel churn is self-inflicted by lack of survey quality controls and poor panel- and sample management. No?

Like Reply Report

I like Matt, the DMS folks, and USamp as a whole but I have to point out that many of the ideas expressed here have been around for years and used by e-Rewards. It was always a matter of course when using them for sample to give them access to quotas through front-end programming and/or a reporting portal, and one reason e-Rewards always did so well from a response rate perspective is because they award points for every click on a link by panelists, whether they qualify or not. They minimized the gaming risk of that policy via good targeting, and overall I always felt that they had the best panel management system as a result of these practices. Nowadays, Pureprofile is doing very similar things with great results on both engagement and quality, and I bet others are as well. As we shift towards more of a social model based on communities and mobile platforms, much of this will really become a non-issue because the systems will be integrated at so many levels that we'll be bale to pinpoint target respondents and have highly engaged relationships with them. The concerns of the panels Matt expresses will be relegated to very specific niche projects within research that require access to broader sample populations, but largely will not impact most commercial projects anymore. My advice is that any panel provider that is not aggressively building internal data collection capabilities for mobile, isn't working on community integration, and isn't focusing on social integration is probably going to be in real trouble within the next 5 years. Rather than worrying about the current paradigm, they should be focused on the next evolutionary shift, and I don't think online survey quota controls via APIs is part of that shift.

Like Reply Report

Matt raises some interesting and important points here. A significant part of this problem stems from aggressive pricing models across the industry. Many panel companies have slashed their CPI models by half in a bid to grow market share, or as I expect in some cases, simply to stay afloat. The underlying and most worrying effect of this is in fact a massive reduction in respondent incentives, which appears to be a strategic move to protect or indeed increase margin. This is a worrying trend and one that we (at Panelbase) fiercely resist for one very important reason - the respondent experience is of paramount importance and incentives should be kept at a level that is fair recompense for a respondent's time and opinions. We can still compete on price where we need to but not to the detriment of respondent incentives. I agree with Dan’s response that there is an element of self-infliction here and those feeling this particular pain will surely recognize it as the cause for single-digit response rates and massive churn. Some of the larger panel companies today pay around 40% of the incentive they used to pay two or three years ago! One notable side effect of this situation, alongside more challenging macro economic issues, is that many clients are now giving far more weight to price than quality. Seemingly it doesn't bother them quite as much as it used to if a respondent only receives the equivalent of 10p for completing a 30 minute survey, but what is the impact on data quality? I believe there also needs to be a sea change from the client side if quality is ever going to regain its status. At a recent MRS meeting set up to discuss many of the issues surrounding respondent incentives, and panel operations in general, a handful of panel company representatives put forward the suggestion that many respondents will in fact complete surveys for no reward at all, and that this is reason enough for all respondents not to be paid, or to receive far lower rewards than should ordinarily be deemed fair. I have to agree with Gregg Lavin that this is a dangerous policy for any company to adopt, unless of course the health, responsiveness and overall longevity of their panel are no longer of significant importance to them. Those with short-term cash grabbing growth strategies may well reach today's objectives but will they be around tomorrow? One thing is for sure, if their panelists aren't paid fairly, they will vote with their feet. Picking up on Leonard’s response and that the evolution of the social and mobile landscapes will render poor incentivisation a non-issue, I can see why he might think that, but perhaps more research needs to be done in to this area to fully understand the respondent’s perspective. Many of the comments I have seen from respondents suggest they are tired of feeling used and abused by the market research process, a problem that is exacerbated by sub-standard profiling, high disqualification rates, poorly designed surveys, inadequate incentives, and unscrupulous fly-by-night panel operators entering the arena. Will further consolidation of technologies and increased respondent mobility not simply shift the problem from online to mobile instead of solving it?

Like Reply Report

I love these comments discussing quality and price publicly. The goal of my blog posts isn't to say that I'm 100% right, but to start the dialogue. Regarding Dan's questions, panel companies can't afford to pay every single respondent that clicks into a survey. It's not merely a factor of not targeting properly, but more a function of technical communication between sampling and surveying software. Eg, a single question can have quotas applied across the answer choices. At the beginning of the project, the easier to fill quotas finish first while the harder quotas linger. When sample companies systems can't read the quota needs in real-time, panelists get burned. Lenny, I agree that mobile and social will change the MR landscape, but I don't believe it will have as dramatic an effect on quant surveys as you say. And, I still think that people will expect a reward, whether they're on Facebook or not. I believe Angus knows exactly what the concerns are being on the inside of a sample company. I wrote another blog post of Panel Quality and Price. Feel free to agree or disagree. http://blog.usamp.com/2011/10/05/sample-quality-is-a-function-of-sample-pricing/

Like Reply Report

We’ve always done this – every panelist who touches a survey gets something, from the ones who complete, to those who screen out for over quotas, etc. Keeps the panelists happy and engaged and turnover low. Hmmmm…..you been speaking to our people???? Best and grooviest wishes – Nelson Nelson Davis, Client Development Director AIP New York Co., LTD. 1001 Ave of the Americas, Suite 1212 New York, NY 10018 Nelson.davis@aip-global.com THE BEST ONLINE PANELS IN ASIA, no kidding – just ask anyone! Proprietary panel coverage – Japan, China, Korea, Taiwan, Hong Kong, Singapore, Thailand, Malaysia, Indonesia, Vietnam, India and The Philippines.

Like Reply Report