OPINION6 July 2009

Lipstick on a pig

Do respondents prefer their surveys boring?

Reg Baker’s The Survey Geek blog is always a good read: if you work in the “research 2.0” field I’d say it’s essential. Even the staunchest enthusiasts for R2.0 would admit that a lot of froth gets talked around the subject – The Survey Geek is cautious and stubbornly insists on looking at the evidence rather than the hype.

His latest post is a good example: Baker takes a look at some work DMS have done which strongly suggests that “eye candy” tools (flash surveys, slider bars) aren’t more appealing to most participants than plain surveys, and may in fact put some off.

Crucially, the participants who were forced into taking the “rich media” survey in the DMS work gave higher ratings to the survey experience, though their drop-out rates were also higher. This leads to a couple of conjectures.

Optimistically, you could say that if the whole industry switches over to rich media surveys everyone will be happier even if left to their own devices they wouldn’t have chosen that option. Pessimistically, you might argue that the quality of the survey experience might have precious little to do with completion rates or data quality.

How could this be? Well, we might have tormented participants so much that they’ve developed a kind of Stockholm Syndrome and come to admire their captors’ multi-attribute grids. But what seems more likely is that an enjoyable experience simply isn’t an outcome that many research participants expect, and therefore it’s stopped being one they care about. With this mindset, cosmetic changes to make the experience more engaging might simply feel contrived. The lesson for research reformers might be that if you’re still making people do the same old things, changing how they’re done might not have the impact you want.

@RESEARCH LIVE

0 Comments