The rules of the game

The idea of gamification has captured the imagination of the research industry recently. But what happens when you apply it in practice? At Esomar Congress in Amsterdam today, Theo Downes-Le Guin shared some findings today from an experiment conducted by Market Strategies International.

The agency ran an experiment to test the effects of gamification. The results were not quite what they had expected – but they were enough to suggest that the idea is well worth experimenting with.

The company created four versions of an online survey about energy issues: a text only, a ‘decoratively visual’ version, a ‘functionally visual’ version (where visual elements were more closely integrated into the way questions were presented) and a gamified version. The gamified survey was inspired by fantasy role-playing games – participants created an avatar and progressed through a fantasy world by answering questions, collecting items like swords and shields along the way. At the end of each version of the survey, they asked respondents how much they’d enjoyed it (this time using the same plain text presentation for the entire sample).

The main advantage of the gamified and ‘functionally visual’ versions of the survey were that respondents found them more enjoyable, even though it took them longer to complete. More respondents described the surveys as interesting, easy to answer and quick.

Surprisingly, the introduction of gamification and visual changes did not seem to introduce any demographic bias, although Downes-Le Guin warned that other ways of introducing game elements might do so because of the appeal of games to certain types of people. Gamification also failed to reduce ‘satisficing’ – the tendency of some respondents to sail through the survey with the minimum attention possible.

Interestingly, the gamified survey actually had a completion rate below 60%, compared to more than 90% for all the other versions. Respondents seemed to be put off by having to load the game or by the instructions – perhaps confused by the nature of the task, or losing trust in the process when they expected a survey and a game appeared. But the biggest disadvantage, Downes-Le Guin said, was the increased cost of creating the survey – a factor which is likely to get easier as the company and the wider industry gets better at gamification.

But even with these concerns, Downes-Le Guin concluded that the apparent benefits of gamification are “probably sufficient reason for us to be experimenting in this area”.