What happened in Vegas

When creating a conference programme, it’s easy to emphasise hype and hyperbole and hard to stress pragmatism. Kudos to the presenters at The Market Research Technology Event (TMRTE) this week for offering nuanced views on where technology is taking the industry. Here are some of the key themes.

Behavioural data needs attitudinal data for context

Chris Jones of Zynga described the integration of surveys and text analytics with behavioural data from players’ experiences with their games. For instance, a simple measure like the Net Promoter Score (NPS) cross-tabulated by the respondent’s current progress within the game Hidden Chronicles provided valuable feedback about where players were getting frustrated.

Sheila Normile of Facebook discussed the role of ethnography, in-home visits and projective qualitative research of Facebook users to help programmers truly understand their target audience, while Sucharita Otta and Marc Harwitz of the American Cancer Society talked about the need to move beyond behavioural segmentation to improve marketing to donors and volunteers – they’ve launched a major initiative to integrate attitudinal data into their segmentation to improve marketing ROI.

Google Consumer Surveys is here to stay

Some researchers have wishfully said that Google Consumer Surveys is the next Google Wave, Google Lively or Google Radio Ads – an experiment destined to be short-lived. However, it was clear from Monica Plaza’s packed closing presentation that Google has entered the market research industry with a long-term vision. In a wide-ranging and remarkably transparent discussion, she said there will be:

• more question types to come, from the prosaic (open-ended numeric entry and matrices) to more creative formats
• open-ended questions, once Google can provide valuable analytics of the verbatim responses
• more screener options, once the volume is there, including targeting people based on DoubleClick’s assessment of their interests (e.g., sports, politics, entertainment)
• more countries – another English-speaking country this year, with other countries to follow as publishers are lined up and national privacy policy requirements are met
• Android support, once a system for inferred demographics is in place

Surveys are also here to stay

Tom H C Anderson of Anderson Analytics said: “While social data is important, surveys will always be important. We have to ask questions to get specific data.” Surveys were not Google’s first choice for helping their publisher partners find additional ways to monetise content: they initially tried microtasks similar to those found on Amazon Mechanical Turk, asking users to classify images or rate search results. To their surprise, they found that surveys had three to four times the participation rate. Turns out people don’t mind answering a question or two. Speaking of which…

Surveys must be short, but questionnaires can be long

For market researchers, stopping at one question is like eating only one potato chip – impossible for all but the most strong willed. The key to satisfying both researchers and respondents is to always field questionnaires to individuals as subsets. For instance, Zynga asks players three-question surveys in its games:

1. the NPS question, ‘How likely are you to recommend…’
2. a randomly selected question about a specific aspect of the game
3. an open-ended question for any additional feedback

If Google has to choose between satisfying the respondent or the researcher, they’ll choose the respondent first every time: the implied contract they are creating for respondents is that they will never have to answer more than two questions. Of course, their survey authoring user interface lets researchers write questionnaires of any length, but with only one branching question (the screener). Now think about the Zynga and Google approach for mobile: while some presenters discussed the challenges of fielding surveys to mobile devices, a similar strategy is probably needed to increase survey participation rates on phones and tablets.

Editor’s note: In case you missed it, be sure to check out our Q&A with Coca-Cola’s Stan Sthanunathan digging into his TMRTE keynote.

Jeffrey Henning (@jhenning) pioneered the enterprise feedback management industry – and the #MRX community on Twitter. He works for Affinnova and previously wrote about crowd-shaped surveys.