Methodology in context

Personality tests

Identifying 14 different personality tests, Lightspeed Research tested them on their panellists and decided the data produced was "rubbish" said Jon Puleston, its vice-president innovation, in his opening session on re-imagining personality tests for commercial market research. The main issues were that the tests were developed by academics, based on old Likert scales and heavily dependent on self-assessment. He found it was very hard to achieve any meaningful differentiation.

Puleston went on to explore behavioural manifestations of personality traits and found that combining behavioural measure made the tests more effective. "The trick is gathering enough data points," said Puleston, pointing to between 50 and 100 needed per individual.

He then added in a taste element and tested it with a profiling approach where respondents were asked to imagine they were going to Mars and then select what music they'd take, extending this out to other fun categories such as movies and food. "It was lots more fun, and we got response rates in the high 90%s."

This technique has yet to be used by clients but said that behavioural AND taste mapping, with enough data points, appeared to be a very reliable means of measuring personality, that removed the self-assessment bias. But Puleston also added that it underlined just how contrary humans are, with fluid personalities that change with context.

Mobile video

The market research community is all too aware of the unreliability of people remembering what they have done, which is where in-the-moment research becomes so valuable. But ethnography is expensive, which is why Jon Cohen, managing partner at Kindling Strategy has turned to mobile video diaries to gain better insight.

"For the first time ever, the default is to be honest," claimed Cohen. "I don't think it’s possible to develop a behaviour change strategy without doing this." He shared videos and provided his top tips for mobile video: keep it simple, use Whatsapp; give people licence to be honest; assume there will be some misunderstanding; let those involved partner up for better quality response, even if the other person doesn't fit the spec; weekends are different to weekdays and be flexible and realistic – you can't be rigid and don't expect too much.

Tackling low-engagement 

Often the issues that are most important for making business decisions are those which people engage with the least, so market researchers must find ways of engaging respondents with less ‘interesting’ topics. Cordelia Hay, research director at BritainThinks, outlined a few approaches the agency has taken, to make the process more engaging.

Finding the emotional or real-life connection with a seemingly dry, irrelevant issue is key, she said. For instance, while the public have little engagement with the future of auditing insurance, they do care about trust and transparency in business.

Flattery can also be powerful and allowing them to take an active role – Hay referenced the approach taken by Thames Water. “We don’t call our respondents respondents – they are members of the customer board. They don’t consider themselves to be ley members of the public, but strategic advisors to the board,” she said.

It’s also key for moderators to give people permission to speak honestly about what they think – including if their opinion is just that they don’t care. This is particularly important in political research. "Giving people permission to have an irrational conversation about politics is very important. Most polling assumes a level of political engagement that most voters don’t have."