Mobile survey optimisation improving, MRS research concludes

In its annual research on mobile optimisation, the MRS found that complete rates for surveys on mobile devices were at 46% of total research surveys, with 50% on desktops and laptops, and 99% completing mobile surveys in 2022 compared with 71% in 2016.

Mobile optimisation also reduced drop-out rates for surveys, with non-optimised mobile surveys having a drop-out rate of 18% compared with 2% for optimised and 3% for laptops and desktop computers.

The research has been run since 2018 using data from Toluna, Dynata, Cint and Kantar, and covers the UK, US, China, Australia, France, Germany, Brazil, the Netherlands, Japan and Canada, with the addition for 2022 of India, Indonesia and South Africa.

This year also saw the third time a separate participant satisfaction survey has been run to identify some of the underlying issues behind the trend data and to test the impact of poor mobile design on participant behaviour and attitudes. 

Mobile was growing in popularity among all age groups compared with desktop and laptops, with women considerably more likely to complete surveys on their phone compared with men ( 53% to 40% across all markets).

A general upwards trend was also observed in mobile starts and completes, in contrast with a fall in laptop and desktop computer starts and completes.

Indonesia and India were the most likely nations to start and complete surveys on a mobile device.

The research found there was no discernible difference between attention check fails for mobile non-optimised and mobile optimised, and equally little difference seen in survey enjoyment.

Discussing the findings during a webinar on the research, Chris Stevens, chief quality officer at Kantar, said that the industry had nearly reached parity in  terms of the survey experience on phones and laptops.

“This is great news, and what we have been focusing on for the last six years,” he said.

“We still have some work to do – we know that 80% of new recruits are joining on phones, so we know this will get higher and higher. We have to keep going here and make sure we do not go back into bad habits.”

Pete Cape, global knowledge director at Dynata, added that work was still needed on mobile optimisation.

“We can’t congratulate ourselves on a job well done by looking at the overall number,” he said.

“All researchers have to get into the habit of checking their surveys on a mobile device for usability. If your excuse for not doing that is that it is fiddly and difficult to do, you have answered your question right there.”

Discussing the dropout rates seen in the research, Marie Hense, vice-president of online data quality at Toluna, questioned whether surveys without mobile optimisation might be skewing their sample.

“An increase in drop-out rates means we will need to replace those respondents somehow,” she said. “That will mean longer fieldwork timelines and higher costs for the data we are collecting.

“But the second question we have to ask ourselves is who are the respondents who are remaining? Are they respondents who have a high tolerance to annoying or tedious tasks, and do they have any differences in the attitudes or behaviours than those who dropped out? There may be a skew we are introducing to our survey by not optimising it.”

Hense added: “Thinking about both worlds is an additional strain on every researcher with testing and making sure our surveys are optimised for any device. But it shows how important this is and will be going forward to consider all the kinds of devices and ensuring optimisation for different kinds of displays.”

Martina Galantucci, lead, project success at Cint, said that improving participant experience was vital for the long-term success of the industry.

“It is not just about attracting more people via mobile, but how to retain them and convert them into completes,” she said. “Keeping your surveys short and making sure the design is appealing is very important.

“The future of surveys should make sure respondents have the same positive experience across all devices. We should allow respondents to choose their device.”