OPINION9 July 2014
There has been a lag between consumers’ uptake of mobile phones and researchers use of the medium for data collection but mobile research is a valuable tool argues Edward Appleton.
Data published from the most recent GRIT survey in Q4 2013 confirms this sense of an industry in catch-up mode. Global mobile penetration rates far outstrip MR use levels.
Why is that?
Are we particularly conservative as a professional group, extremely reluctant to embrace innovation until every last nagging methodological doubt has been removed? Or is it more that the types of research we often engage in – especially online quant. – are often ill-suited to the demands of the far smaller screens of a smart or mobile phone? Or is all the excitement around mobile essentially PR and marketing hype, characterised by hyperbole and unrealistic projections?
I have talked to a range of mobile MR practitioners over the past few months – established players, tech companies operating in the mobile MR space, qual specialists, ethnographers – to get a sense of what mobile can and can’t do, and to figure why mobile is still relatively insignificant in overall data collection.
Here are some of my thoughts:
- Mobile does add something unique to the MR toolkit; don’t just disregard it as hype. Mobile allows more contextual understanding, brings researchers closer to the time of something meaningful happening and the data is often enriched with tagged or commented photos. Triangulation is simpler to execute; self-ethnography too.
- The different format does require a design re-think: shorter, more playful, no grids, fewer open-enders. It does require more effort.
- The reward for being mobile-attuned in our questionnaire designs – more succinct, shortened lists, irrelevant questions eradicated, no duplication – can be a higher response quality. Shorter surveys can, according to validation work executed by TNS, have higher predictive validity than longer ones.
- Mobile works well when integrated into a multi-modal design, seldom as a stand-alone tool. It works well for a range of studies in both quant and qual – diaries, shopper insights, consumer decision journeys.
- Shifting to mobile as a data collection mode needs careful stakeholder management, the data patterns may well be different to those generated by a traditional online survey.
My overall take out was that mobile is indeed a mega-trend – sustainable, global, impacting across society and business – and is far too central to so many different areas and industries to be treated as ‘just another data collection mode’. Executing mobile is also eminently affordable.
So why are we, perhaps, treating mobile as publishers would have treated the e-book about 10 years ago?
Maybe mobile is pushing us harder than we like to re-shape many of our legacy research approaches. It would effectively mean us having to push back on overly long questionnaires, repetitive questions, grids – often to budget stakeholders, with the potential risk of losing a piece of business or perceived authority. Engaging with mobile also drags us into areas that haven’t necessarily demonstrated a huge business impact in MR – social media, for example, and arguably gamification too – but do require us to engage differently in our insights quest. Mobile is a playful, fleeting, visual medium – effective mobile research needs to respect that, mimic behaviour as it evolves, rather than just taking a survey that was designed for a laptop and drop it onto a mobile device.
I sense that mobile is beginning to reach a critical mass (with complete levels of about 20%, 30% predicted to be reached within the next two to three years) at a time where there are many more disruptive forces moving through our industry such as DIY software.
Added to that, there’s increasingly little that seems controversial about mobile; validation is continuing apace, so it’s easy to shrug one’s shoulders, accept its role, rehearse the arguments about getting nearer ‘the moment’, overcome potential memory gaps, improve data quality, but not see it as hugely disruptive. Alternatively, and I think more likely, mobile could be something more radical – a Trojan horse, forcing us to change the way we approach research design –embrace the micro-survey, ask fewer questions, triangulate more regularly through multiple data sources.
Predicting the future is often a fool’s game. However, given how central mobile is to so much of new communications and technology innovations – just think how the combination of smart phones and NFC will transform the retail environment – it would be foolish not to assume that mobile in one shape or other will radically re-shape our approach to how we engage with research participants.
We need to be prepared: by having a mobile mindset whenever we approach a research design; asking the question, how can mobile help? It’s likely to be the way more consumers will wish to engage with us in future.
In the Moment by Edward Appleton is available in paperback, and can be ordered online at lulu.com at a cost of £17.50.
The world's leading job site for research and insight
Resources Group
Senior Research Manager – Global Communications Consultancy
£45,000–£50,000 (flex) + Superb benefits
CMA
Director, Market Research
£96,923–116,307
Resources Group
Associate Research Director – Focus on Ad hoc studies! – Innovative Insight Consultancy
£45,000–£55,000 + great benefits
Related Articles
-
Brought to you by:
©2024 The Market Research Society,
15 Northburgh Street, London EC1V 0JR
Tel: +44 (0)20 7490 4911
info@mrs.org.uk -
© Copyright 2024 Research Live
10 Comments
Simon Chadwick
10 years ago
Thanks for this very thoughtful piece, Edward. One thing I would emphasize is that there really are at least two types of mobile research. Type 1 is where we do surveys (even if shorter and more succinct) via mobile; and Type 2 is all the sexy stuff such as auto-ethnography, in the moment and shopper insights. Where Type 1 Mobile is concerned, researchers should understand that they do not have a choice. More than 20% of online surveys are now being accessed by people using their mobile devices. Which means that, unless we want to lose these people in our surveys, we will HAVE to change the way in which we design surveys and the way in which we think about them. Some research companies are sticking their heads in the sand and are telling their data collection partners to discard completes done on mobile devices. That way madness and destruction lie.
Like Reply Report
Michael Virks, YouthSight
10 years ago
The market research industry at large always seems to be in catch-up mode about one thing or another. This is the same industry that was busy phasing out cardboard punch cards in the second half of the 90s. There are still software providers out there specifically peddling “mobile survey” technology, as if this was a special case that you should plan for and set up separately. Very slowly, some big survey software suites are coming around to responsive systems that do the needful automatically. But the vast majority that we looked at still have a long way to go, and even the better offerings involve having to zoom in and out of the survey on a mobile repeatedly. Hey, you even got a lot of big players in the market still deploying Flash surveys, which at this point essentially only work on Windows PCs, nothing else. But as you say, it’s not just technology; it’s also about a change in attitude and approach to questionnaire design. While it’s debatable whether 30-40 minute surveys ever really made sense, they most certainly do not for mobile data collection. You can realistically inflict 15-20 minutes tops on someone taking a survey on a mobile. This will mean a stronger trend away from random sampling, towards panels: if you already have all the demographics you need, the questionnaires will be much shorter. You also need to think about what kind of questions you ask, and how you ask them. A 30-item answer list is a lot less viable on a mobile device. Long question texts don’t work well, pop-up images and info boxes are not user-friendly on mobile devices. Overall, the trend will be towards simpler, shorter questions. Because opening surveys on mobile devices is already the standard, not the exception that “also” has to be catered for. We’ve been banging on about “Mobile First” for quite some time now. But it’s not like anyone will really get a choice in the matter. Our respondents don’t care about how much time we as an industry spent sitting on our hands about this. If it doesn’t work nice and easy, no matter what device they open it on, they’ll just lose interest and move on, never to return. We’ve pushed this hard – and it’s surprisingly hard to properly design and program a layout that just works and is a joy to use on all devices – and have been rewarded by a significant increase in response rates (and positive feedback to boot). Just over 45% of our respondents complete our surveys on mobiles now, and we’re set to hit the 50% mark in only a month or two.
Like Reply Report
Edward Appleton
10 years ago
Simon, Michael - thanks very much for sharing your thoughts. @Simon - type of survey or study is for sure the key driver, alongside objectives and audience focus. Why - as you state - would a data collection company discard mobile completes? Any quality or validation issues? Sounds extremely odd. @Michael - absolutely key, so thanks for doing that, to stress that for many younger audiences, eg "youth", students, young professionals, mobile first makes eminent sense. I wonder how long it will be before it becomes "!mobile only". Any guesses from your Agency? Surprised by your suggested max. length for mobile being 15 - 20 mins, what sort of categories or areas were these studies on? Seems very long for keeping mobile engagement high.
Like Reply Report
Ernst Aben
10 years ago
Like many things, if you change the medium also change the content. You cannot push a traditional market research survey to a mobile platform. The same why newspapers slowly discover that movie users want another type of experience. Gamefication is one of the ways but making surveys easier to take more fun and engaging will be the only way for consumers to take surveys in mass on mobile devices. The large market research companies are usually to slow on acting on this. Hence many companies like ours are filling the gap ;)
Like Reply Report
Colin Deller
10 years ago
Although there are comparisons to moving from paper to online - it feels like there is a difference here with mobile. In that yes we shouldn't be holding back mobile, but we shouldn't be pushing it just because customers are using this technology. It is at the end of the day just another methodology to collect data, so much so that we should be applying this based on the business question. Do I want to know what people experience when they consume a product, if so then mobile probably works very well in terms of collecting that. For other business questions the in the moment requirement is irrelevant. Online research will not go away, there are business questions that fundamentally require it, but there is education required in the industry as to how for certain issues mobile works better - we just need solutions in place that make that decision as easy as commissioning an online study.
Like Reply Report
Michael Virks, YouthSight - MRS Operations Awards
10 years ago
@Edward: The 15-20 minute estimate is largely based on our main sign-up questionnaire, which will take that length and actually has a lower(!) drop-out rate on mobiles with our new responsive template than on desktop. Which to us indicates that this is perfectly workable if you design the survey questions and the layout right. We do generally try to push towards 7-12 minutes with actual questionnaires though. @Colin: In most situations, there will be no realistic way to control whether respondents access an online survey on their mobile or on their desktop/laptop. So unless your systems are ready for this, you will be dealing with a comparatively high drop-out rate and rather annoyed respondents. I mean, sure, I have a bit of a chuckle when I click on an invite from Big Panel Provider Ltd on my mobile email and just get a page with code dumped onto it instead of a question, but I suspect most actual respondents will feel a tad different...
Like Reply Report
Colin Deller
10 years ago
Sorry that wasn't really my point - my point was that the business objective should define the optimal device used, I don't feel we should just change the way we do research just because people are using a type of device more-so than another. If I want people to spend 30 seconds studying a detailed image and getting feedback and they try to access the survey via phone, its quite straightforward for the survey to know this, stop the survey and put up a message asking that the respondent access it via a PC at a later time without any issues. Yes that means drop out rates might be higher but I don't want people trying to answer a survey that was never meant for a mobile phone where they cannot answer the question in the manner we would like. The opposite also applies in terms of pushing people to mobiles. Fundamentally the phone is another source of data, along with my online gained research data, my big data databases, the ongoing tracking data, and so on and so forth. Selecting from all these data/methodologies what best suits the business question seems to be the most logical approach?
Like Reply Report
Anon
10 years ago
But Colin I think you are missing a big point in that because of 'mobile' - and incidentally we should distinguish between phone and tablet too - people aren't using PC/Laptops as much. Younger techies are already beginning to drop them (laptops/desktops) as personal devices, this trend will continue (and I believe spread to work too) and relatively soon desktop/laptops ownership will decline and your PC audience won't be representative of the general population.
Like Reply Report
Colin Deller
10 years ago
No I get 'mobile' in fact it is something I am developing. Ipsos for instance claim that in 2-3 years 20-30% of survey will be done on mobile which is very believable. My point really is that we should push a solution depending on the business objective, if mobile is perfect for answering it then fine, if not, then its something else. Just my opinion that you use the right tool for the job, how would you conduct something like shelf testing on a mobile device for instance? I am not sure the 'representative' argument works that well, as pointed out mobile response is the realm of younger tech savvy etc, there are still some to go until that itself becomes more representative (if you can even say self selected people on panels is representative) than online. I am not saying don't move to mobile, I think we just need to seriously consider the device restrictions that mobile brings and the objectives of the research before we force through some research on mobile.
Like Reply Report
dave bostock
10 years ago
Very good article on the use of mobile for research purposes Edward. I think one of the main advantages of capturing responses via mobile is the accuracy it tends to give. Here at Cello Health we commissioned a project to understand the reality of living with Crones disease and captured data via a mobile diary and then did a tradition on-line survey that had the more widely used "Thinking about the last 2 weeks ..." question in there on the same measures. The interesting findings in this was that the mobile diary captured the variety of levels of discomfort with say pain over a 2 week period, where as the online only reported the higher levels that the patient captured in that time period .. I believe it is known as the peak end effect. Utilizing mobile in this way allows for us to get a more detailed picture of the patients experience over time. Both results are valid by the way .. one is how the patient feels on a daily basis allowing for us to get a truer picture of the impact of the disease on their day to day life and the recalled measure is also correct (even if it is a little skewed to the higher end of the scale) as this is what the patient would be telling a health care professional when presenting with the disease. Mobile is the only way we can efficiently capture and engage with respondents in this manner.
Like Reply Report