OPINION21 July 2009
Do-it-yourself tools are opening up market research to the masses. But Richard Thornton (pictured) and Bo Mattsson of Cint argue that the DIY trend is about much more than just survey tools, and could have a profound impact on the role that research plays in business.
Research’s July cover story addressed ‘do it yourself’ research, a topic that has raised its head at conferences this year as the industry tries to understand what implications these tools have for the more traditional methods of conducting research.
But DIY research is not the new kid on the block that some may be led to believe. What is new is that there finally seems to be a real debate starting about its validity and whether there is a place for the tools that enable it. The Research article has helped bring this topic to a wider audience, but it really only looks at DIY 1.0. We argue that we have already moved to DIY 2.0 (Cint and a number of other technology-led firms servicing the online research industry certainly have).
DIY 2.0
What does DIY 2.0 mean? First it should be free, if the objective is to encourage conversion and achieve critical mass in the use of such tools. Free survey and reporting tools for field and tab are a starting point, with SurveyMonkey being one of the most popular. But SurveyMonkey is only free for some uses – it’s still a subscription model at heart. There are also many other free web-based tools out there. You only need to type ‘free survey templates’ into Google to see 1.5 million results. This in itself poses a number of questions, not least how to go about choosing the right ones.
Secondly, it means an automated, self-service approach. We are seeing significant developments in self-service survey packages which seem to be positioned to attract SMEs, consumers and corporate clients directly as much as MR agencies. Typically these are in the shape of short, snappy PR or omnibus-type surveys or quick polls. There are also client portals where clients can log in and access proposal and project information as well as track fieldwork, and automated sampling and tracking tools to increase efficiency in both cost and production of deploying and delivering samples.
“What firms have been buying through MR agencies for all these years is, to a large extent, access to the consumer. Access is the last big hindrance to doing real market research on your own”
Thirdly, DIY means making sample accessible to anyone. One of the most important things to understand about DIY research is that what firms have been buying through MR agencies for all these years is, to a large extent, access to the consumer. Access is the last big hindrance to doing real market research on your own, and DIY has the potential to bring it to the masses: MR agencies, SMEs, consumersand large clients. But not just any sample – the quality has to be evident in the sample for all research and there is no reason why DIY tools cannot work with the same principles. Researchers rightly focus on the quality of the data source, and will scrutinise this more closely with DIY approaches, so providing access to robust, reliable and engaged samples are important if this approach is to take off. This is the aim of the Cint Panel Exchange, launched in 2004, a marketplace for online panels accessible to all for direct self-service online sampling.
MR for the masses
Access is now becoming a commodity, reachable by anyone, and this poses a challenge for MR firms as they focus on their core competency, insights. Like many others in our space, our core clients are MR agencies and we work with them very successfully, but we also see a valuable commercial opportunity in opening new channels using technology. Examples include the implementation of widget applications on external websites which are being used to find and connect people to surveys.
The real DIY revolution in research will put consumers at our fingertips, and this may not be so far away – we can look to the financial markets as a barometer for what may come in the online research world. In many markets where online DIY trading took off in the 1990s, the number of consumers trading stock grew by a factor of 20 to 30 in ten years. The main reason behind this change was the internet’s ability to make the stock markets accessible to anyone. By bringing market research to the individual student fixing his thesis, or to an SME checking consumer attitudes in a neighbouring country for a budget of £1000, we may be seeing the same change in research.
What it all means
So what are the dangers and how do we embrace DIY while protecting research as a science? The skill of choosing the appropriate methodology for your research is still, and will remain, absolutely critical. DIY research is about empowering everyone to conduct more basic, quick-turnaround surveys rather than complex, highly tuned research with tightly designed sample frames and stringent quality measures. There is also a valid argument that some people using DIY tools won’t have sufficient statistical expertise to make the right calls about results derived from the research, or to carry out studies with the appropriate design and sampling.
It depends, of course, who you are researching, what you want to achieve and how much your budget is. The current tough economic climate may work in favour of DIY. The DIY approach can provide users with a fast, accurate and inexpensive service at a time when marketing budgets are coming under intense scrutiny and where access to information is more critical than ever. And if researchers know their client’s industry, they can play the role of marketing strategists and offer value through strategic recommendations based on the results delivered.
Richard Thornton is Cint’s UK managing director, and Bo Mattsson is CEO.
The world's leading job site for research and insight
Spalding Goobey Associates
Research Analyst – North Africa
£27,000–£32,000 p.a. (depending on experience)
Resources Group
Associate Research Director – Global Communications & Public Affairs
£50–£60,000 + 30 days annual leave + private health + other great bens!
Resources Group
Research Manager – Environment, Education, Housing, Wellbeing – Social Research Agency
£38 – 48,000 + great benefits
-
Brought to you by:
©2024 The Market Research Society,
15 Northburgh Street, London EC1V 0JR
Tel: +44 (0)20 7490 4911
info@mrs.org.uk -
© Copyright 2024 Research Live
9 Comments
tomewing
15 years ago
I think the "real DIY revolution in research" will be when the people being surveyed get to define the questions and issues at stake in the research. :)
Like Reply Report
Rick Frank
15 years ago
Well, I will have to disagree somewhat. Yes, DIY is fine if you know what you are doing, but the prevalence of DIY polls, questions, surveys, follow-up sales, is I think muddying the waters & diluting the perceived value of Market Research as done. Garbage is garbage out has always been true and if respondents don't know garbage from a serious research project who wins in this scenario? And if anyone can do it why pay for anything? Give me my insights for free. Is CINT planning on giving away panel sample for free as well? I note the software tools and infrastructure to use it should be free but the 'access' to respondents is seemingly excluded from the free model. Really. MR is not about 'free' it is about added value, and yes sometimes some things are done for free (or rather the free part is included in assumptions about prices of things charged for). But free does not pay your wages, free does not pay ISP charges, free only works in a business model is there is a two way exchange of free. Take take take with no give only works short term. When you advocate free be careful what you wish for. Personally I'd like free access to panel sample. When can I expect this? DIY 3.0?
Like Reply Report
Richard Thornton
15 years ago
Valid point and I think that could well be DIY 3.0 where the participant drives the debate and content of what is "researched". To some extent we are seeing this on the social network and branding sites (Starbucks and some of the airlines are good examples where there are discussion boards leading the debate).
Like Reply Report
Richard Thornton
15 years ago
Rick, thanks for your feedback. Let me try and clarify a couple of points as I was limited to a 1000 words in the article. I definitely do not under-value the services of an MR agency, far from it. Our core client base are MR agencies and our services compliment theirs very nicely. My point on the "free" arguement was to merely state that if we really want to see DIY research and tools hit the mass market, accessibility needs to be free or as low cost as possible to reduce the barriers. Sample right now is not free unless one is able to build or access their own lists but the CPI's in the marketplace right now are as low as £1.50 per interview from many providers anyway and don't seem to yet be bottoming out. Like MR agencies core competencies are at a value (and rightly so), so should sample be if it can be backed up by the relevant level of quality and processes that go into managing engaged panelists. Cint's model is such that we don't own the panels so the panel owner dictates the sale price for it's sample. It's a marketplace so people can sell as high/low as they like if there are buiyers willingto pay the price. My comments are not to devalue the MR agency role as I think more MR agencies will start to embrace DIY tools (free or not) and incorporate them into their existing methods and ways of working. Just because we talk about DIY it doesn't mean that the MR agency is frozen out. Our model is self service and we have many agencies globally using these tools directly to service their corporate clients. However, for consumers and SMEs the emergence and availability of tools does potentially change the landscape and I think the industry needs to recognise this.
Like Reply Report
Gill Wales
15 years ago
DIY is great as a means for firms to engage with their customers and generate customer feedback. But the purveyors of DIY solutions need to make clear the limitations of the DIY solution in unskilled hands. Surveys as a basis for encouraging customer feedback is one thing. Surveys as a basis for predicting demand or determining future strategy is an altogether different kettle of fish. It's not just sample quality or statistical analysis, it's also questionnaire design. If a DIY survey user doesn't understand the many and varied ways in which the question they think they're asking is different from the question the respondent thinks they're answering then all sorts of horrible messes can - and do - ensue. This is not to protest that 'only us experts' can be trusted to design a survey, but purveyors of DIY solutions must not encourage the view that someone without training can run a valid and reliable piece of research. The consequences of basing business decisions on carelessly designed research are considerably greater for the client than for the market research industry. Trained market researchers do NOT simply 'add value'. What they do is ensure validity and reliability. If surveys don't have those two essential ingredients then clients would be better off saving their money and not conducting them at all.
Like Reply Report
Mikey B
15 years ago
Just a reaction from client-side/B2B which may not be where this development is truely aimed. But my concern is relatively simple. In the years I've been working with research agencies there's been a consistent refrain of 'we're never valued for the work we do'. And this debate is one that seems to paralyze market research periodically. It's rather like the Marketing 'why don't we have a seat at the Board' debate. I agree. I don't think market research is valued appropriately by the buyers and users. I think the industry has an image problem. And there is little evidence that this situation is being addressed sensibly and collectively. And then I read about the promotion of DIY research e.g. forum/survey application in Google Docs and my mind does a back-flip. From my little rock orbiting on the outer reaches of the research galaxy I see the 'low barrier to entry' that DIY provides as a real pain in the posterior. Just based on my experience the case goes like this - Ø Market research is generally a poorly understood as a discipline by Board members, budget holders and commissioning managers. Ø There are many reasons for this but one of the contributing factors is the 'how hard can it be?' attitude which is fuelled by the ready availability of cheap and fast (seen as good things) DIY survey tools. Ø The buyers decision becomes focussed of methodology and content - hey, lets do it on-line and we'll write the questions ourselves - and not on business objective. Ø These surveys are often/usually poorly written and poorly executed and superficially analysed. Ø As a result the findings (rarely recommendations) are less than insightful and often a poor reflection of the Executives' own prejudices. Ø The Board member, budget holder and commissioning manager walk away thinking - it was cheap and quick but didn’t really tell me much. I'm not sure this market research thing is all its cracked up to be. Maybe we'll skip it next time. Ø And of course the impact on respondents is equally interesting as pestering them with inane questions that have little internal logic or coherent narrative creates a secondary negative impact which in my B2B world hits another set of decision makers. I know this stuff is here to stay. I'm not convinced the research industry has completely worked out the best way to incorporate DIY into its business model.
Like Reply Report
Alex Wilke
15 years ago
An interesting debate although I think it is not really useful to talk about DIY research in such general terms. To stick with the analogy, the fact that professional power tools are now readily available at affordable costs does not mean that everyone has the skill to use them. As tempting as it sounds, hiring a bull dozer and knocking down your wall and building an extension requires more thought, planning and experience than just having the right tools. Similarly, the ability to communicate with respondents doesn’t make someone a great communicator or even researcher. Surely, DIY research has been around for years by using paper questionnaires or untrained call centre staff. DYI research his even easier when it involves monitoring and visiting social networking sites and engaging with respondents in a web2.0 context, but - as we have seen many times - it can dangerously back fire if rules and etiquettes aren’t observed. So if the real new development is fast access to respondents who couldn’t be easily contacted before, than the CINT model makes sense if it was free - which makes me wonder whether the model is more about allowing those who do have access to people to monetize these contacts. Creating online surveys is not rocket science from a technical perspective, but there are many aspects which need careful consideration. Data Protection and data storage and backup is one of them. Other aspects are respondent experience and questionnaire design (beyond flow and wording) – layout design, cross browser compatibility, randomization, rotation, filtering and all the techniques that researchers use to ensure sound data quality and enhance respondent experience. ‘Professional DIY’ tools ensure that users are prompted not to forget these important aspects when setting up and testing questionnaires and allow researchers to concentrate on the research objectives. DIY tools which result in unsatisfactory respondent experience are a big problem and will further reduce response rates and willingness to participate in research, which are essential to our business. Surely, DIY research is tempting when it’s not obvious where researchers ad value - when they act merely as fieldwork facilitators and pocket the margin. Companies have research needs which can be satisfied in-house without compromising on quality with today’s technology - especially if they involve continuous, repeat or tracking surveys and as long as attention to sampling is not neglected. But Companies also have research needs which can only be met by research professionals and require complex or at least carefully designed research and analysis. Researchers need to concentrate on these core skills - the times where projects are run by junior staff and yield large profit margins in fieldwork and data collection might be coming to an end as clients become more cost conscious and aware of technology solutions which allow them to bring this aspect in house at lower costs.
Like Reply Report
Johnny Caldwell
15 years ago
It is rather worrying that this B&Q approach should even be considered a threat to the professionalism of the MR Industry, something that the MRS itself has actually spent a good 60 years plus trying establish out there in the wider community. Perhaps we might as well just cancel all the stats, sampling and analysis courses we run and hold up our hands & surrender to the continual dumbing down / trivialising of what I still believe is a crucial science in both the social and commercial community. If Board Members are going to start making serious strategic decisions based on simplistic evidence of questionable quality then good luck to them. You wouldn't take medication that hadn't been through a rigorous research process, the side effects in case will start itch almost immediately !
Like Reply Report
Martin Lindgren
15 years ago
A thought on the matter of survey design and DIY: I'd say that the divide is not so much between unskilled DIY users and skilled professional researchers, but between professionals with and without skills, or the power/will/motivation to use those skills. Being a member of a few online panels, I come across horrible surveys as well as very good ones, both kinds coming from major organizations within the industry, and sometimes the former being made in systems allowing much more smoothness than what is used. And, being a survey programmer myself (in Confirmit), I've met clients from outside the industry who are very thoughtful when considering respondents' likeliness to answer surveys - obviously, judging from the surveys I just mentioned, superior to that of some insiders. That said, I think Cint has a good case when providing tools and panel access to "outsiders", since being a research company in itself apparently does not guarantee well designed surveys. And when people need surveys not possible in e.g. Cint's own DIY system "Survey Pirate", they can turn to something stronger, just like the small firm eventually switches from homemade ads to an agency. Martin Lindgren Stockholm Business Research
Like Reply Report