OPINION24 March 2011

SurveyMonkey makes its case

SurveyMonkey’s chief methodologist takes on the critics who blame the rise of DIY survey software for a fall in response rates and quality.

Taking to the stage yesterday afternoon in a session on online research was Philip Garland, chief methodologist for SurveyMonkey.

“I was surprised to find out they had a chief methodologist,” said session chair Martin Oxley. He said he meant it as a compliment, but the room took it to be an amusing dig at the DIY survey software developer.

Garland shrugged it off. He said he knows SurveyMonkey has a bad reputation among certain sections of the research industry. He asked the room to raise their hands if they thought – now or ever – that SurveyMonkey was “a bad word”. A majority raised their hands.

In recent years people have started to be bombarded with surveys, many of them of dubious quality. With the rise in surveys has come a corresponding decline in response rates.

SurveyMonkey and other DIY software suppliers have been blamed for this state of affairs, Garland said, as their arrival on the scene coincided with the rise in survey invites.

But addressing delegates, he said: “You of all people should know that correlation is not causation.”

Garland’s pitch to the audience was that the research industry should drop their hostility to SurveyMonkey. The people taking the company’s surveys are just like anyone else, he said, and to prove it they ran their own studies of presidential approval and economic confidence to compare to Gallup’s numbers. SurveyMonkey’s raw, unweighted data seemed acceptably close in comparison on most occasions.

The company also has sheer weight of numbers on its side. SurveyMonkey did 600 million completes last year, Garland said – a staggering number.



13 years ago

David Garland's article in online research sector analysis (Research magazine, March 2011) asks many basic questions about market research. Recognising that DIY survey tools are being used by people who have little or no understanding of the principles of research, the Independent Consultants Group recently launched a free guide. It was written by members of the Market Research Society and includes the basics of question design, sampling and analysis. It's partly aimed at organisations who don't have the resources to commission surveys from professionals. But also at client-side research managers who need to get a grip on departments doing their own thing. By passing on the DIY guide, client-side research managers can stop DIYers making basic mistakes and remind them of the in-house experience available just along the corridor. The guide is available to download on the ICG website: http://indepconsultants.co.uk/resources/diy-research-guides/

Like Report

13 years ago

The article that Gill refers to above is now available online at bit.ly/diy0311

Like Report