Multiple panel membership ‘doesn't affect quality’
US— Respondents who are members of multiple panels are not necessarily a threat to data quality, according to results of a new study by the Advertising Research Foundation (ARF).
Findings of the study into duplication and overlap of online panel members concluded that multiple membership only has an impact on data quality when members take part in the same survey more than once.
Duplication rates when combining two panels – a realistic scenario for a survey – ranged from 2% to 19%, with the average “well below 10%”, the study said. However, for surveys taking place in localised markets the rate could be significantly higher.
Preliminary results of the Foundations of Quality research-on-research project, conducted across a broad array of online panels, were presented at the foundation’s annual Re:think conference in April. Yesterday saw the release of the first of several papers setting out the findings in full at a meeting of the foundation’s Online Research Quality Council (ORQC).
The paper was written by Bob Walker, president of research agency Surveys & Forecasts; together with the ARF’s chief research officer Joel Rubinson and senior vice president Raymond Pettit.
Further papers will look at survey response quality, inter-study comparability and benchmarking. The full industry report, incorporating all the papers, will be published later this year.
Committees of the ORQC focusing on industry solutions and metrics will review the results on overlap and duplication in panels to come up with principles, recommendations and guidelines for the industry, the foundation said.