Terms outlined for inquiry into Australian polling

The AMSRO is conducting the review to determine why all the polls incorrectly forecast the election result.

“An impartial and comprehensive industry-wide inquiry process is required to examine the performance of the polls and restore public confidence in the credibility of polling,” said Craig Young, president of the AMSRO.

The inquiry panel will be tasked with assessing the accuracy of the polls, evaluating whether there is any previous pattern of inaccuracies, and the causes for any inaccuracies.

The draft terms of reference for the panel also include:

• to consult and seek evidence and insights from polling companies and other relevant stakeholders, including those who commission and publish polls
• to assess the adequacy of information provided to, and by, the media and other commentators about how polls were conducted and what the results mean
• to submit a preliminary report to AMSRO as soon as practicable, with a view to publication of a final report by AMSRO as soon as possible thereafter (timing to be determined by the inquiry panel).

The panel will be chaired by Darren Pennay, the founder and former chief executive of Australia’s Social Research Centre. Pennay said: “The focus of the review will be to objectively assess the quality of the polling data by looking at the techniques and processes employed across the different polling organisations. The approach needs to be broad and comprehensive, without any a priori assumptions about what, if anything went wrong.”

The inquiry will not assess the relative performance of individual polling companies, but instead focus on practices and processes in use across the industry. Pennay added: “We will aim to mask or de-identify the polling companies’ data, and panel members will sign non-disclosure agreements. The inquiry aims to deliver an evidence-based approach to find ways to improve the conditions for polling and the processes and methods used, so all companies can operate with a full appreciation of contemporary best practice.”

The panel will take responsibility for the inquiry’s report and recommendations, operating at arm’s length from the AMSRO. AMSRO has also established an independent advisory board to offer strategic guidance and input. 

AMSRO’s Young said: “We hope that all the polling companies will work with the inquiry to achieve better outcomes for the entire polling industry.  No currently active polling organisations are, or will be, represented on the inquiry panel – polling can be quite competitive, so it’s essential that there is complete objectivity in the approach and no scope for competitive concerns to arise. We want everyone to have confidence in the process and outcomes.”

Other members of the inquiry panel include:

• Murray Goot, emeritus professor of politics, Macquarie University
• Phil Hughes, Asia-Pacific head of statistical consulting at Engine
• John Stirton, independent polling expert who ran the Fairfax Poll between 1997 and 2014
• Paul Lavrakas, expert in survey methodology and survey error
• Jill Sheppard, lecturer in the School of Politics and International Relations, ANU
• Dina Neiger, chief statistician, the Social Research Centre and accredited statistician, Statistical Society of Australia
• Patrick Moynihan, associate director of international research methods at Pew Research Center and previously senior polling analyst for ABC News
• Patrick Sturgis, professor at London School of Economics, who led a polling inquiry after the 2015 UK general election
• Kristen Olson, associate professor and vice-chair of the Department of Sociology at the University of Nebraska-Lincoln.