Using statistical models for exam results ‘impossible task', says Ofqual chair
In a written statement to the education select committee ( 2nd September), Taylor said that trying to deliver results that were comparable to exams “proved to be an impossible task”.
“A ‘better’ algorithm would not have made the outcomes significantly more acceptable,” he said, adding that the “inherent limitations of the data and the nature of the process” made it unacceptable.
Ofqual initially advised the government to hold socially distanced exams, but the education secretary took the decision to use statistical standardised teacher assessments, Taylor said.
The four nations all backtracked on their moderated grades following widespread criticism over the fairness of the algorithmic models used by qualifications regulators. Ofqual head Sally Collier resigned from her role last week.
In his statement to the select committee, Taylor said the process used by Ofqual was “not biased” and that its analyses had found no widening of the attainment gap.
However, he admitted that the approach could not standardise very small classes, ultimately benefiting smaller schools and putting larger schools and colleges at a disadvantage. Despite this, Ofqual still viewed standardisation as “preferable” because it reduced the relative advantage of private schools overall.
Taylor said: “Ultimately, the approach failed to win public confidence, even in circumstances where it was operating exactly as we had intended it to. While sound in principle, candidates who had reasonable expectations of achieving a grade were not willing to accept that they had been selected on the basis of teacher rankings and statistical predictions to receive a lower grade. To be told that you cannot progress as you wanted because you have been awarded a lower grade in this way was unacceptable and so the approach had to be withdrawn.”
Taylor, who is also chair of the Centre for Data Ethics and Innovation, said Ofqual had considered a range of algorithm designs and proposals for the approach.
The Office for Statistics Regulation (OSR) is conducting a review of the process behind the statistical models used by qualifications regulators after the Royal Statistical Society raised concerns over quality and trustworthiness.

We hope you enjoyed this article.
Research Live is published by MRS.
The Market Research Society (MRS) exists to promote and protect the research sector, showcasing how research delivers impact for businesses and government.
Members of MRS enjoy many benefits including tailoured policy guidance, discounts on training and conferences, and access to member-only content.
For example, there's an archive of winning case studies from over a decade of MRS Awards.
Find out more about the benefits of joining MRS here.
0 Comments