NEWS3 September 2020

Using statistical models for exam results ‘impossible task', says Ofqual chair

Covid-19 News Public Sector UK

UK – Roger Taylor, chair of exam regulator Ofqual, has said that using a different algorithm would not have led to more “acceptable” grading of A-level and GCSE results in the absence of exams this year.

Algorithms code_Crop

In a written statement to the education select committee ( 2nd September), Taylor said that trying to deliver results that were comparable to exams “proved to be an impossible task”.

“A ‘better’ algorithm would not have made the outcomes significantly more acceptable,” he said, adding that the “inherent limitations of the data and the nature of the process” made it unacceptable.

Ofqual initially advised the government to hold socially distanced exams, but the education secretary took the decision to use statistical standardised teacher assessments, Taylor said.

The four nations all backtracked on their moderated grades following widespread criticism over the fairness of the algorithmic models used by qualifications regulators. Ofqual head Sally Collier resigned from her role last week.  

In his statement to the select committee, Taylor said the process used by Ofqual was “not biased” and that its analyses had found no widening of the attainment gap.

However, he admitted that the approach could not standardise very small classes, ultimately benefiting smaller schools and putting larger schools and colleges at a disadvantage. Despite this, Ofqual still viewed standardisation as “preferable” because it reduced the relative advantage of private schools overall.

Taylor said: “Ultimately, the approach failed to win public confidence, even in circumstances where it was operating exactly as we had intended it to. While sound in principle, candidates who had reasonable expectations of achieving a grade were not willing to accept that they had been selected on the basis of teacher rankings and statistical predictions to receive a lower grade. To be told that you cannot progress as you wanted because you have been awarded a lower grade in this way was unacceptable and so the approach had to be withdrawn.”

Taylor, who is also chair of the Centre for Data Ethics and Innovation, said Ofqual had considered a range of algorithm designs and proposals for the approach. 

The Office for Statistics Regulation (OSR) is conducting a review of the process behind the statistical models used by qualifications regulators after the Royal Statistical Society raised concerns over quality and trustworthiness.