Facing facts: can promise tracking hold governments to account?

x Sponsored content on Research Live and in Impact magazine is editorially independent.
Find out more about advertising and sponsorship.

Distinguishing truth from fiction has become ever more pressing in the past couple of years. Fake news, a 24-hour news cycle, and a revival of political rhetoric favouring conjecture over fact has led to a climate of distrust and misinformation.

In this muddled quagmire that some have termed a ‘post-truth world’, the role of non-partisan, fact-checking organisations – such as Full Fact – has grown, as people seek truth from falsehood.

Echo chambers, multiple demands on our time and attention, and partisan mainstream media can make it difficult for people to understand what progress has been made on government pledges. To tackle this, two students from the universities of Warwick and Cambridge have produced a research paper outlining a promise-tracking system that they hope could hold politicians to account and identify the progress made on their pledges.

Staying on track

Jordan Urban and Adam Feldman, authors of The Future of Promise Tracking paper, embarked on their research during an internship at Full Fact. At secondary school, the two were part of a group of students that built GovTracker, a website to monitor UK government pledges, sparking their interest in the wider democratic benefits of promise tracking. The most important function of the technique, says Urban, is to ensure government accountability. 

“The government receives an electoral mandate to act on the promises it has made. If we don’t ensure that the government is keeping its promises, it may ignore them, and this can usually be considered as acting undemocratically.”

The need for promise tracking is heightened by our constant access to news, often across multiple media sources, which can make it difficult to remember whether the government has acted on a mandate. “We are bombarded by stories all day, every day, so it is very easy to have short memories,” says Urban. “People generally care about manifesto promises during elections, but partly because of the way news cycles work – and partly because most people have better things to do – very few track them to see if they are implemented.” 

While distrust in politicians isn’t a new phenomenon, Feldman feels promise tracking is particularly relevant in today’s environment. He says: “The reason for the lack of trust in politics is it has moved from the relatively shallow issues of expenses scandals and sleaze to more deeper issues of truth and delivering on promises. Voters overestimate the betrayal of opposition parties and many underestimate the achievements of government. 

“Our website has constantly shown a higher percentage of promises kept than people expect. A promise tracker could really help people perceive the achievements of governments more accurately.”

One of the biggest requirements of a promise tracker is measuring progress in a non-partisan way – that is, defining progress as ‘the extent to which the government has achieved its stated aims’ rather than subjective judgements about whether a government’s agenda is ‘good’ or ‘bad’. However, for Urban and Feldman, the biggest challenge of non-partisanship has not been philosophical, but practical: obtaining primary, objective sources.

When they were working on GovTracker, they used three main sources of information to determine the status of a promise. Primary sources – such as those produced by the government, including data from the Office for National Statistics and Hansard, which keeps a record of everything said in the Houses of Parliament – were used to reference 75% of GovTracker’s statuses in 2017. Non-partisan sources, such as the BBC, Full Fact and the Institute for Government, and partisan sources – such as The Guardian, The Telegraph and The Times – were also referenced. For 18% of the promises they collected, however, they could find no evidence of progress.

Another challenge when building a promise tracker is the resource required to compile and gather the information needed. In their paper, Urban and Feldman highlight the potential of automated fact-checking technologies to identify and track a promise when it’s made in the public domain. Full Fact is currently working to automatically detect certain types of claims made – and ‘promises’ are a particular type within the taxonomy of claims. While they aren’t types that the organisation can check, it believes it might be able to spot them automatically in future.

Measuring progress 

A promise tracker could be used for empirical analysis of a government’s progress, as long as it weights the promises it collects by importance – one of the paper’s recommendations. Previous promise-tracking research has either weighted pledges equally, which results in the artificial weighting of less important promises, or has focused on an arbitrary set of promises that happen to be measured. 

Urban and Feldman are exploring an experimental method of choosing appropriate weights for different policies, by polling a sample of the UK population about which of the government’s 2018 promises they consider to be most important. Their intention is to make the data and methodology publicly available for anyone looking to create a promise tracker in future.

The poll will allow the researchers to create an empirical measure of government progress, rather than relying on ‘expert’ opinion or clunky estimation. They can then weight government promises according to their relevance to a mandate and produce a mark for overall progress.

The full Future of Promise Tracking report can be read here.

This article was first published in Issue 22 of Impact.