The controversy surrounding A-Level results during the pandemic highlighted the pitfalls of using government algorithms to make decisions about us.
There was public outcry when it emerged that A-level students were given unfairly low grades based on a government algorithm which marked students down by 40 percent, particularly high performing students in lower performing schools. The algorithm entrenched inequalities that those unable to sit exams could do nothing about.
Faced with protests and a legal challenge, the UK government retracted the use of the algorithm and the grades this created.
For tech justice group Foxglove Legal, a small team of lawyers, activists and tech experts who led the legal challenge alongside A-level student Curtis Parfitt-Ford, it was a perfect example of the biased algorithms we’re often unaware of but that make hugely consequential decisions about our lives.
“The A-level grading system used in the pandemic was manifestly flawed, it downgraded 40 percent of students’ grades and those most affected were highly performing students in underperforming schools which were mostly state schools,” Martha Dark, a founder and director of Foxglove Legal said.
“It was the first time there was a protest about the use of algorithms in society and government decision making. People have a stake in these systems and they should have a say.”
Founded in 2019 by Cori Crider, Rosa Curling and Martha Dark, Foxglove Legal has built a track record of holding big tech and governments alike to account. They use creative campaigning and litigation against the increasing use of opaque and discriminatory algorithms in government decision-making; the spread of harmful technologies and the accumulation of power by tech giants.
They have so far launched four judicial reviews which have successfully challenged public sector decision making. In August 2020, the same month as successfully challenging the A-level grading algorithm, the team also worked with the Joint Council for the Welfare of Immigrants (JCWI) to challenge a racist government algorithm which meant visa applications from red-flagged countries faced intensive scrutiny and were much more likely to be refused.
The Home Office rolled back and committed to not using nationality as a criteria.
A further two judicial review successes came as the team worked alongside Open Democracy over the government’s Covid-19-related software contract with Palantir.
“Technology is the way power is now exercised; private companies have more power than most states,” Martha said. “The government works for us but we have very little information about the systems in use to make decisions about us. We have the right to understand them and challenge them when they are unfair.”
As a small team, they are strategic about what they take on. Foxglove Legal are currently concerned about the transparency, consultation and procurement of a new NHS data platform which they believe could be making similar mistakes that led to the pause of the ‘NHS data grab’ plans to share GP data. And they have recently been supporting the Greater Manchester Coalition of Disabled People to challenge a government algorithm which they believe targets disabled people for benefit fraud in a disproportionate and discriminatory way.
But change may be happening slowly. A public sector algorithmic transparency standard has been suggested by government and is being piloted by local councils and offers some hope the public sector may move towards developing these systems in the open, with community consultation and engagement.
“At the moment these systems are totally shrouded in secrecy,” Martha said, “I think there needs to be a fundamental shift in how our government approaches this decision making. I’m glad that Foxglove is part of that.”
- JRCT made a 36-month grant to Foxglove Legal in 2020 in support of their work to address opacity and unfairness in UK public sector algorithmic decision making. The grant was made under JRCT’s Power and Accountability programme which supports those working for a world in which power is more equally shared and powerful institutions are accountable to society and aligned with the long term public interest.