Researchers at Northumbria
University, Newcastle University, and the University of Glasgow are to stage
mock criminal trials featuring AI-generated evidence, offering the first
comprehensive look at the responsible use of AI in the courtrooms of Scotland,
and England and Wales.
The trials will examine how
juries, lawyers and judges respond when AI-generated evidence, including facial
recognition and large language model outputs, is presented in court. The
findings will help to ensure that trials remain fair as AI tools are introduced
at more stages of the criminal justice system.
The first trial will take
place in Glasgow on 11-12 November, followed by a second in Newcastle upon Tyne
on 16-17 March. The case is fictional but realistic, with actors playing the
witnesses and the defendant.
Working with a research
recruitment agency, the team has recruited 15 members of the public to act as
mock jurors for the Scottish trial and deliberate as a group. The trial will
also feature two real professionals giving evidence: an expert witness in
biometric systems and a serving police officer.
The Scottish trial will be
presided over by Colin Sutherland, Lord Carloway, former Lord Justice Clerk of
Scotland. Advocate depute Alan Parfrey will lead for the prosecution and Tony Lenehan KC for the defence. Ahead of the
Glasgow trial, a pilot study has taken place in Northumbria University’s moot
court room, with Northumbria law students assuming the roles of the judge, counsel, and
mock jurors.
Professor Marion Oswald MBE,
Professor of Law at Northumbria University, Senior Visiting Fellow at the
Centre for Emerging Technology and Security (Alan Turing Institute) and
PROBabLE Futures Lead, said: “The mock trials are of major importance to the
project. The research findings from the mock trials will enable us to explore
how the introduction of AI-based evidence into a criminal trial scenario will
affect fairness and justice and so inform future rules and guidelines for the
legal profession and court process. We will explore how jury members react to
the introduction of AI-based evidence - do they misunderstand its probabilistic
nature, do they over-rely on it to the exclusion of other relevant evidence, or
do they ignore it?! What about the lawyers involved - do they understand what
AI-based evidence means, or do they need more training and support? It is
crucial to explore these questions so that AI-based evidence can be introduced
into court in a way that will not undermine the fairness of a trial.”
Dr Angela Paul, Research
Fellow on the PROBabLE Futures project, based in the Law department at
Northumbria University, said: “Probabilistic systems can produce false
positives and false negatives, and as information passes from one stage of the
system to the next, these errors can compound; a ‘chaining’ effect that must be
flagged.
That is why outputs that may
end up as evidence in the future in courts are being tested through our mock
trials. In our mock trial, we have asked the advocates to treat the materials
we gave them as if they had received them in a real case, so their arguments,
and their exchanges with the expert witness, follow a realistic course. This
lets us observe how the evidence was interpreted by counsel, the judge and the
jury. A large-scale research simulation like this produces empirical findings
that would otherwise be missing.”
Dr Mackenzie Jorgensen,
Research Fellow on the PROBabLE Futures project, based in the Computer Science department
at Northumbria University, said: “I believe it is critical that we explore AI
literacy within the mock trial. AI is already seeping into the criminal justice
system and AI-generated evidence may soon enter the courtroom in England and
Wales (if not already!). To prepare for this, we need to understand how jurors,
lawyers, and judges interpret AI-generated evidence, and AI full stop, in terms
of its trustworthiness.”
Dr Elizabeth McClory-Tiarks,
Lecturer in Law and Criminal Justice at Newcastle University and PROBabLE
Futures Co-Investigator, said: “We hope that the trials will tell us more about
how much trust jurors might place in AI outputs and what influences their
decision-making in relation to this, as well as where misunderstandings about
uncertainty in AI outputs could arise. We will capture not just data about the
trials, but also data about pre-trial preparation by the legal professionals
involved. This will provide a more complete picture of their understanding of
uncertainty in AI outputs and how this influences the way that they use the
information in the trial process.”
The mock trials are part of PROBabLE Futures (Probabilistic AI Systems in Law Enforcement Futures), a
four-year, £3.4m Responsible AI UK-funded Keystone Project led by Northumbria
University with the universities of Glasgow, Northampton, Leicester, Newcastle
and Cambridge. The project is developing a framework to understand the
implications of uncertainty and to build confidence in future probabilistic AI
in law enforcement, with the interests of justice and responsibility at its
heart.
Its research includes mapping the AI ecosystem in law enforcement, including
the use of large language models, and developing guidance, checklists and
frameworks to guide the assessment of scientific and legal validity. The trial
scenario draws on that mapping and was refined through a systematic approach to
scoping and months of consultation with partners.
Professor Oswald, principal
investigator of PROBabLE Futures, added: “PROBabLE Futures is a Responsible AI
UK Keystone project led by Northumbria University and involving a genuinely
integrated interdisciplinary team across six universities. Our focus is on the
use of AI in law enforcement, including how the use of AI by the police can
impact the criminal justice system further down the chain, and how humans react
to the outputs of AI - do they trust them, misunderstand them or over-rely on
them?”
