Artificial intelligence used to predict Human Rights Court decisions

Artificial intelligence can be used to predict decisions of the European Court of Human Rights, university researchers have announced.

A team from University College London (UCL), the University of Sheffield and the University of Pennsylvania achieved 79% accuracy by automatically analysing case text using a machine learning algorithm. The study is described today in PeerJ Computer Science.

In developing the method, the team found that judgments by the ECtHR are highly correlated to non-legal facts rather than directly legal arguments, suggesting that judges of the Court are, in the jargon of legal theory, "realists" rather than "formalists". This supports findings from previous studies of the decision-making processes of other high level courts, including the US Supreme Court.

They identified English language datasets for 584 cases relating to articles 3, 6 and 8 of the Convention and applied an AI algorithm to find patterns in the text. To prevent bias and mislearning, they selected an equal number of violation and non-violation cases.

The most reliable factors for predicting the court’s decision were found to be the language used as well as the topics and circumstances mentioned in the case text. The "circumstances" section of the text includes information about the factual background to the case. The 79% accuracy figure was achieved by combining the information extracted from the abstract "topics" that the cases cover, and "circumstances" across data for all three articles.

“We don’t see AI replacing judges or lawyers, but we think they’d find it useful for rapidly identifying patterns in cases that lead to certain outcomes. It could also be a valuable tool for highlighting which cases are most likely to be violations of the European Convention on Human Rights,” explained Dr Nikolaos Aletras, who led the study at UCL Computer Science.

“The study, which is the first of its kind, corroborates the findings of other empirical work on the determinants of reasoning performed by high level courts. It should be further pursued and refined, through the systematic examination of more data,” added co-author Dr Dimitrios Tsarapatsanis, a lecturer in law at the University of Sheffield.

The team said that ideally they would test and refine their algorithm using the applications made to the court rather than the published judgments, but without access to those data they rely on the court-published summaries of these submissions.