Legal Judgement Prediction for UK Courts

Legal Judgement Prediction (LJP) is the task of automatically predicting the outcome of a court case given only the case document. During the last five years researchers have successfully attempted this task for the supreme courts of three jurisdictions: the European Union, France, and China. Motivation includes the many real world applications including: a prediction system that can be used at the judgement drafting stage, and the identification of the most important words and phrases within a judgement. The aim of our research was to build, for the first time, an LJP model for UK court cases. This required the creation of a labelled data set of UK court judgements and the subsequent application of machine learning models. We evaluated different feature representations and different algorithms. Our best performing model achieved: 69.05% accuracy and 69.02 F1 score. We demonstrate that LJP is a promising area of further research for UK courts by achieving high model performance and the ability to easily extract useful features

The Decisions and Ideal Points of British Law Lords

Policy-sensitive models of judicial behaviour, whether attitudinal or strategic, have largely passed Britain by. This article argues that this neglect has been benign, because explanations of judicial decisions in terms of the positions of individual judges fare poorly in the British case. To support this argument, the non-unanimous opinions of British Law Lords between 1969 and 2009 are analysed. A hierarchical item-response model of individual judges’ votes is estimated in order to identify judges’ locations along a one-dimensional policy space. Such a model is found to be no better than a null model that predicts that every judge will vote with the majority with the same probability. Locations generated by the model do not represent judges’ political attitudes, only their propensity to dissent. Consequently, judges’ individual votes should not be used to describe them in political terms

Identification, Categorisation and Forecasting of Court Decisions

Masha Medvedeva’s PhD dissertation ‘Identification, Categorisation and Forecasting of Court Decisions’ focuses on automatic prediction and analysis of judicial decisions. In her thesis she discusses her work on forecasting, categorising and analyzing outcomes of the European Court of Human Rights (ECtHR) and case law across Dutch national courts. Her dissertation demonstrates the potential of such research, but also to highlight its limitations and identify challenges of working with legal data, and attempts to establish a more standard way of conducting research in automatic prediction of judicial decisions. Medvedeva provides an analysis of the systems for predicting court decisions available today, and finds that the majority of them are unable to forecasts future decisions of the court while claiming to be able to do so. In response she provides an online platform JURI Says that has been developed during her PhD, and is available at jurisays.com. The system forecasts decisions of the ECtHR based on information available many years before the verdict is made, thus being able to predict court decisions that have not been made yet, which is a novelty in the field. In her dissertation Medvedeva argues against ‘robo-judges’ and replacing judges with algorithms, and discusses how predicting decisions and making decisions are very different processes, and how automated systems are very vulnerable to abuse

Prediction, Persuasion, and the Jurisprudence of Behaviorism

There is a growing literature critiquing the unreflective application of big data, predictive analytics, artificial intelligence, and machine-learning techniques to social problems. Such methods may reflect biases rather than reasoned decision making. They also may leave those affected by automated sorting and categorizing unable to understand the basis of the decisions affecting them. Despite these problems, machine-learning experts are feeding judicial opinions to algorithms to predict how future cases will be decided. We call the use of such predictive analytics in judicial contexts a jurisprudence of behaviourism as it rests on a fundamentally Skinnerian model of cognition as a black-boxed transformation of inputs into outputs. In this model, persuasion is passé; what matters is prediction. After describing and critiquing a recent study that has advanced this jurisprudence of behaviourism, we question the value of such research. Widespread deployment of prediction models not based on the meaning of important precedents and facts may endanger the core rule-of-law values. </jats:p