Forensic science, reliability and scientific validity: Advice from America

In this article we review an important report produced by the US President’s Council of Advisors on Science and Technology, Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods (the PCAST report).2 The PCAST report builds on an earlier report prepared by the National Research Council, Strengthening Forensic Science in the United States: A Path Forward published in 2009 (the NRC report).3 These reports are focused on the organisation, funding and practice of the forensic sciences in the US. In their deliberate and unflinching concern with probative value, particularly the validity and reliability of procedures used by forensic scientists and the way opinions are expressed in expert reports and testimony, both have application to England and Wales. Both reports speak directly to forensic scientists, law enforcement, lawyers and courts. Forensic scientists, advocates, judges and legislators must respond to these criticisms and recommendations if we hope to place the forensic sciences on firm scientific foundations.

Forensic Science Evidence and the Limits of Cross-Examination

The ability to confront witnesses through cross-examination is conventionally understood as the most powerful means of testing evidence, and one of the most important features of the adversarial trial. Popularly feted, cross-examination was immortalised in John Henry Wigmore’s (1863–1943) famous dictum that it is ‘the greatest legal engine ever invented for the discovery of truth’. Through a detailed review of the cross-examination of a forensic scientist, in the first scientifically-informed challenge to latent fingerprint evidence in Australia, this article offers a more modest assessment of its value. Drawing upon mainstream scientific research and advice, and contrasting scientific knowledge with answers obtained through cross-examination of a latent fingerprint examiner, it illuminates a range of serious and apparently unrecognised limitations with our current procedural arrangements. The article explains the limits of cross-examination and the difficulties trial and appellate judges — and by extension juries — experience when engaging with forensic science evidence

Critical reading and writing (CRW) in first year psychology: Mass screening and targeted assistance

Many beginning students struggle with their university study because their high-school experience did not yield the basic or academic literacy skills essential to tertiary learning activities. A diagnostic program was designed to identify and assist students in developing psychology-specific academic literacy skills in the large Introductory Psychology 1A course at The University of New South Wales. In an early lecture period, all students were required to make a written response to a text passage (CRW test). This test required them to take and argue a position. Trained assessors marked their responses according to a number of criteria that ranged from spelling and grammar to the logic of their argument (the position taken was irrelevant). The bottom-scoring 50 students were then contacted and offered special tutorials to assist them with writing their laboratory report. Following these, a second CRW test was offered to the assisted group of students as well as to a control group of students (a second chance to make up percentage points). Students who participated in the tutorials showed improvement on some, but not all, assessment criteria. The implications of these findings are discussed in terms of discipline- vs. non-discipline-specific assessment criteria, and in terms of a cost-benefit analysis of the exercise

Speaker identification in courtroom contexts – Part III: Groups of collaborating listeners compared to forensic voice comparison based on automatic-speaker-recognition technology

Expert testimony is only admissible in common-law systems if it will potentially assist the trier of fact. In order for a forensic-voice-comparison expert’s testimony to assist a trier of fact, the expert’s forensic voice comparison should be more accurate than the trier of fact’s speaker identification. “Speaker identification in courtroom contexts – Part I” addressed the question of whether speaker identification by an individual lay listener (such as a judge) would be more or less accurate than the output of a forensic-voice-comparison system that is based on state-of-the-art automatic-speaker-recognition technology. The present paper addresses the question of whether speaker identification by a group of collaborating lay listeners (such as a jury) would be more or less accurate than the output of such a forensic-voice-comparison system. As members of collaborating groups, participants listen to pairs of recordings reflecting the conditions of the questioned- and known-speaker recordings in an actual case, confer, and make a probabilistic consensus judgement on each pair of recordings. The present paper also compares group-consensus responses with “wisdom of the crowd” which uses the average of the responses from multiple independent individual listeners

Speaker identification in courtroom contexts - Part I: Individual listeners compared to forensic voice comparison based on automatic-speaker-recognition technology

Expert testimony is only admissible in common law if it will potentially assist the trier of fact to make a decision that they would not be able to make unaided. The present paper addresses the question of whether speaker identification by an individual lay listener (such as a judge) would be more or less accurate than the output of a forensic-voice-comparison system that is based on state-of-the-art automatic-speaker-recognition technology. Listeners listen to and make probabilistic judgements on pairs of recordings reflecting the conditions of the questioned- and known-speaker recordings in an actual case. Reflecting different courtroom contexts, listeners with different language backgrounds are tested: Some are familiar with the language and accent spoken, some are familiar with the language but less familiar with the accent, and others are less familiar with the language. Also reflecting different courtroom contexts: In one condition listeners make judgements based only on listening, and in another condition listeners make judgements based on both listening to the recordings and considering the likelihood-ratio values output by the forensic-voice-comparison system. [Abstract copyright: Copyright © 2022 The Author(s). Published by Elsevier B.V. All rights reserved.

Public attitudes towards the use of automatic facial recognition technology in criminal justice systems around the world

Automatic facial recognition technology (AFR) is increasingly used in criminal justice systems around the world, yet to date there has not been an international survey of public attitudes toward its use. In Study 1, we ran focus groups in the UK, Australia and China (countries at different stages of adopting AFR) and in Study 2 we collected data from over 3,000 participants in the UK, Australia and the USA using a questionnaire investigating attitudes towards AFR use in criminal justice systems. Our results showed that although overall participants were aligned in their attitudes and reasoning behind them, there were some key differences across countries. People in the USA were more accepting of tracking citizens, more accepting of private companies’ use of AFR, and less trusting of the police using AFR than people in the UK and Australia. Our results showed that support for the use of AFR depends greatly on what the technology is used for and who it is used by. We recommend vendors and users do more to explain AFR use, including details around accuracy and data protection. We also recommend that governments should set legal boundaries around the use of AFR in investigative and criminal justice settings

A comment on the PCAST report:skip the “match”/“non-match” stage

This letter comments on the report “Forensic science in criminal courts: Ensuring scientific validity of feature-comparison methods” recently released by the President's Council of Advisors on Science and Technology (PCAST). The report advocates a procedure for evaluation of forensic evidence that is a two-stage procedure in which the first stage is “match”/“non-match” and the second stage is empirical assessment of sensitivity (correct acceptance) and false alarm (false acceptance) rates. Almost always, quantitative data from feature-comparison methods are continuously-valued and have within-source variability. We explain why a two-stage procedure is not appropriate for this type of data, and recommend use of statistical procedures which are appropriate

Just Cognition: Scientific Research on Bias and Some Implications for Legal Procedure and Decision‐Making

Common law judges have traditionally been concerned about bias and the appearance of bias. Bias is believed to threaten the administration of justice and the legitimacy of legal decision‐making, particularly public confidence in the courts. This article contrasts legal approaches to bias with a range of biases, particularly cognitive biases, familiar to scientists who study human cognition and decision‐making. Research reveals that judges have narrowly conceived the biases that threaten legal decision‐making, insisting that some potential sources of bias are not open to review and that they are peculiarly resistant to bias through legal training and judicial experience. This article explains how, notwithstanding express concern with bias, there has been limited legal engagement with many risks known to actually bias decision‐making. Through examples, and drawing upon scientific research, it questions legal approaches and discusses the implications of more empirically‐based approaches to bias for decision making and institutional legitimacy

In what way are you qualified? Understanding epistemic (in)competence and expert persuasion through the courtroom of My Cousin Vinny

Has anyone seen the movie My Cousin Vinny? Expert opinion evidence is ubiquitous in civil and criminal justice procedures. Its use is longstanding, widespread and influential. However, non-expert factfinders have been criticized for their ineffective management of improper and unvalidated forensic science evidence. Some argue this mishandling arises from the epistemic incompetence of judges and juries. In this paper we use scriptwriter Dale Launer’s persuasive expert character (Mona Lisa Vito), from the film My Cousin Vinny, to explore the epistemic (in)competence of non-expert evaluators. When placed in the context of expert persuasion scholarship, this analysis reveals strengths and weaknesses of non-expert evaluations of expert witnesses. In particular, issues relating to the foundation of expert opinions, the certainty of expert conclusions, and tendency to stray outside ones area of expertise. These matters are examined as potential targets for interventions to improve the reception and handling of expert opinion evidence, as well as the fairness and rectitude of criminal justice procedures

In what way are you qualified? Understanding epistemic (in)competence and expert persuasion through the courtroom of My Cousin Vinny

Has anyone seen the movie My Cousin Vinny? Expert opinion evidence is ubiquitous in civil and criminal justice procedures. Its use is longstanding, widespread and influential. However, non-expert factfinders have been criticized for their ineffective management of improper and unvalidated forensic science evidence. Some argue this mishandling arises from the epistemic incompetence of judges and juries. In this paper we use scriptwriter Dale Launer’s persuasive expert character (Mona Lisa Vito), from the film My Cousin Vinny, to explore the epistemic (in)competence of non-expert evaluators. When placed in the context of expert persuasion scholarship, this analysis reveals strengths and weaknesses of non-expert evaluations of expert witnesses. In particular, issues relating to the foundation of expert opinions, the certainty of expert conclusions, and tendency to stray outside ones area of expertise. These matters are examined as potential targets for interventions to improve the reception and handling of expert opinion evidence, as well as the fairness and rectitude of criminal justice procedures