Eyewitness-Identification Evidence: Scientific Advances and the New Burden on Trial Judges

An increasingly strong case can be made for the argument that mistaken-eyewitness identification is the primary cause of the conviction of the innocent in the United States. The strongest single body of evidence in support of this proposition is the collection of cases in which forensic DNA testing was used to exonerate people who had been convicted by juries and were serving hard time (some on death row). These cases are well documented and tracked at the Innocence Project website and, as of this writing, there were 267 fully exonerated cases, of which 203 (76%) were cases involving mistaken-eyewitness identification

ROC analysis of lineups obscures information that is critical for both theoretical understanding and applied purposes

Our previous article (Wells et al., 2015a. Journal of Applied Research in Memory and Cognition) showed how ROC analysis of lineups does not measure underlying discriminability or control for response bias. Wixted and Mickes (2015. Journal of Applied Research in Memory and Cognition) concede these points. Hence, in this article we focus more on how forcing the 3. ×. 2 lineup into the 2. ×. 2 structure required for ROC analysis obscures important underlying phenomena of theoretical value. Moreover, ROC analysis fails to account for the unique diagnostic properties of exonerating eyewitness behaviors (filler identifications and rejections). We describe how an examination of the full 3. ×. 2 structure helps reveal the critical underlying phenomena that ROC analysis hides. We also show how a Bayesian approach yields a family of diagnosticity functions that exposes the unique diagnosticity of all three eyewitness behaviors (suspect identifications, filler identifications, and rejections). Moreover, we show how Bayesian methods can examine diagnosticity as a function of witness confidence for all three eyewitness behaviors, which gives it a significant applied advantage over ROC analysis

ROC analysis of lineups does not measure underlying discriminability and has limited value

Some researchers have been arguing that eyewitness identification data from lineups should be analyzed using Receiver Operating Characteristic (ROC) analysis because it purportedly measures underlying discriminability. But ROC analysis, which was designed for 2. ×. 2 tasks, does not fit the 3. ×. 2 structure of lineups. Accordingly, ROC proponents force lineup data into a 2. ×. 2 structure by treating false-positive identifications of lineup fillers as though they were rejections. Using data from lineups versus showups, we illustrate how this approach misfires as a measure of underlying discriminability. Moreover, treating false-positive identifications of fillers as if they were rejections hides one of the most important phenomena in eyewitness lineups, namely filler siphoning. Filler siphoning reduces the risk of mistaken identification by drawing false-positive identifications away from the innocent suspect and onto lineup fillers. We show that ROC analysis confuses filler siphoning with an improvement in underlying discriminability, thereby fostering misleading theoretical conclusions about how lineups work

Fair Lineups Improve Outside Observers' Discriminability, Not Eyewitnesses' Discriminability: Evidence for Differential Filler-Siphoning Using Empirical Data and the WITNESS Model

Fair lineups (good fillers) better sort between innocent and guilty suspect identifications than do biased lineups (poor fillers). Why are fair lineups better? Some argue that the fair-lineup advantage is an improvement in eyewitness discriminability through some mechanism such as diagnostic-feature detection. Others argue that the fair lineups do not improve eyewitnesses’ discriminability at all but instead improve the discriminability of outside observers who are privy to which lineups members are known-fillers (the differential filler-siphoning mechanism). Experiment 1 used a forced-choice paradigm to show that fair lineups do not improve eyewitness discriminability. The second experiment used the WITNESS model to show that differential filler-siphoning and the fair-lineup advantage readily surfaces and nicely patterns experimental data based on minimal assumptions even though witness memory strength was held constant. Together, these two experiments support differential filler-siphoning and the idea that fair lineups enhance the outside observer’s discriminability, not the eyewitness’s discriminability.This Unpublished paper published as Smith, Andrew & Smalarz, Laura & Wells, Gary & Lampinen, James & Mackovichova, Simona. (2020). Fair Lineups Improve Outside Observers' Discriminability, Not Eyewitnesses' Discriminability: Evidence for Differential Filler-Siphoning Using Empirical Data and the WITNESS Model. 10.13140/RG.2.2.23329.22887/1

Deviation from Perfect Performance Measures the Diagnostic Utility of Eyewitness Lineups but Partial Area Under the ROC Curve Does Not

When one lineup identification procedure leads to both fewer innocent–suspect identifications and fewer culprit identifications than does some other lineup procedure, it is difficult to determine whether the procedures differ in diagnostic accuracy. In an influential article, Wixted and Mickes (2012) argued that measures of probative value do not inform diagnostic accuracy in these situations but that the partial area under the receiver operator characteristic curve (pAUC) does. In more recent research, we have found that pAUC does not necessarily indicate which of two lineup procedures has higher expected utility. When two lineup procedures produce different innocent-suspect identification rates, it leads to differential truncation of the ROC curves. As a result, diagnostic utility as measured by the pAUC is confounded with witness confidence level. We introduce a novel receiver operator characteristic measure, deviation from perfect performance (DPP), that unconfounds diagnostic utility and witness confidence level and consistently indicates which of two lineup procedures has higher expected utility. Our findings suggest that eyewitness scientists should abandon pAUC as a measure of diagnostic accuracy and embrace deviation from perfect performance

SmithSupplementalMaterial – Supplemental material for Increasing the Similarity of Lineup Fillers to the Suspect Improves the Applied Value of Lineups Without Improving Memory Performance: Commentary on Colloff, Wade, and Strange (2016)

<p>Supplemental material, SmithSupplementalMaterial for Increasing the Similarity of Lineup Fillers to the Suspect Improves the Applied Value of Lineups Without Improving Memory Performance: Commentary on Colloff, Wade, and Strange (2016) by Andrew M. Smith, Gary L. Wells, Laura Smalarz and James Michael Lampinen in Psychological Science</p