Proficiency Tests to Estimate Error Rates in the Forensic Sciences

A proficiency test is an assessment of the performance of laboratory personnel using samples whose sources are known to the proficiency test administrator but unknown to the examinee. Proficiency tests can help identify reasonable first pass estimates for the rates at which various types of errors occur. It is crucial to obtain error rate estimates because the reliability and probative value of forensic science evidence is inextricably linked to the rates at which examiners make errors. Without such information, legal decision makers have no scientifically meaningful way of thinking about the risk of false identification and false non-identification associated with forensic reports

Comment: Experts who don\u27t know they don\u27t know

Sadly, the conclusion reached by Green and Armstrong (2006) – that experts should not be used for predicting the decisions that people will make in conflicts – comes as no surprise. Decades ago, Armstrong himself taught us that expertise beyond a minimal level does not improve judgmental accuracy across a variety of domains (Armstrong, 1980). More recently, Tetlock (2006) drove home the point in a study of hundreds of political experts who made thousands of forecasts over many years. Like Green and Armstrong (2006), Tetlock (2006) found that that expert forecasts were frequently inaccurate. In a nod to Armstrong\u27s previous work, Tetlock (2006) suggests that avid readers of the New York Times should be able to predict political events as well as highly trained experts

Out to Lunch: Saks & koehler Reply to Rudin & Imman\u27s Commentary

At several points in their comment on our article in Science (1), Rudin & Inman (2, 3) asserted or clearly implied that we had been dishonest in our presentation. In each of those instances Rudin & Inman\u27s charges are groundless, as we demonstrate below. Had Rudin & Inman examined the actual source [see Fig. 1, right], they would have discovered that the words were indeed those of Moenssens, that they were consistent with the context in which they appeared, that Moenssens was not quoting Zain or anyone else, and that Saks & Koehler had accurately attributed the statement to its author, Andre Moenssens

Questions about forensic science: Response

THE ESSENTIAL MESSAGE OF OUR REVIEW WAS that forensic individualization/identification science is on course for a paradigm shift in which its future will be more scientifically grounded than its past. Harmon and Budowle take issue with the simple point that traditional forensic science assumes that markings produced by different people and objects are observably different. The notion of uniqueness is widespread in forensic science writing, thinking, and practice. We added the qualifier discernible to the uniqueness assumption to indicate that criminalists do not refer to uniqueness in the abstract or as a metaphysical property. They mean that conclusions about object uniqueness are attainable in practice [(1), p. 45 and p. 123]

Comment: Experts who don\u27t know they don\u27t know

Sadly, the conclusion reached by Green and Armstrong (2006) – that experts should not be used for predicting the decisions that people will make in conflicts – comes as no surprise. Decades ago, Armstrong himself taught us that expertise beyond a minimal level does not improve judgmental accuracy across a variety of domains (Armstrong, 1980). More recently, Tetlock (2006) drove home the point in a study of hundreds of political experts who made thousands of forecasts over many years. Like Green and Armstrong (2006), Tetlock (2006) found that that expert forecasts were frequently inaccurate. In a nod to Armstrong\u27s previous work, Tetlock (2006) suggests that avid readers of the New York Times should be able to predict political events as well as highly trained experts

Questions about forensic science: Response

THE ESSENTIAL MESSAGE OF OUR REVIEW WAS that forensic individualization/identification science is on course for a paradigm shift in which its future will be more scientifically grounded than its past. Harmon and Budowle take issue with the simple point that traditional forensic science assumes that markings produced by different people and objects are observably different. The notion of uniqueness is widespread in forensic science writing, thinking, and practice. We added the qualifier discernible to the uniqueness assumption to indicate that criminalists do not refer to uniqueness in the abstract or as a metaphysical property. They mean that conclusions about object uniqueness are attainable in practice [(1), p. 45 and p. 123]

Proficiency Tests to Estimate Error Rates in the Forensic Sciences

A proficiency test is an assessment of the performance of laboratory personnel using samples whose sources are known to the proficiency test administrator but unknown to the examinee. Proficiency tests can help identify reasonable first pass estimates for the rates at which various types of errors occur. It is crucial to obtain error rate estimates because the reliability and probative value of forensic science evidence is inextricably linked to the rates at which examiners make errors. Without such information, legal decision makers have no scientifically meaningful way of thinking about the risk of false identification and false non-identification associated with forensic reports

Comments on Getting Scarred and Winning Lotteries: Effects of Exemplar Cuing and Statistical Format on Imagining Low-Probability Events, By Newell, Mitchell, and Hayes (2008)

Newell, Mitchell, and Hayes (NMH) conduct three experiments designed to test whether exemplar cuing (EC) theory or a statistical format theory provides a more accurate account for how people make judgments about low-probability events. They report finding support for the statistical format theory and little or no support for EC. However, NMH misstate the requirements for the production of exemplars in EC theory. As a result, they confuse non-exemplar conditions with exemplar conditions in their experiments, and find results that are virtually irrelevant to EC theory