The legal and scientific challenge of black box expertise

Preprint DOI: dx.doi.org/10.31234/osf.io/qte4vLegal commentators widely agree that forensic examiners should articulate the reasons for their opinions. However, findings from cognitive science strongly suggest that people have little insight into the information they rely on to make decisions. And as individuals gain expertise, they rely more on cognitive shortcuts that are not directly accessible through introspection. That is to say, the expert’s mind is a black box — both to the expert and to the trier of fact. This article focuses on black box expertise in the context of forensic examiners who interpret visual pattern evidence (eg fingerprints). The authors review black box expertise through the lens of cognitive scientific research. They then suggest that the black box nature of this expertise strains common law admissibility rules and trial safeguards.Rachel A. Searston and Jason M. Chi

Collective intelligence in fingerprint analysis

When a fingerprint is located at a crime scene, a human examiner is counted upon to manually compare this print to those stored in a database. Several experiments have now shown that these professional analysts are highly accurate, but not infallible, much like other fields that involve high-stakes decision-making. One method to offset mistakes in these safety-critical domains is to distribute these important decisions to groups of raters who independently assess the same information. This redundancy in the system allows it to continue operating effectively even in the face of rare and random errors. Here, we extend this "wisdom of crowds" approach to fingerprint analysis by comparing the performance of individuals to crowds of professional analysts. We replicate the previous findings that individual experts greatly outperform individual novices, particularly in their false-positive rate, but they do make mistakes. When we pool the decisions of small groups of experts by selecting the decision of the majority, however, their false-positive rate decreases by up to 8% and their false-negative rate decreases by up to 12%. Pooling the decisions of novices results in a similar drop in false negatives, but increases their false-positive rate by up to 11%. Aggregating people's judgements by selecting the majority decision performs better than selecting the decision of the most confident or the most experienced rater. Our results show that combining independent judgements from small groups of fingerprint analysts can improve their performance and prevent these mistakes from entering courts.Jason M. Tangen, Kirsty M. Kent and Rachel A. Searsto

Releasing natural categories from visual crowding with meaning

Program of the 60th Annual Meeting published as part of Abstracts of the Psychonomic SocietyRachel A. Searston and Carly Sulliva

A multiple-task reduction approach to measuring perceptual expertise in fingerprint analysis

Presentation - General talks #12Searston R. A., Tangen, J. M., Thompson, M. B

Collective intelligence in perceptual decision making

Program of the 60th Annual Meeting published as part of Abstracts of the Psychonomic SocietyJason Tangen, Rachel A. Searston, Kirsty Ken

Specific versus varied practice in perceptual expertise training

Published December 2022We used a longitudinal randomized control experiment to compare the effect of specific practice (training on one form of a task) and varied practice (training on various forms of a task) on perceptual learning and transfer. Participants practiced a visual search task for 10 hours over 2 to 4 weeks. The specific practice group searched for features only in fingerprints during each session, whereas the varied practice group searched for features in five different image categories. Both groups were tested on a series of tasks at four time points: before training, midway through training, immediately after training ended, and 6 to 8 weeks later. The specific group improved more during training and demonstrated greater pre-post performance gains than the varied group on a visual search task with untrained fingerprint images. Both groups improved equally on a visual search task with an untrained image category, but only the specific group's performance dropped significantly when tested several weeks later. Finally, both groups improved equally on a series of untrained fingerprint tasks. Practice with respect to a single category (versus many) instills better near transfer, but category-specific and category-general visual search training appear equally effective for developing task-general expertise. (PsycInfo Database Record (c) 2022 APA, all rights reserved).Samuel G. Robson, Jason M. Tangen, Rachel A. Searsto

An evidence accumulation model of perceptual discrimination with naturalistic stimuli

Evidence accumulation models have been used to describe the cognitive processes underlying performance in tasks involving 2-choice decisions about unidimensional stimuli, such as motion or orientation. Given the multidimensionality of natural stimuli, however, we might expect qualitatively different patterns of evidence accumulation in more applied perceptual tasks. One domain that relies heavily on human decisions about complex natural stimuli is fingerprint discrimination. We know little about the ability of evidence accumulation models to account for the dynamic decision process of a fingerprint examiner resolving if 2 different prints belong to the same finger or different fingers. Here, we apply a dynamic decision-making model—the linear ballistic accumulator (LBA)—to fingerprint discrimination decisions to gain insight into the cognitive processes underlying these complex perceptual judgments. Across 3 experiments, we show that the LBA provides an accurate description of the fingerprint discrimination decision process with manipulations in visual noise, speed-accuracy emphasis, and training. Our results demonstrate that the LBA is a promising model for furthering our understanding of applied decision-making with naturally varying visual stimuli

Measuring and simulating human perceptual categorisation performance using Signal Detection Theory

Samuel Robson, Rachel Searston, Matthew Thompson, Brooklyn Corbett, Jason Tange

How low can you go? Detecting style in extremely low resolution images

Humans can see through the complexity of scenes, faces, and objects by quickly extracting their redundant low-spatial and low-dimensional global properties, or their style. It remains unclear, however, whether semantic coding is necessary, or whether visual stylistic information is sufficient, for people to recognize and discriminate complex images and categories. In two experiments, we systematically reduce the resolution of hundreds of unique paintings, birds, and faces, and test people's ability to discriminate and recognize them. We show that the stylistic information retained at extremely low image resolutions is sufficient for visual recognition of images and visual discrimination of categories. Averaging over the 3 domains, people were able to reliably recognize images reduced down to a single pixel, with large differences from chance discriminability across 8 different image resolutions. People were also able to discriminate categories substantially above chance with an image resolution as low as 2 × 2 pixels. We situate our findings in the context of contemporary computational accounts of visual recognition and contend that explicit encoding of the local features in the image, or knowledge of the semantic category, is not necessary for recognizing and distinguishing complex visual stimuli

The effect of fingerprint expertise on visual short-term memory

Brooklyn Corbett, Jason Tangen, Rachel Searston, Matthew Thompson, Samuel Robso