Effects of forensically-relevant facial concealment on acoustic and perceptual properties of consonants

This thesis offers a thorough investigation into the effects of forensically-relevant facial concealment on speech acoustics and perception. Specifically, it explores the extent to which selected acoustic-phonetic and auditory-perceptual properties of consonants are affected when the talker is wearing ‘facewear’ while speaking. In this context, the term ‘facewear’ refers to the various types of face-concealing garments and headgear that are worn by people in common daily communication situations; for work and leisure, or as an expression of religious, social and cultural affiliation (e.g. surgical masks, motorcycle helmets, ski and cycling masks, or full-face veils such as the niqāb). It also denotes the face or head coverings that are typically used as deliberate (visual) disguises during the commission of crimes and in situations of public disorder (e.g. balaclavas, hooded sweatshirts, or scarves). The present research centres on the question: does facewear influence the way that consonants are produced, transmitted, and perceived? To examine the effects of facewear on the acoustic speech signal, various intensity, spectral, and temporal properties of spoken English consonants were measured. It was found that facewear can considerably alter the acoustic-phonetic characteristics of consonants. This was likely to be the result of both deliberate and involuntary changes to the talker’s speech productions, and of sound energy absorption by the facewear material. The perceptual consequences of the acoustic modifications to speech were assessed by way of a consonant identification study and a talker discrimination study. The results of these studies showed that auditory-only and auditory-visual consonant intelligibility, as well as the discrimination of unfamiliar talkers, may be greatly compromised when the observer’s judgements are based on ‘facewear speech’. The findings reported in this thesis contribute to our understanding of how auditory and visual information interact during natural speech processing. Furthermore, the results have important practical implications for legal cases in which speech produced through facewear is of pivotal importance. Forensic speech scientists are therefore advised to take the possible effects of facewear on speech into account when interpreting the outcome of their acoustic and auditory analyses of evidential speech recordings, and when evaluating the reliability of earwitness testimony

Hyper-realistic face masks : a new challenge in person identification

We often identify people using face images. This is true in occupational settings such as passport control as well as in everyday social environments. Mapping between images and identities assumes that facial appearance is stable within certain bounds. For example, a person's apparent age, gender and ethnicity change slowly, if at all. It also assumes that deliberate changes beyond these bounds (i.e., disguises) would be easy to spot. Hyper-realistic face masks overturn these assumptions by allowing the wearer to look like an entirely different person. If unnoticed, these masks break the link between facial appearance and personal identity, with clear implications for applied face recognition. However, to date, no one has assessed the realism of these masks, or specified conditions under which they may be accepted as real faces. Herein, we examined incidental detection of unexpected but attended hyper-realistic masks in both photographic and live presentations. Experiment 1 (UK; n = 60) revealed no evidence for overt detection of hyper-realistic masks among real face photos, and little evidence of covert detection. Experiment 2 (Japan; n = 60) extended these findings to different masks, mask-wearers and participant pools. In Experiment 3 (UK and Japan; n = 407), passers-by failed to notice that a live confederate was wearing a hyper-realistic mask and showed limited evidence of covert detection, even at close viewing distance (5 vs. 20 m). Across all of these studies, viewers accepted hyper-realistic masks as real faces. Specific countermeasures will be required if detection rates are to be improved

Perception auditive des effets de différents masques anti-COVID sur la parole conversationnelle, déclamée et le chant : étude de cas

Quels sont les effets perceptifs, en environnement calme, du port de six types de masques anti-COVID sur l'identification et la discrimination de consonnes, celle de phrases parlées, déclamées et chantées en français produits par un homme et une femme ? 21 auditeurs ont identifié puis discriminé les consonnes /p, t, k, b, d, g, f, s, ʃ, v, z, ʒ/ en intervocalique produites par la locutrice, puis 39 auditeurs ont discriminé une phrase parlée, déclamée (par un locuteur et une locutrice) et chantée (par la locutrice, chanteuse) avec et sans masque : l'identification consonantique est peu affectée, excepté pour /b/ avec le masque à fenêtre transparente, mais la présence des masques à fenêtre transparente et FFP2 est fortement discriminée et avec certitude par rapport à la condition sans masque, à l'exception du chant qui semble peu sensible à l'atténuation de timbre entendue et verbalisée par les auditeurs pour les autres tâches.Quins són els efectes perceptius, en un ambient no sorollós, de portar sis tipus de mascareta anti-COVID en la identificació i la discriminació de consonants, d'enunciats orals, declamats i cantats en francès produïts per un home i una dona? 21 informants van identificar i després discriminar les consonants /p, t, k, b, d, g, f, s, ʃ, v, z, ʒ/ en posició intervocàlica produïdes per la locutora, després 39 informants van discriminar un enunciat parlat, declamat (per un locutor i una locutora) i cantat (per la locutora, que era cantant) amb mascareta i sense: la identificació consonàntica gairebé no es va veure afectada, tret del cas de /b/ amb la mascareta transparent, però la utilització de mascaretes transparents i FFP2 certament està molt discriminada respecte a l'absència de mascaretes, lleva del cas del cant que sembla poc sensible a l'atenuació del timbre percebuda i verbalitzada pels informants per a les altres tasques.What are the perceptual effects, in a quiet environment, of wearing six types of anti-COVID masks on the identification and discrimination of consonants, and on discrimination in spoken, declaimed and sung French sentences produced by one man and one woman? 21 listeners identified then discriminated the consonants /p, t, k, b, d, g, f, s, ʃ, v, z, ʒ/ in intervocalic position, produced by the female speaker. 39 listeners discriminated a spoken, declaimed (produced by a male and a female speaker) and sung (produced by the female speaker, a singer) sentence with and without mask: consonant identification was little affected, except for /b/ with the transparent window mask, but the presence of the transparent window and FFP2 masks was strongly discriminated with certainty compared to the no-mask condition, with the exception of singing, which appeared to be insensitive to the timbre attenuation heard and verbalized by the listeners for the other tasks.¿Cuáles son los efectos perceptivos, en un ambiente no ruidoso, de llevar seis tipos de mascarilla anti-COVID en la identificación y la discriminación de consonantes, de enunciados orales, declamados y cantados en francés producidos por un hombre y una mujer? 21 informantes identificaron y luego discriminaron las consonantes /p, t, k, b, d, g, f, s, ʃ, v, z, ʒ/ en posición intervocálica producidas por la locutora, luego 39 informantes discriminaron un enunciado hablado, declamado (por un locutor y una locutora) y cantado (por la locutora, que era cantante) con mascarilla y sin ella: la identificación consonántica apenas se vio afectada, salvo para /b/ con la mascarilla transparente, pero la utilización de mascarillas transparentes y FFP2 ciertamente está muy discriminada con respecto a la ausencia de mascarillas, salvo en el caso del canto que parece poco sensible a la atenuación del timbre percibida y verbalizada por los informantes para las demás tareas

The Effect Of Acoustic Variability On Automatic Speaker Recognition Systems

This thesis examines the influence of acoustic variability on automatic speaker recognition systems (ASRs) with three aims. i. To measure ASR performance under 5 commonly encountered acoustic conditions; ii. To contribute towards ASR system development with the provision of new research data; iii. To assess ASR suitability for forensic speaker comparison (FSC) application and investigative/pre-forensic use. The thesis begins with a literature review and explanation of relevant technical terms. Five categories of research experiments then examine ASR performance, reflective of conditions influencing speech quantity (inhibitors) and speech quality (contaminants), acknowledging quality often influences quantity. Experiments pertain to: net speech duration, signal to noise ratio (SNR), reverberation, frequency bandwidth and transcoding (codecs). The ASR system is placed under scrutiny with examination of settings and optimum conditions (e.g. matched/unmatched test audio and speaker models). Output is examined in relation to baseline performance and metrics assist in informing if ASRs should be applied to suboptimal audio recordings. Results indicate that modern ASRs are relatively resilient to low and moderate levels of the acoustic contaminants and inhibitors examined, whilst remaining sensitive to higher levels. The thesis provides discussion on issues such as the complexity and fragility of the speech signal path, speaker variability, difficulty in measuring conditions and mitigation (thresholds and settings). The application of ASRs to casework is discussed with recommendations, acknowledging the different modes of operation (e.g. investigative usage) and current UK limitations regarding presenting ASR output as evidence in criminal trials. In summary, and in the context of acoustic variability, the thesis recommends that ASRs could be applied to pre-forensic cases, accepting extraneous issues endure which require governance such as validation of method (ASR standardisation) and population data selection. However, ASRs remain unsuitable for broad forensic application with many acoustic conditions causing irrecoverable speech data loss contributing to high error rates

Face perception and hyper-realistic masks

Previous research has shown that deliberate disguise deteriorates human and automatic face recognition, with consequences for person identification in criminal situations. Common forms of deliberate disguise (e.g. balaclavas or hoodies) are easy to detect. When such disguises are used, viewer can distinguish between an unmasked individual – whose identity they knowingly can observe from facial appearance – and a masked individual – whose identity they knowingly cannot. Hyper-realistic silicone masks change this. Their recent use in criminal settings suggests that they effectively disguise identity and are difficult to detect. In this thesis, I first show that viewers are strikingly poor at distinguishing hyper-realistic masks from real faces under live and photographic test conditions, and are worse in other-race conditions. I also show large individual differences in discriminating realistic masks from real faces (5%-100% accuracy), and use an image analysis to isolate information that high performers use for effective categorisation. The analysis reveals an informative region directly below the eyes, which is used by high performers but not low performers. These findings point to selection and training as routes to improved mask detection. Second, I examine the reliability of estimates made of the person beneath the mask. Demographic profiling and social character estimates are poor, and results show that recognition rates were only just above chance, even for familiar viewers. This analysis highlights a systematic bias in these estimates: demographics, traits and social characteristics of the mask were attributed to those of the wearer. This bias has theoretical and applied consequences. First, it supports the automaticity with which viewers use a face to judge a person, even when they know the face is not that of the person. Second, it suggests that predictions of the person underneath the mask, by familiar and unfamiliar viewers alike, should be treated with great caution

Investigating the phonetic and linguistic features used by speakers to communicate an intent to harm

This research aims to examine the phonetic and linguistic features which can be associated with a threatening intent. At present, there is a range of threat assessment resources and descriptions in legal cases which provide insight surrounding the content and production of threatening language. However, the veracity of these descriptions has not been thoroughly explored in empirical research. Through the examination of authentic and simulated threatening language data, this research provides a broad overview of the usage of phonetic and linguistic features to convey a threatening intent to harm. A set of 10 authentic speech recordings where a direct (or explicitly-worded) threat was present were analysed in relation to a sample of non-threatening speech. In addition, simulated threatening and non-threatening speech and texts were collected from 41 participants under experimental conditions. These threatening and non-threatening data were compared with respect to mean fundamental frequency, intensity, articulation rate and changes to vocal tract features and vocal settings. The simulated data were also examined for the use of lexical features which have previously been associated with the actualisation of harm. The results of these analyses suggest that there is no compelling evidence to support the assertion of a `threatening tone of voice'. There were, however, tendencies for these speakers to raise their mean fundamental frequency, intensity and articulation rate during threatening speech production relative to their non-threatening speech. There was also evidence to suggest that a number of lexical features used by these participants also corresponded to previous examinations of authentic threatening texts. It is suggested that on the basis of these findings, the production of threatening language is a considerably more complex and varied behaviour than might be expected. These findings have notable implications for the development of threat assessment tools, and for the description of a `threatening manner' in legal contexts

The effect of sampling variability on overall performance and individual speakers’ behaviour in likelihood ratio-based forensic voice comparison

In the past years, there is increasing awareness and acceptance among forensic speech scientists of using Bayesian reasoning and likelihood ratio (LR) framework for forensic voice comparison (FVC) and expressing expert conclusions. Numerous studies have explored overall performance using numerical LRs. Given that the data used for validation is a sample coming from an unknown distribution, little attention has been paid to the effect of sampling variability or individuals’ behaviour. This thesis investigates these issues using linguistic-phonetic variables. First, it investigates how different configurations of training, test and reference speakers affect overall performance. The results show that variability in overall performance is mostly caused by varying the test speakers, while less variability is caused by sampling variability in the reference and training speakers. Second, this thesis explores the effect of sampling variability on overall performance and individuals’ behaviour in relation to the use of linguistic-phonetic features. Results show that sampling variability affects overall performance to different extents using different features, while combining more features does not always improve overall performance. Sampling variability has limited effects on individuals in same-speaker comparisons, and most speakers are less affected by sampling variability in different-speaker comparisons when four or more features are used. Third, this thesis explores the effect of sampling variability on overall performance in relation to score distributions. Results reveal that system validity and reliability are more affected by different- speaker score skewness, and less affected by same-speaker score skewness. Using different calibration methods reduces the effect of sampling variability to different extents. The results in this thesis have implications for both FVC using numerical LRs and FVC in general, as experts need to make pragmatic decisions whether numerical LR is used or not, and every decision made has implication to final evaluation results. Further, the results on score skewness and different calibration methods have potential contribution for improving FVC performance using automatic systems

From urgent understanding to raw writing: the synergy of journalism and fiction in the novels of Sorj Chalandon

Sorj Chalandon (1952-) was previously an acclaimed foreign correspondent in Ireland, Lebanon, Iraq, Somalia and Afghanistan for the French centre-left daily newspaper Libération between 1973 and 2007. He has been a journalist for the satirical weekly Le Canard enchaîné since 2009. He is also a prize-winning author of ten novels, published between 2005 and 2021, the most recent of which was nominated for the Prix Goncourt. Yet despite these achievements, there is no extended academic inquiry into his writing. My thesis addresses this deficiency by examining the transition and interplay between his journalism and his fiction. Chalandon’s profound association with the visual emerges from his early cartoons in Libération and his productive collaborations with photographers. Although he emphasises the distinction between his reporting and his fiction –letting others speak and finding his own voice– I argue that his engagement with imagery constitutes a crucial bridge between them. My approach has entailed the textual, historical and cultural reading of his work through diverse but interconnected prisms. Given the markedly autobiographical genesis of most of his fiction, my research proceeds from a scrutiny of the self-image he presents to colleagues, other writers and to me in two written interviews. I interrogate his journalism through his empathy with his reporting subjects and his imperative to hear and impart their stories. This instinct is palpable in his ‘geopolitical’ novels set in Ireland and Lebanon, where he suffered considerable psychological trauma. The source of his most enduring anguish is his relationship with his father, which I pursue across four ‘paternal’ novels, culminating in a shocking dénouement. Finally, my analysis of his female first-person novel, emanating from the cancer diagnoses of himself and his wife, illuminates the progressive emancipation of his women characters