Traitement neuronal des voix et familiarité : entre reconnaissance et identification du locuteur

La capacité humaine de reconnaitre et d’identifier de nombreux individus uniquement grâce à leur voix est unique et peut s’avérer cruciale pour certaines enquêtes. La méconnaissance de cette capacité jette cependant de l’ombre sur les applications dites « légales » de la phonétique. Le travail de thèse présenté ici a comme objectif principal de mieux définir les différents processus liés au traitement des voix dans le cerveau et les paramètres affectant ce traitement. Dans une première expérience, les potentiels évoqués (PÉs) ont été utilisés pour démontrer que les voix intimement familières sont traitées différemment des voix inconnues, même si ces dernières sont fréquemment répétées. Cette expérience a également permis de mieux définir les notions de reconnaissance et d’identification de la voix et les processus qui leur sont associés (respectivement les composantes P2 et LPC). Aussi, une distinction importante entre la reconnaissance de voix intimement familières (P2) et inconnues, mais répétées (N250) a été observée. En plus d’apporter des clarifications terminologiques plus-que-nécessaires, cette première étude est la première à distinguer clairement la reconnaissance et l’identification de locuteurs en termes de PÉs. Cette contribution est majeure, tout particulièrement en ce qui a trait aux applications légales qu’elle recèle. Une seconde expérience s’est concentrée sur l’effet des modalités d’apprentissage sur l’identification de voix apprises. Plus spécifiquement, les PÉs ont été analysés suite à la présentation de voix apprises à l’aide des modalités auditive, audiovisuelle et audiovisuelle interactive. Si les mêmes composantes (P2 et LPC) ont été observées pour les trois conditions d’apprentissage, l’étendue de ces réponses variait. L’analyse des composantes impliquées a révélé un « effet d’ombrage du visage » (face overshadowing effect, FOE) tel qu’illustré par une réponse atténuée suite à la présentation de voix apprise à l’aide d’information audiovisuelle par rapport celles apprises avec dans la condition audio seulement. La simulation d’interaction à l’apprentissage à quant à elle provoqué une réponse plus importante sur la LPC en comparaison avec la condition audiovisuelle passive. De manière générale, les données rapportées dans les expériences 1 et 2 sont congruentes et indiquent que la P2 et la LPC sont des marqueurs fiables des processus de reconnaissance et d’identification de locuteurs. Les implications fondamentales et en phonétique légale seront discutées.The human ability to recognize and identify speakers by their voices is unique and can be critical in criminal investigations. However, the lack of knowledge on the working of this capacity overshadows its application in the field of “forensic phonetics”. The main objective of this thesis is to characterize the processing of voices in the human brain and the parameters that influence it. In a first experiment, event related potentials (ERPs) were used to establish that intimately familiar voices are processed differently from unknown voices, even when the latter are repeated. This experiment also served to establish a clear distinction between neural components of speaker recognition and identification supported by corresponding ERP components (respectively the P2 and the LPC). An essential contrast between the processes underlying the recognition of intimately familiar voices (P2) and that of unknown but previously heard voices (N250) was also observed. In addition to clarifying the terminology of voice processing, the first study in this thesis is the first to unambiguously distinguish between speaker recognition and identification in terms of ERPs. This contribution is major, especially when it comes to applications of voice processing in forensic phonetics. A second experiment focused more specifically on the effects of learning modalities on later speaker identification. ERPs to trained voices were analysed along with behavioral responses of speaker identification following a learning phase where participants were trained on voices in three modalities : audio only, audiovisual and audiovisual interactive. Although the ERP responses for the trained voices showed effects on the same components (P2 and LPC) across the three training conditions, the range of these responses varied. The analysis of these components first revealed a face overshadowing effect (FOE) resulting in an impaired encoding of voice information. This well documented effect resulted in a smaller LPC for the audiovisual condition compared to the audio only condition. However, effects of the audiovisual interactive condition appeared to minimize this FOE when compared to the passive audiovisual condition. Overall, the data presented in both experiments is generally congruent and indicate that the P2 and the LPC are reliable electrophysiological markers of speaker recognition and identification. The implications of these findings for current voice processing models and for the field of forensic phonetics are discussed

Psychological and neurophysiological correlates of social vocal control

Flexible vocal behaviour is an important paralinguistic signal in social interactions, but it is unclear whether such social vocal control can occur volitionally and specifically, and which speaker characteristics contribute to this ability. In this dissertation, the psychological and neurophysiological mechanisms involved in social vocal control are addressed using a novel social vocal control task, behavioural ratings and functional neuroimaging. Speakers evoked specific social trait impressions in listeners through targeted vocal modulations. While speakers’ self-reported level of Machiavellianism predicted the ability to evoke favourable trait percepts in listeners, dispositional empathy was not related to vocal control performance. Processing regions involved in vocomotor control and social cognitive processing interacted on the level of the left inferior frontal gyrus during social vocal control, while performance was associated with activation in somatosensory and social feedback regions. Social vocal control might be a route for impression management and support successful social interactions

An exploration into aphantasia: the inability to form voluntary mental imagery

Congenital aphantasia is a variation of the human experience, characterised by a life-long inability to generate voluntary mental imagery, and so far, has been examined in the visual domain. In a series of 10 experiments, this thesis took an experimental approach to examine the possible explanations of congenital aphantasia and the nature of the experience in the visual and non-visual domains, using objective measures and matched samples. Chapter 2 examines the findings of the early published studies within larger experimental designs, and the results confirmed self-reported differences in object imagery but not spatial imagery. Chapter 3 investigates whether aphantasia may be associated with differences in personality or deficits in broader cognitive functions. The results showed no evidence of a difference between individuals with aphantasia and neurotypical imagers on personality or selected neuropsychological measures. Aphantasic participants were slower in a task during trials that had greater working memory load, however, individual differences in performance were apparent and four aphantasic subgroups identified. Chapter 4 showed no difference in accuracy or response time in complex visuospatial working memory tasks requiring allocentric and egocentric transformations. However, aphantasic participants exhibited greater variability in their response times for front/back orientations within an egocentric task. In Chapter 5, a task that attempted to isolate ‘visual’ from spatial imagery showed no difference in performance across visual and spatial features. Nevertheless, self-reports of nonvisual sensory imagery showed variability in the range of imagery experience across the other senses. Chapter 6 extended exploration of mental imagery in aphantasia beyond the visual modality by examining behavioural performance in two auditory imagery tasks, in which no differences in accuracy or reaction time were evident. Taken together, this research shows that despite the differences in self-reported experience, limited group differences are found between aphantasic and neurotypical participants on a range of imagery and visuospatial working memory tasks. Nevertheless, individual differences in performance were apparent. Further research should investigate the processes adopted by these subgroups, or whether (or not) individuals with aphantasia have unconscious mental imagery

Challenges in analysis and processing of spontaneous speech

Selected and peer-reviewed papers of the workshop entitled Challenges in Analysis and Processing of Spontaneous Speech (Budapest, 2017

The role of phonology in visual word recognition: evidence from Chinese

Posters - Letter/Word Processing V: abstract no. 5024The hypothesis of bidirectional coupling of orthography and phonology predicts that phonology plays a role in visual word recognition, as observed in the effects of feedforward and feedback spelling to sound consistency on lexical decision. However, because orthography and phonology are closely related in alphabetic languages (homophones in alphabetic languages are usually orthographically similar), it is difficult to exclude an influence of orthography on phonological effects in visual word recognition. Chinese languages contain many written homophones that are orthographically dissimilar, allowing a test of the claim that phonological effects can be independent of orthographic similarity. We report a study of visual word recognition in Chinese based on a mega-analysis of lexical decision performance with 500 characters. The results from multiple regression analyses, after controlling for orthographic frequency, stroke number, and radical frequency, showed main effects of feedforward and feedback consistency, as well as interactions between these variables and phonological frequency and number of homophones. Implications of these results for resonance models of visual word recognition are discussed.postprin

Interactive effects of orthography and semantics in Chinese picture naming

Posters - Language Production/Writing: abstract no. 4035Picture-naming performance in English and Dutch is enhanced by presentation of a word that is similar in form to the picture name. However, it is unclear whether facilitation has an orthographic or a phonological locus. We investigated the loci of the facilitation effect in Cantonese Chinese speakers by manipulating—at three SOAs (2100, 0, and 1100 msec)—semantic, orthographic, and phonological similarity. We identified an effect of orthographic facilitation that was independent of and larger than phonological facilitation across all SOAs. Semantic interference was also found at SOAs of 2100 and 0 msec. Critically, an interaction of semantics and orthography was observed at an SOA of 1100 msec. This interaction suggests that independent effects of orthographic facilitation on picture naming are located either at the level of semantic processing or at the lemma level and are not due to the activation of picture name segments at the level of phonological retrieval.postprin

Attention Restraint, Working Memory Capacity, and Mind Wandering: Do Emotional Valence or Intentionality Matter?

Attention restraint appears to mediate the relationship between working memory capacity (WMC) and mind wandering (Kane et al., 2016). Prior work has identifed two dimensions of mind wandering—emotional valence and intentionality. However, less is known about how WMC and attention restraint correlate with these dimensions. Te current study examined the relationship between WMC, attention restraint, and mind wandering by emotional valence and intentionality. A confrmatory factor analysis demonstrated that WMC and attention restraint were strongly correlated, but only attention restraint was related to overall mind wandering, consistent with prior fndings. However, when examining the emotional valence of mind wandering, attention restraint and WMC were related to negatively and positively valenced, but not neutral, mind wandering. Attention restraint was also related to intentional but not unintentional mind wandering. Tese results suggest that WMC and attention restraint predict some, but not all, types of mind wandering

Torque #1: Mind, Language and Technology

A deliberately broad interdisciplinary collection of essays and artworks, many newly commissioned for the project, that reflect upon the plasticity of the brain, the adaptability of technology and the malleability of language, and the twisting together of these facets. The title for the book (and the subsequent publishing imprint started around it) refers to an esoteric concept by the neuroscientist Tim Crow, that the "cerebral torque", a quality of the human brain, is responsible for both language and schizophrenia. This term is detached from its clinical context to explore the wider connotations of technical aspects of language and technology. As the editors in the book's introduction (c. 1500 words) state: "We have appropriated and redirected 'torque', freeing it from the specifics of Crow's theory, but maintaining its essence as a twisting force with roots in the conjunction of the brain and language, and with the addition of a technological dimension - the twisting of wires, the torsion of the machine. In essence, Crow's use of 'cerebral torque' allows him to posit schizophrenia as 'the price homo sapiens pay for language'. Our use of the term also allows us to ask: what, then, is the price we pay for technology?" This conceit sets the stage for an interdisciplinary examination of how language, mind and technology intertwine in the contemporary sphere. The contributors and their work each offer unique models of navigating this territory, making their own artefacts, writing their own scripts, and the book collects these into a logical sequence. Important contributions to the book included the bringing together for the first time, Lambros Malafouris' concept of "expanded mind" from his cognitive archaeological research at Oxford, with the contemporary poetics work of such writers as Robert Shepard, and developing a narrative connection between the net art practices of Mez Breeze, with more recent theory on code and networks by Geoff Cox and Anna Munster, in exploring how these concepts reshape what language is and does in ways that resonate with contemporary art practices such as those of Lawrence Abu Hamdan and Cecile B Evans. As well as co-authoring the introduction, in his own chapter (c. 5000 words) Jones contributes a detailed examination of the relationship between "schizo-cultures" -- specifically the 1970s meeting of continental postmodernist thinkers and the American avant garde at a conference of this name (Lotringer 2013) -- and "convergence culture": the increasingly complex relationships formed by “media convergence, participatory culture, and collective intelligence” (Jenkins 2014). The book was pivotal for the Torque research project, setting its thematic parameters, and opening up a specific play of live, participatory and theoretical work that have been combined in subsequent editions in this series. This book was published by Link Editions, as a freely downloadable PDF, and launched at with a series of events at FACT in Liverpool, and with a performance event at Richmix in London. Additional public impact was developed with the Torque Triptych installation at FACT's Typemotion exhibition (2015) where the book was available as an electronic edition, a leather bound book, and as a speed reader; and the Being Read Being Reading Being Read and Reading Beings exhibition at Furtherfield, where the book was broadcast to the park, and presented as a text animation and sound work inside the gallery