L'apport des informations visuelles des gestes oro-faciaux dans le traitement phonologique des phonèmes natifs et non-natifs : approches comportementale, neurophysiologique

During audiovisual speech perception, like in face-to-face conversations, we can takeadvantage of the visual information conveyed by the speaker's oro-facial gestures. Thisenhances the intelligibility of the utterance. The aim of this work was to determine whetherthis “audiovisual benefit” can improve the identification of phonemes that do not exist in ourmother tongue. Our results revealed that the visual information contributes to overcome thephonological deafness phenomenon we experience in an audio only situation (Study 1). AnERP study indicates that this benefit could be due to the modulation of early processing in theprimary auditory cortex (Study 2). The audiovisual presentation of non native phonemesgenerates a P50 that is not observed for native phonemes. The linguistic background affectsthe way we use visual information. Early bilinguals take less advantage of the visual cuesduring the processing of unfamiliar phonemes (Study 3). We examined the identificationprocesses of native plosive consonants with a gating paradigm to evaluate the differentialcontribution of auditory and visual cues across time (Study 4). We observed that theaudiovisual benefit is not systematic. Phoneme predictability depends on the visual saliencyof the articulatory movements of the speaker.En situation de perception audiovisuelle de la parole, comme lors des conversations face-àface,nous pouvons tirer partie des informations visuelles fournies par les mouvements orofaciauxdu locuteur. Ceci améliore l’intelligibilité du discours. L'objectif de ce travail était dedéterminer si ce « bénéfice audiovisuel » permet de mieux identifier les phonèmes quin’existent pas dans notre langue. Nos résultats révèlent que l’utilisation de l’informationvisuelle permet de surmonter les difficultés posées par la surdité phonologique dont noussommes victimes lors d'une présentation auditive seule (Etude 1). Une étude EEG indique quel’apport des informations visuelles au processus d’identification de phonèmes non natifspourrait être dû à une modulation précoce des traitements effectués par le cortex auditifprimaire (Etude 2). En présentation audiovisuelle les phonèmes non natifs donnent lieu à uneP50, ce qui n’est pas observé pour les phonèmes natifs. Il semblerait également quel'expérience linguistique affecte l'utilisation des informations visuelles puisque des bilinguesprécoces semblent moins aptes à exploiter ces indices pour distinguer des phonèmes qui neleur sont pas familiers (Etude 3). Enfin, l’étude de l’identification de consonnes plosivesnatives avec une tâche de dévoilement progressif nous a permis d’évaluer la contributionconjointe et séparée des informations auditives et visuelles (Etude 4). Nous avons observé quel’apport de la modalité visuelle n’est pas systématique et que la prédictibilité de l’identité duphonème dépend de la saillance visuelle des mouvements articulatoires du locuteur

Facial Modelling and animation trends in the new millennium : a survey

M.Sc (Computer Science)Facial modelling and animation is considered one of the most challenging areas in the animation world. Since Parke and Waters’s (1996) comprehensive book, no major work encompassing the entire field of facial animation has been published. This thesis covers Parke and Waters’s work, while also providing a survey of the developments in the field since 1996. The thesis describes, analyses, and compares (where applicable) the existing techniques and practices used to produce the facial animation. Where applicable, the related techniques are grouped in the same chapter and described in a chronological fashion, outlining their differences, as well as their advantages and disadvantages. The thesis is concluded by exploratory work towards a talking head for Northern Sotho. Facial animation and lip synchronisation of a fragment of Northern Sotho is done by using software tools primarily designed for English.Computin

Human-Computer Interaction

In this book the reader will find a collection of 31 papers presenting different facets of Human Computer Interaction, the result of research projects and experiments as well as new approaches to design user interfaces. The book is organized according to the following main topics in a sequential order: new interaction paradigms, multimodality, usability studies on several interaction mechanisms, human factors, universal design and development methodologies and tools

Iconicity in Language and Speech

Die vorliegende Arbeit befasst sich mit dem großen Oberthema der Ikonizität und ihrer Verbreitung auf verschiedenen linguistischen Ebenen. Ikonizität bezeichnet die Ähnlichkeit zwischen der sprachlichen Form und ihrer Bedeutung (vgl. Perniss und Vigliocco, 2014). So wie eine Skulptur einem Objekt oder einer Person ähnelt, kann auch der Klang oder die Form von Wörtern der Sache ähneln, auf die sie verweisen. Frühere theoretische Ansätze betonen, dass die Arbitrarität von sprachlichen Zeichen und deren Bedeutung ein Hauptmerkmal menschlicher Sprache ist und Ikonizität für die Sprachevolution eine Rolle gespielt haben mag, jedoch in der heutigen Sprache zu vernachlässigen ist. Im Gegensatz dazu ist das Hauptanliegen dieser Arbeit, das Potenzial und die Bedeutung von Ikonizität in der heutigen Sprache zu untersuchen. Die einzelnen Kapitel der Dissertation können als separate Teile betrachtet werden, die in ihrer Gesamtheit das umfassende Spektrum der Ikonizität sichtbar machen. Von der sprachevolutionären Debatte ausgehend wird in den einzelnen Kapiteln auf die unterschiedlichen Ebenen der Ikonizität eingegangen. Es werden experimentelle Untersuchungen zur Lautsymbolik, am Beispiel der deutschen Pokémon-Namen, zur ikonischen Prosodie und zu ikonischen Wörtern, den sogenannten Ideophonen, vorgestellt. Die Ergebnisse der einzelnen Untersuchungen deuten auf die weite Verbreitung der Ikonizität im heutigen Deutsch hin. Darüber hinaus entschlüsselt diese Dissertation das kommunikative Potenzial der Ikonizität als eine Kraft, die nicht nur die Entstehung der Sprache ermöglichte, sondern auch nach Jahrtausenden bestehen bleibt, sich immer wieder neu entfaltet und uns tagtäglich in mündlicher, schriftlicher Form und in Gesten begegnet.This dissertation is concerned with the major theme of iconicity and its prevalence on different linguistic levels. Iconicity refers to a resemblance between the linguistic form and the meaning of a referent (cf. Perniss and Vigliocco, 2014). Just like a sculpture resembles an object or a model, so can the sound or shape of words resemble the thing they refer to. Previous theoretical approaches emphasize that arbitrariness of the linguistic sign is one of the main features of human language; iconicity, however, may have played a role for language evolution, but is negligible in contemporary language. In contrast, the main point of this thesis is to explore the potential and the importance of iconicity in the language nowadays. The individual chapters of the dissertation can be viewed as separate parts that, taken together, reveal the comprehensive spectrum of iconicity. Starting from the language evolutionary debate, the individual chapters address iconicity on different linguistic levels. I present experimental evidence on sound symbolism, using the example of German Pokémon names, on iconic prosody, and on iconic words, the so-called ideophones. The results of the individual investigations point to the widespread use of iconicity in contemporary German. Moreover, this dissertation deciphers the communicative potential of iconicity as a force that not only enabled the emergence of language, but also persists after millennia, unfolding again and again and encountering us every day in speech, writing, and gestures

Perceiving Sociable Technology: Exploring the Role of Anthropomorphism and Agency Perception on Human-Computer Interaction (HCI)

With the arrival of personal assistants and other AI-enabled autonomous technologies, social interactions with smart devices have become a part of our daily lives. Therefore, it becomes increasingly important to understand how these social interactions emerge, and why users appear to be influenced by them. For this reason, I explore questions on what the antecedents and consequences of this phenomenon, known as anthropomorphism, are as described in the extant literature from fields ranging from information systems to social neuroscience. I critically analyze those empirical studies directly measuring anthropomorphism and those referring to it without a corresponding measurement. Through a grounded theory approach, I identify common themes and use them to develop models for the antecedents and consequences of anthropomorphism. The results suggest anthropomorphism possesses both conscious and non-conscious components with varying implications. While conscious attributions are shown to vary based on individual differences, non-conscious attributions emerge whenever a technology exhibits apparent reasoning such as through non-verbal behavior like peer-to-peer mirroring or verbal paralinguistic and backchanneling cues. Anthropomorphism has been shown to affect users’ self-perceptions, perceptions of the technology, how users interact with the technology, and the users’ performance. Examples include changes in a users’ trust on the technology, conformity effects, bonding, and displays of empathy. I argue these effects emerge from changes in users’ perceived agency, and their self- and social- identity similarly to interactions between humans. Afterwards, I critically examine current theories on anthropomorphism and present propositions about its nature based on the results of the empirical literature. Subsequently, I introduce a two-factor model of anthropomorphism that proposes how an individual anthropomorphizes a technology is dependent on how the technology was initially perceived (top-down and rational or bottom-up and automatic), and whether it exhibits a capacity for agency or experience. I propose that where a technology lays along this spectrum determines how individuals relates to it, creating shared agency effects, or changing the users’ social identity. For this reason, anthropomorphism is a powerful tool that can be leveraged to support future interactions with smart technologies