Exploring the utility of giving robots auditory perspective-taking abilities

Presented at the 12th International Conference on Auditory Display (ICAD), London, UK, June 20-23, 2006.This paper reports on work in progress to develop a computational auditory perspective taking system for a robot. Auditory perspective taking is construed as the ability to reason about inferred or posited factors that affect an addressee's perspective as a listener for the purpose of presenting auditory information in an appropriate and effective manner. High-level aspects of this aural interaction skill are discussed, and a prototype adaptive auditory display, implemented in the context of a robotic information kiosk, is described and critiqued. Additionally, a sketch of the design and goals of a user study planned for later this year is given. A demonstration of the prototype system will accompany the presentation of this research in the poster session

The sound of social cognition: Toddlers’ understanding of how sound influences others

Understanding others’ perceptions is a fundamental aspect of social cognition. Children’s construal of visual perception is well investigated, but there is little work on children’s understanding of others’ auditory perception. The current study assesses toddlers’ recognition that producing different sounds can affect others differentially—auditory perspective taking. Two- and three-year-olds were familiarized with two objects, one loud and one quiet. The adult then introduced a doll, and children were randomly assigned to one of two goals: either to wake the doll or to let her sleep. Children’s object choice and the sound intensity they produced significantly varied in the predicted direction as a function of the goal task. These findings reveal young children’s understanding of the effects of sound on other people’s behavior and psychological states

Javan Gibbon (Hylobates moloch) Non-Vocal Social Communication and Gesture Use With Conspecifics

I explored gestures used by captive Javan gibbons (Hylobates moloch) at the Gibbon Conservation Center (Santa Clarita, CA). I hypothesized that a sender gibbon’s gesture modality would vary with the recipient gibbon’s attentional state and the sender would be equally likely to use all modalities (tactile, visual, actions, and facial expressions) when the recipient was attending (facing the sender), but would use more tactile gestures and actions when the recipient was non-attending (oriented away from the sender). I collected data from 10 individuals using all-occurrences sampling and an ethogram to score behaviors from video recordings. In 1,143 interactions, gibbons used visual gestures and facial expressions significantly more when the recipient was attending and tactile gestures significantly more when the recipient was non-attending. There was no significant difference in actions. These data show that juvenile Javan gibbons used gestures appropriate to recipient’s attentional state in three out of the four modalities

Designing Sound for Social Robots: Advancing Professional Practice through Design Principles

Sound is one of the core modalities social robots can use to communicate with the humans around them in rich, engaging, and effective ways. While a robot's auditory communication happens predominantly through speech, a growing body of work demonstrates the various ways non-verbal robot sound can affect humans, and researchers have begun to formulate design recommendations that encourage using the medium to its full potential. However, formal strategies for successful robot sound design have so far not emerged, current frameworks and principles are largely untested and no effort has been made to survey creative robot sound design practice. In this dissertation, I combine creative practice, expert interviews, and human-robot interaction studies to advance our understanding of how designers can best ideate, create, and implement robot sound. In a first step, I map out a design space that combines established sound design frameworks with insights from interviews with robot sound design experts. I then systematically traverse this space across three robot sound design explorations, investigating (i) the effect of artificial movement sound on how robots are perceived, (ii) the benefits of applying compositional theory to robot sound design, and (iii) the role and potential of spatially distributed robot sound. Finally, I implement the designs from prior chapters into humanoid robot Diamandini, and deploy it as a case study. Based on a synthesis of the data collection and design practice conducted across the thesis, I argue that the creation of robot sound is best guided by four design perspectives: fiction (sound as a means to convey a narrative), composition (sound as its own separate listening experience), plasticity (sound as something that can vary and adapt over time), and space (spatial distribution of sound as a separate communication channel). The conclusion of the thesis presents these four perspectives and proposes eleven design principles across them which are supported by detailed examples. This work contributes an extensive body of design principles, process models, and techniques providing researchers and designers with new tools to enrich the way robots communicate with humans

The expression of empathy in human's closest relatives, bonobos and chimpanzees: current and future directions

Empathy is a complex, multi‐dimensional capacity that facilitates the sharing and understanding of others' emotions. As our closest living relatives, bonobos (Pan paniscus) and chimpanzees (P. troglodytes) provide an opportunity to explore the origins of hominin social cognition, including empathy. Despite certain assumptions that bonobos and chimpanzees may differ empathically, these species appear to overlap considerably in certain socio‐emotional responses related to empathy. However, few studies have systematically tested for species variation in Pan empathic or socio‐emotional tendencies. To address this, we synthesise the growing literature on Pan empathy to inform our understanding of the selection pressures that may underlie the evolution of hominin empathy, and its expression in our last common ancestor. As bonobos and chimpanzees show overlaps in their expression of complex socio‐emotional phenomena such as empathy, we propose that group comparisons may be as or more meaningful than species comparisons when it comes to understanding the evolutionary pressures for such behaviour. Furthermore, key differences, such as how humans and Pan communicate, appear to distinguish how we experience empathy compared to our closest living relatives

Comment évaluer la théorie de l’esprit? Revue systématique des outils d’évaluation destinés aux enfants d’âge préscolaire

Essai doctoral présenté en vue de l’obtention du grade de doctorat en psychologie, option neuropsychologie clinique (D.Psy.)La Théorie de l’Esprit (TDE), soit l’habileté à inférer des états mentaux à soi-même et à autrui, est un domaine de recherche ralliant plusieurs disciplines, incluant la psychologie sociale et développementale, la neuropsychologie, les neurosciences sociales et l’orthophonie. Les habiletés de la TDE ont été maintes fois reliées à plusieurs marqueurs d’adaptation sociale, telles des compétences relationnelles et communicationnelles de meilleure qualité. Par ailleurs, la TDE est altérée dans le contexte de nombreuses conditions cliniques. Malgré l’énorme quantité d’études dédiées à la TDE, identifier des outils de mesures appropriés destinés aux enfants d’âge préscolaire demeure un défi. Cet essai a pour but de faciliter l’identification d’outils de mesures de la TDE pour les enfants de 0-5 ans en créant un inventaire de ceux-ci et de leurs caractéristiques. Une introduction positionne l’importance de la TDE à titre d’habileté sociocognitive, la définit et la distingue de construits socio-cognitifs apparentés, survole sa trajectoire développementale et soulève les défis reliés à son évaluation. Une revue systématique de la littérature, sous forme d’article scientifique, présente ensuite la méthodologie utilisée et l’inventaire des outils de mesures réalisé, et permet de souligner la grande variété d’outils évaluant la TDE, mais également de nombreux écueils méthodologiques et psychométriques associés à la création et au choix d’outils appropriés, incluant le nombre limité de sous-habiletés visées, le manque de standardisation et la pauvreté des informations psychométriques disponibles. Une discussion générale est ensuite fournie et relève les apports théoriques, méthodologiques et cliniques de cette recherche pour le domaine de la TDE.Theory of mind (TOM), the ability to infer mental states to self and others, has been a pervasive research theme across many disciplines including developmental, neuro-, and social psychology, social neuroscience and speech therapy. TOM abilities have been consistently linked to markers of social adaptation, such as better communication skills and quality social relationships, and are affected in a broad range of clinical conditions. Despite the wealth and breadth of research dedicated to TOM, identifying appropriate assessment tools for the preschool population remains challenging. This work aims to facilitate the choice and use of adequate measures for children aged 0 to 5 years by generating a comprehensive inventory of TOM measures and listing their characteristics. The introduction highlights the importance of TOM as a social-cognitive ability, defines TOM and distinguishes it from related yet distinct sociocognitive constructs, provides information on its developmental trajectory and raises challenges associated with TOM assessment. A systematic review of the literature is then presented in the form of an article and provides details on the methods used and the inventory of TOM measures generated. The remarkable variety of measures that have been created to assess TOM is highlighted, but also the numerous methodological and psychometric challenges associated with developing and choosing appropriate measures, including issues related to the limited range of sub-abilities targeted, lack of standardisation across studies and paucity of psychometric information provided. Finally, a general conclusion provides the opportunity to discuss the theoretical, methodological and clinical contributions of this project

Les primates non humains ont-ils une théorie de l’esprit ? =

L’expression « théorie de l’esprit » réfère à la capacité d’attribuer des états mentaux à soi-même et aux autres. Les états mentaux étant de différents types (intentions, connaissances, etc.), la théorie de l’esprit englobe plusieurs aptitudes. Celles-ci se retrouvent-elles chez d’autres animaux, notamment chez les primates non humains, ou sont-elles propres à notre espèce ? Cette question a fait l’objet de près de 40 ans de recherche et, en dépit de difficultés méthodologiques et de controverses persistantes, des avancées importantes ont été réalisées, en particulier depuis le début des années 2000. De sorte que l’on sait aujourd’hui que les grands singes adaptent leurs comportements aux buts, aux intentions, aux connaissances des autres, ainsi qu’à leur perspective visuelle et, avec apparemment plus de difficulté, leur perspective auditive. Des études récentes suggèrent que ces hominidés ont également une compréhension, certes implicite, des fausses croyances, une aptitude qui semblait jusqu’ici spécifique à l’humain. Or l’attribution de fausses croyances est l’élément fondamental de la théorie de l’esprit, car elle nécessite de concevoir que l’état mental d’un individu peut ne pas correspondre à la réalité, et donc que les états mentaux sont des représentations que l’on se fait du monde, plutôt que des reflets fidèles de celui-ci. De futures investigations devraient permettre d’expliquer la divergence entre les résultats de ces études récentes et ceux des études antérieures, mais aussi de mieux comprendre la nature des difficultés rencontrées par les grands singes en ce qui a trait aux états mentaux non congruents avec la réalité. En dépit d’efforts notables, la recherche tend encore à se concentrer sur ces espèces d’hominidés, tout particulièrement sur les chimpanzés, si bien que l’on en sait encore relativement peu sur les compétences des autres primates. La compréhension des perspectives visuelle et auditive, ainsi que celle du lien entre ces modalités sensorielles et l’acquisition d’informations ont toutefois été mises en évidence chez des singes de l’Ancien Monde, et l’attribution de buts et d’intentions s’étendrait même à des espèces du Nouveau Monde. Ainsi, ce qui semblait être une différence de nature entre l’humain et les autres primates serait davantage une différence de degré.Theory of mind, the ability to attribute mental states to oneself and to others, is a multifaceted phenomenon as there are different kinds of mental states (intention, knowledge, belief, etc.). Is theory of mind present in other animals, particularly in nonhuman primates, or is it unique to humans? This question has been investigated for almost 40 years and, despite some methodological difficulties and controversies (e.g. mindreading vs. behavior-reading), important advances have been made since the early 2000s. In fact, we now know that great apes behave according to others’ goals, intentions, visual perspective, and knowledge, and, to a lesser extent it seems, to others’ auditory perspective. Furthermore, recent studies suggest that these hominids also understand false beliefs, albeit implicitly, a capacity that was previously thought to be found only in humans. Understanding false-belief is central to theory of mind as it requires one’s to realize that mental states are internal representations rather than direct reflections of reality, and can therefore be inaccurate. Future investigations should explain the discrepancies between these recent studies and previous ones which reported negative results, and should also clarify the nature of the difficulties experienced by great apes when it comes to reality-incongruent mental states. Although noteworthy efforts have been made, research is still mostly centered around great apes, especially chimpanzees, so much so that we still know very little about theory of mind abilities in other primates. Visual and auditory perspective-taking have nevertheless been reported in some Old World monkeys, as well as their understanding that knowledge is acquired through these sensory modalities. Some work has also shown ability to attribute goals and intentions to others in New World monkeys. Thus, what seemed to be a difference of kind between humans and other primates now appears to be mainly one of degree

Blind children's understanding of vision

This thesis explores the effects of congenital blindness upon the development of understanding of vision, and draws on the theoretical frameworks of understanding of aspectuality and visual perspective-taking. Six studies investigated how blind children demonstrated their understanding of vision. Where appropriate, their performance was compared to a control group of sighted children. The views of parents and educators of congenitally blind children were sought in order to gain insight into social influences upon the development of blind children’s understanding of vision. Sixteen congenitally blind children aged from 3;6 to 14;8, 168 typically developing sighted children aged from 3;6 to 14;8 and 58 sighted adults took part in the studies. There were three main findings. First, the associative stage in understanding of the aspectuality of knowledge for blind children may be manifest in the relationship between touching and knowing. Second, when utilising a more naturalistic setting than has commonly been used, blind children aged between three and 11 years were able to hide successfully, raising questions about the position that Level 1 perspective-taking is not present in blind children until the age of at least six or seven years, and possibly not until ten years. Third, blind participants demonstrated an understanding of mind earlier than has been found in other studies, suggesting that the development of theory of mind in congenitally blind children may not be as delayed as was previously thought. Several educational issues were raised, including the manner in which blind children are taught about vision, and their placement in ability-groups in mainstream classrooms