Toward a Motor Theory of Sign Language Perception

Researches on signed languages still strongly dissociate lin- guistic issues related on phonological and phonetic aspects, and gesture studies for recognition and synthesis purposes. This paper focuses on the imbrication of motion and meaning for the analysis, synthesis and evaluation of sign language gestures. We discuss the relevance and interest of a motor theory of perception in sign language communication. According to this theory, we consider that linguistic knowledge is mapped on sensory-motor processes, and propose a methodology based on the principle of a synthesis-by-analysis approach, guided by an evaluation process that aims to validate some hypothesis and concepts of this theory. Examples from existing studies illustrate the di erent concepts and provide avenues for future work.Comment: 12 pages Partiellement financ\'e par le projet ANR SignCo

An embodied and grounded perspective on concepts

By the mainstream view in psychology and neuroscience, concepts are informational units, rather stable, and are represented in propositional format. In the view I will outline, instead, concepts correspond to patterns of activation of the perception, action and emotional systems which are typically activated when we interact with the entities they refer to. Starting from this embodied and grounded approach to concepts, I will focus on different research lines and present some experimental evidence concerning concepts of objects, concepts of actions, and abstract concepts. I will argue that, in order to account for abstract concepts, embodied and grounded theories should be extended

Odors, words and objects

The paper focuses on concepts and words referring to odors and to objects that have an odor. We argue that odors are an interesting object of study since they are evanescent, and since odor words do not refer to concrete and manipulable objects, but to scents evoked by objects. A second reason why odors are interesting is that some languages, as the Western ones, lack a specific odor lexicon, comparable in richness and variety to the color lexicon, and that performance on odors naming is typically worse than performance in color naming. In this work we discuss three main issues. First, we illustrate literature showing that, even if odor words do not have concrete referents, many languages encode them quite easily: the case of odors suggests that word meaning cannot be exhausted by the relationship with a referent, and highlights the importance of the social sharing of meaning. Second, we have discussed the peculiar status of odor concepts and words. Given their ambiguous status, their simple existence poses problems both to theories according to which concrete and abstract concepts do not differ, and to theories according to which they represent a dichotomy. Finally, we present an experiment in which we show that names of objects evoke their smell, and that these smells evoke approach and avoidance movements, in line with theories according to which words are grounded in both sensorial and motor systems

On The Linguistic Effects Of Articulatory Ease, With A Focus On Sign Languages

Spoken language has a well-known drive for ease of articulation, which Kirchner (1998, 2004) analyzes as reduction of the total magnitude of all biomechanical forces involved. We extend Kirchner\u27s insights from vocal articulation to manual articulation, with a focus on joint usage, and we discuss ways that articulatory ease might be realized in sign languages. In particular, moving more joints and/or joints more proximal to the torso results in greater mass being moved, and thus more articulatory force being expended, than moving fewer joints or moving more distal joints. We predict that in casual conversation, where articulatory ease is prized, moving fewer joints should be favored over moving more, and moving distal joints should be favored over moving proximal joints. We report on the results of our study of the casual signing of fluent signers of American Sign Language, which confirm our predictions: in comparison to citation forms of signs, the casual variants produced by the signers in our experiment exhibit an overall decrease in average joint usage, as well as a general preference for more distal articulation than is used in citation form. We conclude that all language, regardless of modality, is shaped by a fundamental drive for ease of articulation. Our work advances a cross-modality approach for considering ease of articulation, develops a potentially important vocabulary for describing variations in signs, and demonstrates that American Sign Language exhibits variation that can be accounted for in terms of ease of articulation. We further suggest that the linguistic drive for ease of articulation is part of a broader tendency for the human body to reduce biomechanical effort in all physical activities

Reenactment: An embodied cognition approach to meaning and linguistic content

A central finding in experimental research identified with Embodied Cognition (EC) is that understanding actions involves their embodied simulation, i.e. executing some processes involved in performing these actions. Extending these findings, I argue that reenactment – the overt embodied simulation of actions and practices, including especially communicative actions and practices, within utterances – makes it possible to forge an integrated EC-based account of linguistic meaning. In particular, I argue: (a) that remote entities can be referred to by reenacting actions performed with them; (b) that the use of grammatical constructions can be conceived of as the reenactment of linguistic action routines; (c) that complex enunciational structures (reported speech, irony, etc.) involve a separate level of reenactment, on which characters are presented as interacting with one another within the utterance; (d) that the segmentation of long utterances into shorter units involves the reenactment of brief audience interventions between units; and (e) that the overall meaning of an utterance can be stated in reenactment terms. The notion of reenactment provides a conceptual framework for accounting for aspects of language that are usually thought to be outside the reach of EC in an EC framework, thus supporting a view of meaning and linguistic content as thoroughly grounded in action and interaction

Symbol Emergence in Robotics: A Survey

Humans can learn the use of language through physical interaction with their environment and semiotic communication with other people. It is very important to obtain a computational understanding of how humans can form a symbol system and obtain semiotic skills through their autonomous mental development. Recently, many studies have been conducted on the construction of robotic systems and machine-learning methods that can learn the use of language through embodied multimodal interaction with their environment and other systems. Understanding human social interactions and developing a robot that can smoothly communicate with human users in the long term, requires an understanding of the dynamics of symbol systems and is crucially important. The embodied cognition and social interaction of participants gradually change a symbol system in a constructive manner. In this paper, we introduce a field of research called symbol emergence in robotics (SER). SER is a constructive approach towards an emergent symbol system. The emergent symbol system is socially self-organized through both semiotic communications and physical interactions with autonomous cognitive developmental agents, i.e., humans and developmental robots. Specifically, we describe some state-of-art research topics concerning SER, e.g., multimodal categorization, word discovery, and a double articulation analysis, that enable a robot to obtain words and their embodied meanings from raw sensory--motor information, including visual information, haptic information, auditory information, and acoustic speech signals, in a totally unsupervised manner. Finally, we suggest future directions of research in SER.Comment: submitted to Advanced Robotic

Constructing a concept of number

Numbers are concepts whose content, structure, and organization are influenced by the material forms used to represent and manipulate them. Indeed, as argued here, it is the inclusion of multiple forms (distributed objects, fingers, single- and two-dimensional forms like pebbles and abaci, and written notations) that is the mechanism of numerical elaboration. Further, variety in employed forms explains at least part of the synchronic and diachronic variability that exists between and within cultural number systems. Material forms also impart characteristics like linearity that may persist in the form of knowledge and behaviors, ultimately yielding numerical concepts that are irreducible to and functionally independent of any particular form. Material devices used to represent and manipulate numbers also interact with language in ways that reinforce or contrast different aspects of numerical cognition. Not only does this interaction potentially explain some of the unique aspects of numerical language, it suggests that the two are complementary but ultimately distinct means of accessing numerical intuitions and insights. The potential inclusion of materiality in contemporary research in numerical cognition is advocated, both for its explanatory power, as well as its influence on psychological, behavioral, and linguistic aspects of numerical cognition

From Introspection to Essence: The Auditory Nature of Inner Speech

To some it is a shallow platitude that inner speech always has an auditory-phonological component. To others, it is an empirical hypothesis with accumulating support. To yet others it is a false dogma. In this chapter, I defend the claim that inner speech always has an auditory-phonological component, confining the claim to adults with ordinary speech and hearing. It is one thing, I emphasize, to assert that inner speech often, or even typically, has an auditory-phonological component—quite another to propose that it always does. When forced to argue for the stronger point, we stand to make a number of interesting discoveries about inner speech itself, and about our means for discriminating it from other psycholinguistic phenomena. Establishing the stronger conclusion also provides new leverage on debates concerning how we should conceive of, diagnose, and explain auditory verbal hallucinations and “inserted thoughts” in schizophrenia

A Review of Verbal and Non-Verbal Human-Robot Interactive Communication

In this paper, an overview of human-robot interactive communication is presented, covering verbal as well as non-verbal aspects of human-robot interaction. Following a historical introduction, and motivation towards fluid human-robot communication, ten desiderata are proposed, which provide an organizational axis both of recent as well as of future research on human-robot communication. Then, the ten desiderata are examined in detail, culminating to a unifying discussion, and a forward-looking conclusion