Shared Perception in Human-Robot Interaction

Interaction can be seen as a composition of perspectives: the integration of perceptions, intentions, and actions on the environment two or more agents share. For an interaction to be effective, each agent must be prone to “sharedness”: being situated in a common environment, able to read what others express about their perspective, and ready to adjust one’s own perspective accordingly. In this sense, effective interaction is supported by perceiving the environment jointly with others, a capability that in this research is called Shared Perception. Nonetheless, perception is a complex process that brings the observer receiving sensory inputs from the external world and interpreting them based on its own, previous experiences, predictions, and intentions. In addition, social interaction itself contributes to shaping what is perceived: others’ attention, perspective, actions, and internal states may also be incorporated into perception. Thus, Shared perception reflects the observer's ability to integrate these three sources of information: the environment, the self, and other agents. If Shared Perception is essential among humans, it is equally crucial for interaction with robots, which need social and cognitive abilities to interact with humans naturally and successfully. This research deals with Shared Perception within the context of Social Human-Robot Interaction (HRI) and involves an interdisciplinary approach. The two general axes of the thesis are the investigation of human perception while interacting with robots and the modeling of robot’s perception while interacting with humans. Such two directions are outlined through three specific Research Objectives, whose achievements represent the contribution of this work. i) The formulation of a theoretical framework of Shared Perception in HRI valid for interpreting and developing different socio-perceptual mechanisms and abilities. ii) The investigation of Shared Perception in humans focusing on the perceptual mechanism of Context Dependency, and therefore exploring how social interaction affects the use of previous experience in human spatial perception. iii) The implementation of a deep-learning model for Addressee Estimation to foster robots’ socio-perceptual skills through the awareness of others’ behavior, as suggested in the Shared Perception framework. To achieve the first Research Objective, several human socio-perceptual mechanisms are presented and interpreted in a unified account. This exposition parallels mechanisms elicited by interaction with humans and humanoid robots and aims to build a framework valid to investigate human perception in the context of HRI. Based on the thought of D. Davidson and conceived as the integration of information coming from the environment, the self, and other agents, the idea of "triangulation" expresses the critical dynamics of Shared Perception. Also, it is proposed as the functional structure to support the implementation of socio-perceptual skills in robots. This general framework serves as a reference to fulfill the other two Research Objectives, which explore specific aspects of Shared Perception. For what concerns the second Research Objective, the human perceptual mechanism of Context Dependency is investigated, for the first time, within social interaction. Human perception is based on unconscious inference, where sensory inputs integrate with prior information. This phenomenon helps in facing the uncertainty of the external world with predictions built upon previous experience. To investigate the effect of social interaction on such a mechanism, the iCub robot has been used as an experimental tool to create an interactive scenario with a controlled setting. A user study based on psychophysical methods, Bayesian modeling, and a neural network analysis of human results demonstrated that social interaction influenced Context Dependency so that when interacting with a social agent, humans rely less on their internal models and more on external stimuli. Such results are framed in Shared Perception and contribute to revealing the integration dynamics of the three sources of Shared Perception. The others’ presence and social behavior (other agents) affect the balance between sensory inputs (environment) and personal history (self) in favor of the information shared with others, that is, the environment. The third Research Objective consists of tackling the Addressee Estimation problem, i.e., understanding to whom a speaker is talking, to improve the iCub social behavior in multi-party interactions. Addressee Estimation can be considered a Shared Perception ability because it is achieved by using sensory information from the environment, internal representations of the agents’ position, and, more importantly, the understanding of others’ behavior. An architecture for Addressee Estimation is thus designed considering the integration process of Shared Perception (environment, self, other agents) and partially implemented with respect to the third element: the awareness of others’ behavior. To achieve this, a hybrid deep-learning (CNN+LSTM) model is developed to estimate the speaker-robot relative placement of the addressee based on the non-verbal behavior of the speaker. Addressee Estimation abilities based on Shared Perception dynamics are aimed at improving multi-party HRI. Making robots aware of other agents’ behavior towards the environment is the first crucial step for incorporating such information into the robot’s perception and modeling Shared Perception

Being-in-the-world-with: Presence Meets Social And Cognitive Neuroscience

In this chapter we will discuss the concepts of “presence” (Inner Presence) and “social presence” (Co-presence) within a cognitive and ecological perspective. Specifically, we claim that the concepts of “presence” and “social presence” are the possible links between self, action, communication and culture. In the first section we will provide a capsule view of Heidegger’s work by examining the two main features of the Heideggerian concept of “being”: spatiality and “being with”. We argue that different visions from social and cognitive sciences – Situated Cognition, Embodied Cognition, Enactive Approach, Situated Simulation, Covert Imitation - and discoveries from neuroscience – Mirror and Canonical Neurons - have many contact points with this view. In particular, these data suggest that our conceptual system dynamically produces contextualized representations (simulations) that support grounded action in different situations. This is allowed by a common coding – the motor code – shared by perception, action and concepts. This common coding also allows the subject for natively recognizing actions done by other selves within the phenomenological contents. In this picture we argue that the role of presence and social presence is to allow the process of self-identification through the separation between “self” and “other,” and between “internal” and “external”. Finally, implications of this position for communication and media studies are discussed by way of conclusion

The direct perception hypothesis: perceiving the intention of another’s action hinders its precise imitation

We argue that imitation is a learning response to unintelligible actions, especially to social conventions. Various strands of evidence are converging on this conclusion, but further progress has been hampered by an outdated theory of perceptual experience. Comparative psychology continues to be premised on the doctrine that humans and nonhuman primates only perceive others’ physical ‘surface behavior’, while mental states are perceptually inaccessible. However, a growing consensus in social cognition research accepts the Direct Perception Hypothesis: primarily we see what others aim to do; we do not infer it from their motions. Indeed, physical details are overlooked – unless the action is unintelligible. On this basis we hypothesize that apes’ propensity to copy the goal of an action, rather than its precise means, is largely dependent on its perceived intelligibility. Conversely, children copy means more often than adults and apes because, uniquely, much adult human behavior is completely unintelligible to unenculturated observers due to the pervasiveness of arbitrary social conventions, as exemplified by customs, rituals, and languages. We expect the propensity to imitate to be inversely correlated with the familiarity of cultural practices, as indexed by age and/or socio-cultural competence. The Direct Perception Hypothesis thereby helps to parsimoniously explain the most important findings of imitation research, including children’s over-imitation and other species-typical and age-related variations

Gender and the senses of agency

This paper details the ways that gender structures our senses of agency on an enactive framework. While it is common to discuss how gender influences higher, narrative levels of cognition, as with the formulation of goals and in considerations about our identities, it is less clear how gender structures our more immediate, embodied processes, such as the minimal sense of agency. While enactivists often acknowledge that gender and other aspects of our socio-cultural situatedness shape our cognitive processes, there is little work on how this shaping takes place. In order to provide such an account, I will first look at the minimal and narrative senses of agency (Gallagher 2012), a distinction that draws from work on minimal and narrative selves (Zahavi 2010). Next I will explain the influence of the narrative sense of agency on the minimal sense of agency through work on intention-formation (Pacherie 2007). After a discussion of the role of gender in the narrative sense of agency, I’ll expand on work by Haslanger (2012) and Young (1980) to offer three ways in which gender influences the minimal sense of agency, showing the effect that gender has on how we perceive our possibilities for interaction in a phenomenologically immediate, pre-reflective manner

Distal engagement: Intentions in perception

Non-representational approaches to cognition have struggled to provide accounts of long-term planning that forgo the use of representations. An explanation comes easier for cognitivist accounts, which hold that we concoct and use contentful mental representations as guides to coordinate a series of actions towards an end state. One non-representational approach, ecological-enactivism, has recently seen several proposals that account for “high-level” or “representation-hungry” capacities, including long-term planning and action coordination. In this paper, we demonstrate the explanatory gap in these accounts that stems from avoiding the incorporation of long-term intentions, as they play an important role both in action coordination and perception on the ecological account. Using recent enactive accounts of language, we argue for a non-representational conception of intentions, their formation, and their role in coordinating pre-reflective action. We provide an account for the coordination of our present actions towards a distant goal, a skill we call distal engagement. Rather than positing intentions as an actual cognitive entity in need of explanation, we argue that we take them up in this way as a practice due to linguistically scaffolded attitudes towards language use

An integrated theory of language production and comprehension

Currently, production and comprehension are regarded as quite distinct in accounts of language processing. In rejecting this dichotomy, we instead assert that producing and understanding are interwoven, and that this interweaving is what enables people to predict themselves and each other. We start by noting that production and comprehension are forms of action and action perception. We then consider the evidence for interweaving in action, action perception, and joint action, and explain such evidence in terms of prediction. Specifically, we assume that actors construct forward models of their actions before they execute those actions, and that perceivers of others' actions covertly imitate those actions, then construct forward models of those actions. We use these accounts of action, action perception, and joint action to develop accounts of production, comprehension, and interactive language. Importantly, they incorporate well-defined levels of linguistic representation (such as semantics, syntax, and phonology). We show (a) how speakers and comprehenders use covert imitation and forward modeling to make predictions at these levels of representation, (b) how they interweave production and comprehension processes, and (c) how they use these predictions to monitor the upcoming utterances. We show how these accounts explain a range of behavioral and neuroscientific data on language processing and discuss some of the implications of our proposal

Seeing With the Two Systems of Thought—a Review of ‘Seeing Things As They Are: a Theory of Perception’ by John Searle (2015)

As so often in philosophy, the title not only lays down the battle line but exposes the author’s biases and mistakes, since whether or not we can make sense of the language game ‘Seeing things as they are’ and whether it’s possible to have a ‘philosophical’ ‘theory of perception’ (which can only be about how the language of perception works), as opposed to a scientific one, which is a theory about how the brain works, are exactly the issues. This is classic Searle—superb and probably at least as good as anyone else can produce, but lacking a full understanding of the fundamental insights of the later Wittgenstein and with no grasp of the two systems of thought framework, which could have made it brilliant. As in his previous work, Searle largely avoids scientism but there are frequent lapses and he does not grasp that the issues are always about language games, a failing he shares with nearly everyone. After providing a framework consisting of a Table of Intentionality based on the two systems of thought and thinking and decision research, I give a detailed analysis of the book. Those wishing a comprehensive up to date framework for human behavior from the modern two systems view may consult my book ‘The Logical Structure of Philosophy, Psychology, Mind and Language in Ludwig Wittgenstein and John Searle’ 2nd ed (2019). Those interested in more of my writings may see ‘Talking Monkeys--Philosophy, Psychology, Science, Religion and Politics on a Doomed Planet--Articles and Reviews 2006-2019 3rd ed (2019), The Logical Structure of Human Behavior (2019), and Suicidal Utopian Delusions in the 21st Century 4th ed (2019

Seeing With the Two Systems of Thought—a Review of ‘Seeing Things As They Are: a Theory of Perception’ by John Searle (2015)(review revised 2019)

As so often in philosophy, the title not only lays down the battle line but exposes the author’s biases and mistakes, since whether or not we can make sense of the language game ‘Seeing things as they are’ and whether it’s possible to have a ‘philosophical’ ‘theory of perception’ (which can only be about how the language of perception works), as opposed to a scientific one, which is a theory about how the brain works, are exactly the issues. This is classic Searle—superb and probably at least as good as anyone else can produce, but lacking a full understanding of the fundamental insights of the later Wittgenstein and with no grasp of the two systems of thought framework, which could have made it brilliant. As in his previous work, Searle largely avoids scientism but there are frequent lapses and he does not grasp that the issues are always about language games, a failing he shares with nearly everyone. After providing a framework consisting of a Table of Intentionality based on the two systems of thought and thinking and decision research, I give a detailed analysis of the book. Those wishing a comprehensive up to date framework for human behavior from the modern two systems view may consult my book ‘The Logical Structure of Philosophy, Psychology, Mind and Language in Ludwig Wittgenstein and John Searle’ 2nd ed (2019). Those interested in more of my writings may see ‘Talking Monkeys--Philosophy, Psychology, Science, Religion and Politics on a Doomed Planet--Articles and Reviews 2006-2019 3rd ed (2019), The Logical Structure of Human Behavior (2019), and Suicidal Utopian Delusions in the 21st Century 4th ed (2019