SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents

Building embodied autonomous agents capable of participating in social interactions with humans is one of the main challenges in AI. Within the Deep Reinforcement Learning (DRL) field, this objective motivated multiple works on embodied language use. However, current approaches focus on language as a communication tool in very simplified and non-diverse social situations: the "naturalness" of language is reduced to the concept of high vocabulary size and variability. In this paper, we argue that aiming towards human-level AI requires a broader set of key social skills: 1) language use in complex and variable social contexts; 2) beyond language, complex embodied communication in multimodal settings within constantly evolving social worlds. We explain how concepts from cognitive sciences could help AI to draw a roadmap towards human-like intelligence, with a focus on its social dimensions. As a first step, we propose to expand current research to a broader set of core social skills. To do this, we present SocialAI, a benchmark to assess the acquisition of social skills of DRL agents using multiple grid-world environments featuring other (scripted) social agents. We then study the limits of a recent SOTA DRL approach when tested on SocialAI and discuss important next steps towards proficient social agents. Videos and code are available at https://sites.google.com/view/socialai.Comment: under review. This paper extends and generalizes work in arXiv:2104.1320

The Effect of Physical Weight and Stimulus Spatial Location on Lexical Decision: Implications for Embodied Cognition

Traditional models of cognition within cognitive psychology have utilised dualistic perspectives and largely ignored the roles of the motor systems and bodily experiences. More recent embodied approaches have sought to combat this dualism by incorporating the motor systems and bodily experiences into their perspectives. Recent research has highlighted the role of bodily experiences in shaping cognition (Proffitt, 2006; Jostmann et al., 2009), how language comprehension can be embodied and grounded in physical experiences (Glenberg and Kaschak, 2002; Zwaan and Yaxley, 2003) and also how stimulus spatial location can influence responses (Meteyard et al., 2008; Dunn et al., 2014). The present study aimed to explore those areas and provide empirical evidence in support as well as explore a gap in current research. The literature search indicated an abundance of embodied system research but a lack of research looking at possible interactions between the systems, it was this gap that was explored within the present study. Utilising a lexical decision task and methods similar to that of Proffitt (2006) three experiments were conducted. A total of 64 participants underwent standard and spatial lexical decision tasks. Three experiments were conducted exploring the bodily effect of weight, stimulus spatial effect and interactions between embodied systems.Results from the three experiments displayed a lack of support for past research regarding the effect of the bodily experience of weight. Results also displayed a main effect of word type leading to the indication that the comprehension of the word/non-word letter strings affected task performance. Analysis of results proposed that a cohesion effect between embodied systems facilitated task performance. It was concluded that further research is needed in order to fully understand the possibility of dominance or cohesion effects within an embodied perspective

Navigating Subjectivity: South, a Psychometric Text Adventure.

South: A Psychometric Text Adventure is an artist’s book and a set of software programs. The South project re-conceptualises the artist’s book and wider bookforms, encouraging models of interaction that are aware of specific locations and individual subjects. These alternatives are a response to what this thesis frames as two rapidly stagnating forms. The thesis argues that both the artist’s book and electronic literature (see the glossary on page 343 for definitions of the key terms used throughout this thesis) have not made a significant impact on the cultural landscape of the early 21st century. Nor have they made a significant use of the key technological changes that have occurred since the first electronic literature emerged in the late 1970s (in the form of interactive fictions, sometimes called ‘Text Adventures’, such as Colossal Cave Adventure (Crowther, 1976)). In order to move forward from the increasingly problematic, disembodied, computational models used in these early digital works (discussed in chapters two, five and six) this thesis specifically recommends the formation of temporally specific, contextualised, relationships between readers and digital texts. The South project presents a multi-linear, situated and embodied form of intra-activity (see glossary) as an alternative to more linear forms of interaction. These ideas and their implications for electronic literature and artist’s books will be clarified and outlined throughout this thesis, as will the rationale for framing them as valid models for moving electronic literature and artist’s books into a position of cultural and technological relevance

New Materialist Explorations into Language Education

This open access book analyzes language education through a socio-material framework. The authors revisit their position as researchers by decentering themselves and humans in general from the main focus of research activities and giving way to the materialities that are agentive but often overlooked parts of our research contexts and processes. Through this critical posthumanist realism, they are able to engage in research that sees society as an ethical interrelationship between humans and the material world and explore the socio-materialities of language education from the perspectives of material agency, spatial and embodied materiality, and human and non-human assemblages. Each chapter explores language educational contexts through a unique lens of (socio)materiality. Based on how the authors conceptualize (socio)materiality, the book is organized in three sections that seek answers to the following overarching questions: In what ways do material agencies emerge in language educational contexts? How are educational choices and experiences intertwined with materialities of spaces and bodies? What assemblages of human and non-human may occur in language education contexts? Each chapter questions, in its own way, the notion of the human subject as rational, enlightened being and sole possessor of agency, and offers examples of allowing for other-than-human agency to enter the picture. Together, the contributors exemplify how researchers who have been committed to social constructionist thinking for most of their careers learn to make space for new theories, thus inspiring and encouraging readers to remain open for new intellectual and embodied endeavors

Designing Embodied Interactive Software Agents for E-Learning: Principles, Components, and Roles

Embodied interactive software agents are complex autonomous, adaptive, and social software systems with a digital embodiment that enables them to act on and react to other entities (users, objects, and other agents) in their environment through bodily actions, which include the use of verbal and non-verbal communicative behaviors in face-to-face interactions with the user. These agents have been developed for various roles in different application domains, in which they perform tasks that have been assigned to them by their developers or delegated to them by their users or by other agents. In computer-assisted learning, embodied interactive pedagogical software agents have the general task to promote human learning by working with students (and other agents) in computer-based learning environments, among them e-learning platforms based on Internet technologies, such as the Virtual Linguistics Campus (www.linguistics-online.com). In these environments, pedagogical agents provide contextualized, qualified, personalized, and timely assistance, cooperation, instruction, motivation, and services for both individual learners and groups of learners. This thesis develops a comprehensive, multidisciplinary, and user-oriented view of the design of embodied interactive pedagogical software agents, which integrates theoretical and practical insights from various academic and other fields. The research intends to contribute to the scientific understanding of issues, methods, theories, and technologies that are involved in the design, implementation, and evaluation of embodied interactive software agents for different roles in e-learning and other areas. For developers, the thesis provides sixteen basic principles (Added Value, Perceptible Qualities, Balanced Design, Coherence, Consistency, Completeness, Comprehensibility, Individuality, Variability, Communicative Ability, Modularity, Teamwork, Participatory Design, Role Awareness, Cultural Awareness, and Relationship Building) plus a large number of specific guidelines for the design of embodied interactive software agents and their components. Furthermore, it offers critical reviews of theories, concepts, approaches, and technologies from different areas and disciplines that are relevant to agent design. Finally, it discusses three pedagogical agent roles (virtual native speaker, coach, and peer) in the scenario of the linguistic fieldwork classes on the Virtual Linguistics Campus and presents detailed considerations for the design of an agent for one of these roles (the virtual native speaker)

The evolution of grounded spatial language

This book presents groundbreaking robotic experiments on how and why spatial language evolves. It provides detailed explanations of the origins of spatial conceptualization strategies, spatial categories, landmark systems and spatial grammar by tracing the interplay of environmental conditions, communicative and cognitive pressures. The experiments discussed in this book go far beyond previous approaches in grounded language evolution. For the first time, agents can evolve not only particular lexical systems but also evolve complex conceptualization strategies underlying the emergence of category systems and compositional semantics. Moreover, many issues in cognitive science, ranging from perception and conceptualization to language processing, had to be dealt with to instantiate these experiments, so that this book contributes not only to the study of language evolution but to the investigation of the cognitive bases of spatial language as well