Machine Understanding of Human Behavior

A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground. If this prediction is to come true, then next generation computing, which we will call human computing, should be about anticipatory user interfaces that should be human-centered, built for humans based on human models. They should transcend the traditional keyboard and mouse to include natural, human-like interactive functions including understanding and emulating certain human behaviors such as affective and social signaling. This article discusses a number of components of human behavior, how they might be integrated into computers, and how far we are from realizing the front end of human computing, that is, how far are we from enabling computers to understand human behavior

Symbol Emergence in Robotics: A Survey

Humans can learn the use of language through physical interaction with their environment and semiotic communication with other people. It is very important to obtain a computational understanding of how humans can form a symbol system and obtain semiotic skills through their autonomous mental development. Recently, many studies have been conducted on the construction of robotic systems and machine-learning methods that can learn the use of language through embodied multimodal interaction with their environment and other systems. Understanding human social interactions and developing a robot that can smoothly communicate with human users in the long term, requires an understanding of the dynamics of symbol systems and is crucially important. The embodied cognition and social interaction of participants gradually change a symbol system in a constructive manner. In this paper, we introduce a field of research called symbol emergence in robotics (SER). SER is a constructive approach towards an emergent symbol system. The emergent symbol system is socially self-organized through both semiotic communications and physical interactions with autonomous cognitive developmental agents, i.e., humans and developmental robots. Specifically, we describe some state-of-art research topics concerning SER, e.g., multimodal categorization, word discovery, and a double articulation analysis, that enable a robot to obtain words and their embodied meanings from raw sensory--motor information, including visual information, haptic information, auditory information, and acoustic speech signals, in a totally unsupervised manner. Finally, we suggest future directions of research in SER.Comment: submitted to Advanced Robotic

Multimodal E-Commerce: A Usability and Social Presence Investigation

This thesis investigates empirically multimodal socially interactive e-commerce interfaces. The overall hypothesis is that multimodal social interaction will improve the usability of e-commerce interfaces and increase the user‘s feeling of social presence, decision making and product understanding when compared to an equivalent non-multimodal socially interactive interface. The investigation consisted eight conditions in three experimental phases. The first experimental phase investigated non-socially interactive, static-socially interactive, and interactive-socially interactive interfaces (three conditions) using an e-commerce platform with a dependent sample of users (n=36). The second experimental phase continued with the comparative evaluation of a further two conditions based on the results of the first phase. An audio and an avatar-based socially interactive conditions were evaluated with two independent groups of users (n=18 for each group). The third experimental phase investigated three socially interactive conditions. These were text with graphics, auditory stimuli, and avatars. The results demonstrate that socially interactive metaphors in e-commerce interfaces improved the ability of users to use presented information effectively, make decisions in comparison to non-social or static social interactive interfaces. An avatar-based socially interactive e-commerce interface improved the user‘s social presence. A set of empirically derived guidelines for the design and use of these metaphors to communicate information in a socially interactive atmosphere is also introduced and discussed

Self-Supervised Vision-Based Detection of the Active Speaker as Support for Socially-Aware Language Acquisition

This paper presents a self-supervised method for visual detection of the active speaker in a multi-person spoken interaction scenario. Active speaker detection is a fundamental prerequisite for any artificial cognitive system attempting to acquire language in social settings. The proposed method is intended to complement the acoustic detection of the active speaker, thus improving the system robustness in noisy conditions. The method can detect an arbitrary number of possibly overlapping active speakers based exclusively on visual information about their face. Furthermore, the method does not rely on external annotations, thus complying with cognitive development. Instead, the method uses information from the auditory modality to support learning in the visual domain. This paper reports an extensive evaluation of the proposed method using a large multi-person face-to-face interaction dataset. The results show good performance in a speaker dependent setting. However, in a speaker independent setting the proposed method yields a significantly lower performance. We believe that the proposed method represents an essential component of any artificial cognitive system or robotic platform engaging in social interactions.Comment: 10 pages, IEEE Transactions on Cognitive and Developmental System

Pro-active Meeting Assistants : Attention Please!

This paper gives an overview of pro-active meeting assistants, what they are and when they can be useful. We explain how to develop such assistants with respect to requirement definitions and elaborate on a set of Wizard of Oz experiments, aiming to find out in which form a meeting assistant should operate to be accepted by participants and whether the meeting effectiveness and efficiency can be improved by an assistant at all

Robotic Rabbit Companions: amusing or a nuisance?

Most of the studies in human-robot interaction involve controlled experiments in a laboratory and only a limited number of studies have put robotic companions into people’s home. Introducing robots into a real-life environment does not only pose many technical challenges but also raises several methodological issues. And even though there might be a gain in ecological validity of the findings, there are other drawbacks that limit the validity of the results. In this paper we reflect on some of these issues based on the experience we gained in the SERA project where a robotic companion was put in the homes of a few people for ten days. We try to draw some general lessons from this experience

Pro-active Meeting Assistants: Attention Please!

This paper gives an overview of pro-active meeting assistants, what they are and when they can be useful. We explain how to develop such assistants with respect to requirement definitions and elaborate on a set of Wizard of Oz experiments, aiming to find out in which form a meeting assistant should operate to be accepted by participants and whether the meeting effectiveness and efficiency can be improved by an assistant at all. This paper gives an overview of pro-active meeting assistants, what they are and when they can be useful. We explain how to develop such assistants with respect to requirement definitions and elaborate on a set of Wizard of Oz experiments, aiming to find out in which form a meeting assistant should operate to be accepted by participants and whether the meeting effectiveness and efficiency can be improved by an assistant at all

‘Super disabilities’ vs ‘Disabilities’?:Theorizing the role of ableism in (mis)representational mythology of disability in the marketplace

People with disabilities (PWD) constitute one of the largest minority groups with one in five people worldwide having a disability. While recognition and inclusion of this group in the marketplace has seen improvement, the effects of (mis)representation of PWD in shaping the discourse on fostering marketplace inclusion of socially marginalized consumers remain little understood. Although effects of misrepresentation (e.g., idealized, exoticized or selective representation) on inclusion/exclusion perceptions and cognitions has received attention in the context of ethnic/racial groups, the world of disability has been largely neglected. By extending the theory of ableism into the context of PWD representation and applying it to the analysis of the We’re the Superhumans advertisement developed for the Rio 2016 Paralympic Games, this paper examines the relationship between the (mis)representation and the inclusion/exclusion discourse. By uncovering that PWD misrepresentations can partially mask and/or redress the root causes of exclusion experienced by PWD in their lived realities, it contributes to the research agenda on the transformative role of consumption cultures perpetuating harmful, exclusionary social perceptions of marginalized groups versus contributing to advancement of their inclusion