Application of a Natural Language Interface to the Teleoperation of a Mobile Robot

IFAC Intelligent Components for Vehicles, Seville, Spain, 1998This paper describes the application of a natural language interface to the teleoperation of a mobile robot. Natural language communication with robots is a major goal, since it allows for non expert people to communicate with robots in his or her own language. This communication has to be flexible enough to allow the user to control the robot with a minimum knowledge about its details. In order to do this, the user must be able to perform simple operations as well as high level tasks which involve multiple elements of the system. For this ones, an adequate representation of the knowledge about the robot and its environment will allow the creation of a plan of simple actions whose execution will result in the accomplishment of the requested tas

Introduction for speech and language for interactive robots

This special issue includes research articles which apply spoken language processing to robots that interact with human users through speech, possibly combined with other modalities. Robots that can listen to human speech, understand it, interact according to the conveyed meaning, and respond represent major research and technological challenges. Their common aim is to equip robots with natural interaction abilities. However, robotics and spoken language processing are areas that are typically studied within their respective communities with limited communication across disciplinary boundaries. The articles in this special issue represent examples that address the need for an increased multidisciplinary exchange of ideas

MaestROB: A Robotics Framework for Integrated Orchestration of Low-Level Control and High-Level Reasoning

This paper describes a framework called MaestROB. It is designed to make the robots perform complex tasks with high precision by simple high-level instructions given by natural language or demonstration. To realize this, it handles a hierarchical structure by using the knowledge stored in the forms of ontology and rules for bridging among different levels of instructions. Accordingly, the framework has multiple layers of processing components; perception and actuation control at the low level, symbolic planner and Watson APIs for cognitive capabilities and semantic understanding, and orchestration of these components by a new open source robot middleware called Project Intu at its core. We show how this framework can be used in a complex scenario where multiple actors (human, a communication robot, and an industrial robot) collaborate to perform a common industrial task. Human teaches an assembly task to Pepper (a humanoid robot from SoftBank Robotics) using natural language conversation and demonstration. Our framework helps Pepper perceive the human demonstration and generate a sequence of actions for UR5 (collaborative robot arm from Universal Robots), which ultimately performs the assembly (e.g. insertion) task.Comment: IEEE International Conference on Robotics and Automation (ICRA) 2018. Video: https://www.youtube.com/watch?v=19JsdZi0TW

Development Issues of Healthcare Robots : Compassionate Communication for Older Adults with Dementia

Although progress is being made in affective computing, issues remain in enabling the effective expression of compassionate communication by healthcare robots. Identifying, describing and reconciling these concerns are important in order to provide quality contemporary healthcare for older adults with dementia. The purpose of this case study was to explore the development issues of healthcare robots in expressing compassionate communication for older adults with dementia. An exploratory descriptive case study was conducted with the Pepper robot and older adults with dementia using high-tech digital cameras to document significant communication proceedings that occurred during the activities. Data were collected in December 2020. The application program for an intentional conversation using Pepper was jointly developed by Tanioka’s team and the Xing Company, allowing Pepper’s words and head movements to be remotely controlled. The analysis of the results revealed four development issues, namely, (1) accurate sensing behavior for “listening” to voices appropriately and accurately interacting with subjects; (2) inefficiency in “listening” and “gaze” activities; (3) fidelity of behavioral responses; and (4) deficiency in natural language processing AI development, i.e., the ability to respond actively to situations that were not pre-programmed by the developer. Conversational engagements between the Pepper robot and patients with dementia illustrated a practical usage of technologies with artificial intelligence and natural language processing. The development issues found in this study require reconciliation in order to enhance the potential for healthcare robot engagement in compassionate communication in the care of older adults with dementia

VR Investigation on Caregivers’ Tolerance towards Communication and Processing Failures

This article was supported by the German Research Foundation (DFG) and the Open Access Publication Fund of Humboldt-Universität zu Berlin.Robots are increasingly used in healthcare to support caregivers in their daily work routines. To ensure an effortless and easy interaction between caregivers and robots, communication via natural language is expected from robots. However, robotic speech bears a large potential for technical failures, which includes processing and communication failures. It is therefore necessary to investigate how caregivers perceive and respond to robots with erroneous communication. We recruited thirty caregivers, who interacted in a virtual reality setting with a robot. It was investigated whether different kinds of failures are more likely to be forgiven with technical or human-like justifications. Furthermore, we determined how tolerant caregivers are with a robot constantly returning a process failure and whether this depends on the robot’s response pattern (constant vs. variable). Participants showed the same forgiveness towards the two justifications. However, females liked the human-like justification more and males liked the technical justification more. Providing justifications with any reasonable content seems sufficient to achieve positive effects. Robots with a constant response pattern were liked more, although both patterns achieved the same tolerance threshold from caregivers, which was around seven failed requests. Due to the experimental setup, the tolerance for communication failures was probably increased and should be adjusted in real-life situations

A Study of Non-Linguistic Utterances for Social Human-Robot Interaction

The world of animation has painted an inspiring image of what the robots of the future could be. Taking the robots R2D2 and C3PO from the Star Wars films as representative examples, these robots are portrayed as being more than just machines, rather, they are presented as intelligent and capable social peers, exhibiting many of the traits that people have also. These robots have the ability to interact with people, understand us, and even relate to us in very personal ways through a wide repertoire of social cues. As robotic technologies continue to make their way into society at large, there is a growing trend toward making social robots. The field of Human-Robot Interaction concerns itself with studying, developing and realising these socially capable machines, equipping them with a very rich variety of capabilities that allow them to interact with people in natural and intuitive ways, ranging from the use of natural language, body language and facial gestures, to more unique ways such as expression through colours and abstract sounds. This thesis studies the use of abstract, expressive sounds, like those used iconically by the robot R2D2. These are termed Non-Linguistic Utterances (NLUs) and are a means of communication which has a rich history in film and animation. However, very little is understood about how such expressive sounds may be utilised by social robots, and how people respond to these. This work presents a series of experiments aimed at understanding how NLUs can be utilised by a social robot in order to convey affective meaning to people both young and old, and what factors impact on the production and perception of NLUs. Firstly, it is shown that not all robots should use NLUs. The morphology of the robot matters. People perceive NLUs differently across different robots, and not always in a desired manner. Next it is shown that people readily project affective meaning onto NLUs though not in a coherent manner. Furthermore, people's affective inferences are not subtle, rather they are drawn to well established, basic affect prototypes. Moreover, it is shown that the valence of the situation in which an NLU is made, overrides the initial valence of the NLU itself: situational context biases how people perceive utterances made by a robot, and through this, coherence between people in their affective inferences is found to increase. Finally, it is uncovered that NLUs are best not used as a replacement to natural language (as they are by R2D2), rather, people show a preference for them being used alongside natural language where they can play a supportive role by providing essential social cues

A Review of Verbal and Non-Verbal Human-Robot Interactive Communication

In this paper, an overview of human-robot interactive communication is presented, covering verbal as well as non-verbal aspects of human-robot interaction. Following a historical introduction, and motivation towards fluid human-robot communication, ten desiderata are proposed, which provide an organizational axis both of recent as well as of future research on human-robot communication. Then, the ten desiderata are examined in detail, culminating to a unifying discussion, and a forward-looking conclusion

Natural language generation for social robotics: Opportunities and challenges

In the increasingly popular and diverse research area of social robotics, the primary goal is to develop robot agents that exhibit socially intelligent behaviour while interacting in a face-to-face context with human partners. An important aspect of face-to-face social conversation is fluent, flexible linguistic interaction: as Bavelas et al. [1] point out, face-to-face dialogue is both the basic form of human communication and the richest and most flexible, combining unrestricted verbal expression with meaningful non-verbal acts such as gestures and facial displays, along with instantaneous, continuous collaboration between the speaker and the listener. In practice, however, most developers of social robots tend not to use the full possibilities of the unrestricted verbal expression afforded by face-to-face conversation; instead, they generally tend to employ relatively simplistic processes for choosing the words for their robots to say. This contrasts with the work carried out Natural Language Generation (NLG), the field of computational linguistics devoted to the automated production of high-quality linguistic content: while this research area is also an active one, in general most effort in NLG is focussed on producing high-quality written text. This article summarises the state-of-the-art in the two individual research areas of social robotics and natural language generation. It then discusses the reasons why so few current social robots make use of more sophisticated generation techniques. Finally, an approach is proposed to bringing some aspects of NLG into social robotics, concentrating on techniques and tools that are most appropriate to the needs of socially interactive robots

Symbol Emergence in Robotics: A Survey

Humans can learn the use of language through physical interaction with their environment and semiotic communication with other people. It is very important to obtain a computational understanding of how humans can form a symbol system and obtain semiotic skills through their autonomous mental development. Recently, many studies have been conducted on the construction of robotic systems and machine-learning methods that can learn the use of language through embodied multimodal interaction with their environment and other systems. Understanding human social interactions and developing a robot that can smoothly communicate with human users in the long term, requires an understanding of the dynamics of symbol systems and is crucially important. The embodied cognition and social interaction of participants gradually change a symbol system in a constructive manner. In this paper, we introduce a field of research called symbol emergence in robotics (SER). SER is a constructive approach towards an emergent symbol system. The emergent symbol system is socially self-organized through both semiotic communications and physical interactions with autonomous cognitive developmental agents, i.e., humans and developmental robots. Specifically, we describe some state-of-art research topics concerning SER, e.g., multimodal categorization, word discovery, and a double articulation analysis, that enable a robot to obtain words and their embodied meanings from raw sensory--motor information, including visual information, haptic information, auditory information, and acoustic speech signals, in a totally unsupervised manner. Finally, we suggest future directions of research in SER.Comment: submitted to Advanced Robotic