A Review of Verbal and Non-Verbal Human-Robot Interactive Communication

In this paper, an overview of human-robot interactive communication is presented, covering verbal as well as non-verbal aspects of human-robot interaction. Following a historical introduction, and motivation towards fluid human-robot communication, ten desiderata are proposed, which provide an organizational axis both of recent as well as of future research on human-robot communication. Then, the ten desiderata are examined in detail, culminating to a unifying discussion, and a forward-looking conclusion

Study on multi-modal conversation system

制度:新 ; 文部省報告番号:甲1783号 ; 学位の種類:博士(工学) ; 授与年月日:2003/3/15 ; 早大学位記番号:新354

Desenvolvimento de um sistema de diálogo para interação com robôs

Mestrado em Engenharia de Computadores e TelemáticaService robots operate in the same environment as humans and perform actions that a human usually performs. These robots must be able to operate autonomously in unknown and dynamic environments, as well as to maneuver with several people and know how to deal with them. By complying with these requirements, they are able to successfully address humans and fulfill their requests whenever they need assistance in a certain task. Natural language communication, including speech that is the most natural way of communication between humans, becomes relevant in the field of Human-Robot Interaction (HRI). By endowing service robots with intuitive spoken interfaces, the specification of the human required tasks is facilitated. However, this is a complicated task to achieve due to the resources involved in creating a sufficiently intuitive spoken interface and because of the difficulty of deploying it in different robots. The main objective of this thesis is the definition, implementation and evaluation of a dialogue system that can be easily integrated into any robotic platform and that functions as a flexible base for the creation of any conversational scenario in the Portuguese language. The system must meet the basic requirements for intuitive and natural communications, namely the characteristics of human-human conversations. A system was developed that functions as a base to give continuity to future work on Spoken Dialog Systems. The system incorporates the client-server architecture, where the client runs on the robot and captures what the user says. The client takes advantage on external dialogue management services. They are executed by the server, which processes the audio obtained, returning an appropriate response given the context of the dialogue. The development was based on a critical analysis of the state of the art in order for the system to be as faithful as possible to what is already done. Through the evaluation phase of the system, it was managed to obtain by few volunteers the conclusion that the main objective was accomplished: a base system was created that is flexible enough to explore different contexts of conversation, such as interacting with children or providing information on a university environment.Os robôs de serviço operam no mesmo ambiente dos humanos e executam ações que um humano normalmente executaria. Estes robôs devem ser capazes de operar de forma autónoma em ambientes desconhecidos e dinâmicos, assim como de manobrar em ambientes com várias pessoas e de saberem lidar com elas. Ao respeitarem estes requisitos, conseguirão abordar com sucesso os humanos e cumprir as suas solicitações sempre que estes precisem de assistência em alguma tarefa. A comunicação por linguagem natural, nomeadamente a fala que é a forma mais abrangente de comunicação entre humanos, torna-se relevante na área da Interação humano-robô (IHR). Ao dotar os robôs de serviço com sistemas de voz intuitivos facilita-se a especificação das tarefas a realizar. No entanto, é uma tarefa complicada de se realizar devido aos recursos envolvidos na criação de uma interação suficientemente intuitiva e devido à dificuldade de funcionar em diversos robôs. O objetivo principal deste trabalho é a definição, implementação e avaliação de um sistema de diálogo que seja de fácil integração em qualquer sistema robótico e que funcione como uma base flexível para qualquer cenário de conversação na língua Portuguesa. Deve obedecer a requisitos base de comunicação intuitiva e natural, nomeadamente a características de conversas entre humanos. Foi desenvolvido um sistema que funciona como uma base para dar continuidade a trabalho futuro em sistemas de diálogo. O sistema incorpora a arquitetura cliente-servidor onde o cliente é executado no robô e capta o que o utilizador diz. O cliente tira partido de serviços de gestão de diálogo externos ao robô, executados pelo servidor, que processa o áudio obtido, devolvendo uma resposta ao cliente adequada ao contexto do diálogo. O desenvolvimento foi baseado numa análise crítica do estado da arte para se tentar manter fiel ao que já foi feito e de forma a se tomarem as principais decisões durante a implementação. Mediante a fase de avaliação do sistema, tanto a nível do ponto de vista da interação como do programador, conseguiu-se obter por parte de alguns voluntários que o objetivo principal foi cumprido: foi criada uma base suficientemente flexível para explorar diferentes contextos de conversação, nomeadamente interagir com crianças ou fornecimento de informações em ambiente universitário

Integration of a voice recognition system in a social robot

Human-Robot Interaction (HRI) 1 is one of the main fields in the study and research of robotics. Within this field, dialog systems and interaction by voice play a very important role. When speaking about human- robot natural dialog we assume that the robot has the capability to accurately recognize the utterance what the human wants to transmit verbally and even its semantic meaning, but this is not always achieved. In this paper we describe the steps and requirements that we went through in order to endow the personal social robot Maggie, developed in the University Carlos III of Madrid, with the capability of understanding the natural language spoken by any human. We have analyzed the different possibilities offered by current software/hardware alternatives by testing them in real environments. We have obtained accurate data related to the speech recognition capabilities in different environments, using the most modern audio acquisition systems and analyzing not so typical parameters as user age, sex, intonation, volume and language. Finally we propose a new model to classify recognition results as accepted and rejected, based in a second ASR opinion. This new approach takes into account the pre-calculated success rate in noise intervals for each recognition framework decreasing false positives and false negatives rate.The funds have provided by the Spanish Government through the project called `Peer to Peer Robot-Human Interaction'' (R2H), of MEC (Ministry of Science and Education), and the project “A new approach to social robotics'' (AROS), of MICINN (Ministry of Science and Innovation). The research leading to these results has received funding from the RoboCity2030-II-CM project (S2009/DPI-1559), funded by Programas de Actividades I+D en la Comunidad de Madrid and cofunded by Structural Funds of the EU

Voice Enabled Interface for Interactive Tour Guide Robots

This paper considers design methodologies in order to develop voice-enabled interfaces for tour-guide robots to be deployed at the Robotics Exposition of the Swiss National Exhibition (Expo.02). Human-robot voice communication presents new challenges for design of fully autonomous mobile robots, in that interactivity must be robot-initiated in conversation and within a dynamic adverse environment. We approached these general problems for a voice enabled interface, tailored to limited computational resources of one on-board processor, when integrating smart speech signal acquisition, automatic speech recognition and synthesis, as well as dialogue system into the multi-modal, multi-sensor interface for the expo tour-guide robot. We also focus on particular issues that need to be addressed in voice-based interaction when planning specific tasks and research experiments for Expo.02 where tour-guide robots will interact with hundred of thousands of visitors during six months, seven days a week, ten hours per day

Comunicação humano-robô através de linguagem falada

Doutoramento em Engenharia ElectrotécnicaNos últimos anos, as tecnologias que dão suporte à robótica avançaram expressivamente. É possível encontrar robôs de serviço nos mais variados campos. O próximo passo é o desenvolvimento de robôs inteligentes, com capacidade de comunicação em linguagem falada e de realizar trabalhos úteis em interação/cooperação com humanos. Torna-se necessário, então, encontrar um modo de interagir eficientemente com esses robôs, e com agentes inteligentes de maneira geral, que permita a transmissão de conhecimento em ambos os sentidos. Partiremos da hipótese de que é possível desenvolver um sistema de diálogo baseado em linguagem natural falada que resolva esse problema. Assim, o objetivo principal deste trabalho é a definição, implementação e avaliação de um sistema de diálogo utilizável na interação baseada em linguagem natural falada entre humanos e agentes inteligentes. Ao longo deste texto, mostraremos os principais aspectos da comunicação por linguagem falada, tanto entre os humanos, como também entre humanos e máquinas. Apresentaremos as principais categorias de sistemas de diálogo, com exemplos de alguns sistemas implementados, assim como ferramentas para desenvolvimento e algumas técnicas de avaliação. A seguir, entre outros aspectos, desenvolveremos os seguintes: a evolução levada a efeito na arquitetura computacional do Carl, robô utilizado neste trabalho; o módulo de aquisição e gestão de conhecimento, desenvolvido para dar suporte à interação; e o novo gestor de diálogo, baseado na abordagem de “Estado da Informação”, também concebido e implementado no âmbito desta tese. Por fim, uma avaliação experimental envolvendo a realização de diversas tarefas de interação com vários participantes voluntários demonstrou ser possível interagir com o robô e realizar as tarefas solicitadas. Este trabalho experimental incluiu avaliação parcial de funcionalidades, avaliação global do sistema de diálogo e avaliação de usabilidade.In recent years, robotics-related technologies have reached a remarkable level of maturity. Service robots can be found in various fields. The next step is the development of intelligent robots, capable of communicating in spoken language and doing useful work in interaction / cooperation with humans. It is then necessary to find a way to efficiently interact with these robots, and with intelligent agents in general, enabling the transmission of knowledge in both directions. We will assume that one can develop a spoken language dialogue system to solve this problem. Therefore, the main goal of this work is the design, implementation and evaluation of a dialogue system that can be used on spoken language interaction between humans and intelligent agents. Throughout this document, we present and discuss the main aspects related to spoken language communication, among humans as well as between humans and machines. We present the main dialogue system categories, with examples of some implemented systems, development tools and a few evaluation techniques. Then, we describe the developed dialog system and its integration in a real robot, including the following aspects: the evolution in the computational architecture of Carl, the robot used in this work; the knowledge acquisition and management module, developed to support the interaction; and the new dialogue manager, based on the “Information State” approach, also designed and implemented within this thesis work. Finally, an experimental evaluation involving the completion of several interaction tasks involving several volunteers proved to be possible to interact with the robot and perform the requested tasks. The evaluation includes a partial evaluation of features, an overall evaluation of the dialogue system and a usability evaluation

A Software Testbed for Assessing Human-Robot Verbal Interaction

Verbal interaction provides a natural and social-style interaction mode by which robots can communicate with general public who is likely unknowledgeable in robotics. This interaction mechanism is also very important for a broad range of users such as hands/eyes-busy users, motor-impaired users, users with vision impairment and users working in hostile environments. Verbal interaction is very popular in robotics especially in personal assistive robots, which are used to help elderly people and in entertainment robots. Several research endeavors have been assigned to endow the robots with verbal interaction as a high-level faculty. However, the language usages of many of them were simple and may not be considered as full speech dialogue systems providing natural language understanding. In this thesis, we investigate a testbed platform that can be deployed to enable human-robot verbal interaction. The proposed approach encompasses a design pattern-based user interface and a user-independent automatic speech recognizer with a modified grammar module in the context of human-robot interaction. The user interface is used to simulate robots response toward multiple users’ voice commands. The performance of the proposed testbed has been evaluated quantitatively using a set of evaluation metrics such as word correct rate, recognition time and success and false action rates. The conducted experiments show the promising features of the system. The results obtained could be refined even further by training the system for more voice commands and the whole system could be ported to real robotic platforms such as Peoplebot to endow it with natural language understanding