Is spoken language all-or-nothing? Implications for future speech-based human-machine interaction

Recent years have seen significant market penetration for voice-based personal assistants such as Apple’s Siri. However, despite this success, user take-up is frustratingly low. This article argues that there is a habitability gap caused by the inevitablemismatch between the capabilities and expectations of human users and the features and benefits provided by contemporary technology. Suggestions aremade as to how such problems might be mitigated, but a more worrisome question emerges: “is spoken language all-or-nothing”? The answer, based on contemporary views on the special nature of (spoken) language, is that there may indeed be a fundamental limit to the interaction that can take place between mismatched interlocutors (such as humans and machines). However, it is concluded that interactions between native and non-native speakers, or between adults and children, or even between humans and dogs, might provide critical inspiration for the design of future speech-based human-machine interaction

AI and robotics to help older adults: Revisiting projects in search of lessons learned

Abstract This article is a retrospective overview of work performed in the domain of Active Assisted Living over a span of almost 18 years. The authors have been creating and refining artificial intelligence (AI) and robotics solutions to support older adults in maintaining their independence and improving their quality of life. The goal of this article is to identify strong features and general lessons learned from those experiences and conceive guidelines and new research directions for future deployment, also relying on an analysis of similar research efforts. The work considers key points that have contributed to increase the success of the innovative solutions grounding them on known technology acceptance models. The analysis is presented with a threefold perspective: A Technological vision illustrates the characteristics of the support systems to operate in a real environment with continuity, robustness, and safety; a Socio-Health perspective highlights the role of experts in the socio-assistance domain to provide contextualized and personalized help based on actual people's needs; finally, a Human dimension takes into account the personal aspects that influence the interaction with technology in the long term experience. The article promotes the crucial role of AI and robotics in ensuring intelligent and situated assistive behaviours. Finally, considering that the produced solutions are socio-technical systems, the article suggests a transdisciplinary approach in which different relevant disciplines merge together to have a complete, coordinated, and more informed vision of the problem

Judgment of the Humanness of an Interlocutor Is in the Eye of the Beholder

Despite tremendous advances in artificial language synthesis, no machine has so far succeeded in deceiving a human. Most research focused on analyzing the behavior of “good” machine. We here choose an opposite strategy, by analyzing the behavior of “bad” humans, i.e., humans perceived as machine. The Loebner Prize in Artificial Intelligence features humans and artificial agents trying to convince judges on their humanness via computer-mediated communication. Using this setting as a model, we investigated here whether the linguistic behavior of human subjects perceived as non-human would enable us to identify some of the core parameters involved in the judgment of an agents' humanness. We analyzed descriptive and semantic aspects of dialogues in which subjects succeeded or failed to convince judges of their humanness. Using cognitive and emotional dimensions in a global behavioral characterization, we demonstrate important differences in the patterns of behavioral expressiveness of the judges whether they perceived their interlocutor as being human or machine. Furthermore, the indicators of interest displayed by the judges were predictive of the final judgment of humanness. Thus, we show that the judgment of an interlocutor's humanness during a social interaction depends not only on his behavior, but also on the judge himself. Our results thus demonstrate that the judgment of humanness is in the eye of the beholder

A Pilot Study with a Novel Setup for Collaborative Play of the Humanoid Robot KASPAR with children with autism

This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.This article describes a pilot study in which a novel experimental setup, involving an autonomous humanoid robot, KASPAR, participating in a collaborative, dyadic video game, was implemented and tested with children with autism, all of whom had impairments in playing socially and communicating with others. The children alternated between playing the collaborative video game with a neurotypical adult and playing the same game with the humanoid robot, being exposed to each condition twice. The equipment and experimental setup were designed to observe whether the children would engage in more collaborative behaviours while playing the video game and interacting with the adult than performing the same activities with the humanoid robot. The article describes the development of the experimental setup and its first evaluation in a small-scale exploratory pilot study. The purpose of the study was to gain experience with the operational limits of the robot as well as the dyadic video game, to determine what changes should be made to the systems, and to gain experience with analyzing the data from this study in order to conduct a more extensive evaluation in the future. Based on our observations of the childrens’ experiences in playing the cooperative game, we determined that while the children enjoyed both playing the game and interacting with the robot, the game should be made simpler to play as well as more explicitly collaborative in its mechanics. Also, the robot should be more explicit in its speech as well as more structured in its interactions. Results show that the children found the activity to be more entertaining, appeared more engaged in playing, and displayed better collaborative behaviours with their partners (For the purposes of this article, ‘partner’ refers to the human/robotic agent which interacts with the children with autism. We are not using the term’s other meanings that refer to specific relationships or emotional involvement between two individuals.) in the second sessions of playing with human adults than during their first sessions. One way of explaining these findings is that the children’s intermediary play session with the humanoid robot impacted their subsequent play session with the human adult. However, another longer and more thorough study would have to be conducted in order to better re-interpret these findings. Furthermore, although the children with autism were more interested in and entertained by the robotic partner, the children showed more examples of collaborative play and cooperation while playing with the human adult.Peer reviewe

Affordable Generative Agents

The emergence of large language models (LLMs) has significantly advanced the simulation of believable interactive agents. However, the substantial cost on maintaining the prolonged agent interactions poses challenge over the deployment of believable LLM-based agents. Therefore, in this paper, we develop Affordable Generative Agents (AGA), a framework for enabling the generation of believable and low-cost interactions on both agent-environment and inter-agents levels. Specifically, for agent-environment interactions, we substitute repetitive LLM inferences with learned policies; while for inter-agent interactions, we model the social relationships between agents and compress auxiliary dialogue information. Extensive experiments on multiple environments show the effectiveness and efficiency of our proposed framework. Also, we delve into the mechanisms of emergent believable behaviors lying in LLM agents, demonstrating that agents can only generate finite behaviors in fixed environments, based upon which, we understand ways to facilitate emergent interaction behaviors. Our code is publicly available at: \url{https://github.com/AffordableGenerativeAgents/Affordable-Generative-Agents}

NICOL: A Neuro-inspired Collaborative Semi-humanoid Robot that Bridges Social Interaction and Reliable Manipulation

Robotic platforms that can efficiently collaborate with humans in physical tasks constitute a major goal in robotics. However, many existing robotic platforms are either designed for social interaction or industrial object manipulation tasks. The design of collaborative robots seldom emphasizes both their social interaction and physical collaboration abilities. To bridge this gap, we present the novel semi-humanoid NICOL, the Neuro-Inspired COLlaborator. NICOL is a large, newly designed, scaled-up version of its well-evaluated predecessor, the Neuro-Inspired COmpanion (NICO). NICOL adopts NICO's head and facial expression display and extends its manipulation abilities in terms of precision, object size, and workspace size. Our contribution in this paper is twofold -- firstly, we introduce the design concept for NICOL, and secondly, we provide an evaluation of NICOL's manipulation abilities by presenting a novel extension for an end-to-end hybrid neuro-genetic visuomotor learning approach adapted to NICOL's more complex kinematics. We show that the approach outperforms the state-of-the-art Inverse Kinematics (IK) solvers KDL, TRACK-IK and BIO-IK. Overall, this article presents for the first time the humanoid robot NICOL, and contributes to the integration of social robotics and neural visuomotor learning for humanoid robots

Designing a robot to evaluate group formations

Robots are making their way in environments inhabited by people. Whether in domestic or public crowded environments, robots should take into consideration social norms and behaviors in order to become a social robot. This dissertation focuses on the problem of how to develop a robotic platform in order to validate human-robot interaction experiences in realistic environments. More specifically, we are concerned with social interactions in human-robot groups in public scenarios, where a variety of people can converge. Our final goal is the develop of a social robot based on certain theories of group behavior and the use of space, known as spatial relationships. The intermediate goals are related with the design and development of the experiences in the wild: as minor changes as possible in the scenario, definition of social tasks, gradual development of a robotic platform as transparent as possible from the robotic side. Initially, this research introduces several preliminary studies of human-robot interaction (HRI) with the PAL Robotics’ REEM robot at the CosmoCaixa Science Museum in Barcelona. Based on certain theories about the use of space as a form of social communication or interaction, the task under study with the commercial robot is as a museum guide, both when the group was in motion (\ie when it was being guided) as well as a group in a static place. Moreover, a second HRI study with REEM robot accomplishing the task of a teacher's assistant was carried out to analyze the perception of the robot's social presence and identity. Likewise, the development of a robotic platform, known as MASHI, for the study of HRI is presented. Based on the service to be completed by the robot, improvements in the experimental robotic platform (structure, morphology, head, face, arms) were carried out in continuous cycles following the development of HRI experiences. This structure should be hold as simple as possible in order to make it `transparent' in the social HRI study. Next, the field study of human-robot social interaction with the MASHI robot with the role of exhibition guide in a cultural center is presented. Based on direct observation techniques, a study is made of the different spatial relationships that are generated when a robot interacts with a person or groups of people. Finally, a novel approach to represent the spatial relationships of HRI in a qualitative way is introduced for future experiences. In this concluding study, we analyze different spatial arrangements generated in a social scenario with a robot within the guide role. As a main conclusion, it can be stated that people follow social norms, in the form of spatial relationships, when interacting with a robot that provide a social service in a public space. Children, however, recurrently challenge these social norms, probably because they are constantly learning about the norms that regulate our coexistence. Spatial relationships are clearly reinforced when the role assigned to the robot is more explicit and understood by people. Spatial relationships can be affected by the characteristics of the environment, either by the available space or by the elements arranged in it, as well as by the number of people who inhabit it. Overall, this dissertation points out that the provided service, and its understanding from the user’s side, is more important that the robotic skills of the robotic platform in order to improve user experiences in public environments.Los robots se abren paso en entornos habitados por personas. Ya sea en entornos domésticos o públicos, los robots deben tener en cuenta ciertas normas y comportamientos sociales para convertirse en un robot social. Esta disertación se centra en el problema de cómo desarrollar una plataforma robótica para validar experiencias de interacción humano-robot en entornos realistas. Más específicamente, nos preocupamos por las interacciones sociales en grupos humano-robot en escenarios públicos, donde una gran variedad de personas puede converger. Nuestro objetivo final es el desarrollo de un robot social basado en ciertas teorías de comportamiento grupal y el uso del espacio, conocidas como relaciones espaciales. Los objetivos intermedios están relacionados con el diseño y desarrollo de las experiencias `en la naturaleza': cambios mínimos como sea posible en el escenario, definición de tareas sociales, desarrollo gradual de una plataforma robótica lo más transparente posible desde el lado robótico. Inicialmente, esta investigación presenta varios estudios preliminares de interacción humano-robot (HRI) con el robot REEM de PAL Robotics en el Museo de Ciencias CosmoCaixa de Barcelona. Basado en ciertas teorías sobre el uso del espacio como una forma de comunicación o interacción social, la tarea en este estudio con el robot comercial es como guía de museo, tanto cuando el grupo estaba en movimiento (es decir, cuando estaba siendo guiado) como cuando el grupo estaba en un lugar estático. Además, se llevó a cabo un segundo estudio de HRI con un robot REEM que realizaba la tarea de un asistente de profesor para analizar la percepción de la presencia e identidad social del robot. Asimismo, se presenta el desarrollo de una plataforma robótica, conocida como MASHI, para el estudio de la HRI. En función del servicio que debe completar el robot, las mejoras en la plataforma robótica experimental (estructura, morfología, cabeza, cara, brazos) se llevaron a cabo en ciclos continuos siguiendo el desarrollo de las experiencias de HRI. Esta estructura debe mantenerse lo más simple posible para que sea 'transparente' en el estudio de HRI social. A continuación, se presenta el estudio de campo de la interacción social humano-robot con el robot MASHI con el papel de guía de exposición en un centro cultural. Con base en técnicas de observación directa, se realiza un estudio de las diferentes relaciones espaciales que se generan cuando un robot interactúa con una persona o grupos de personas. Finalmente, se introduce un enfoque novedoso para representar las relaciones espaciales de la HRI de forma cualitativa para las experiencias futuras. En este estudio final, analizamos diferentes arreglos espaciales generados en un escenario social con un robot con el rol de guía. Como conclusión principal, se puede afirmar que las personas siguen normas sociales, en forma de relaciones espaciales, cuando interactúan con un robot que brinda un servicio social en un espacio público. Los niños, sin embargo, desafían recurrentemente estas normas sociales, probablemente porque están aprendiendo constantemente sobre las normas que regulan nuestra convivencia. Las relaciones espaciales se refuerzan claramente cuando el rol asignado al robot es más explícito y entendido por las personas. Las relaciones espaciales pueden verse afectadas por las características del entorno, ya sea por el espacio disponible o por los elementos dispuestos en él, así como por el número de personas que lo habitan. En general, esta disertación señala que el servicio prestado, y su comprensión del lado del usuario, es más importante que las habilidades robóticas de la plataforma robótica con el fin de mejorar las experiencias del usuario en entornos público

Designing a robot to evaluate group formations

Robots are making their way in environments inhabited by people. Whether in domestic or public crowded environments, robots should take into consideration social norms and behaviors in order to become a social robot. This dissertation focuses on the problem of how to develop a robotic platform in order to validate human-robot interaction experiences in realistic environments. More specifically, we are concerned with social interactions in human-robot groups in public scenarios, where a variety of people can converge. Our final goal is the develop of a social robot based on certain theories of group behavior and the use of space, known as spatial relationships. The intermediate goals are related with the design and development of the experiences in the wild: as minor changes as possible in the scenario, definition of social tasks, gradual development of a robotic platform as transparent as possible from the robotic side. Initially, this research introduces several preliminary studies of human-robot interaction (HRI) with the PAL Robotics’ REEM robot at the CosmoCaixa Science Museum in Barcelona. Based on certain theories about the use of space as a form of social communication or interaction, the task under study with the commercial robot is as a museum guide, both when the group was in motion (\ie when it was being guided) as well as a group in a static place. Moreover, a second HRI study with REEM robot accomplishing the task of a teacher's assistant was carried out to analyze the perception of the robot's social presence and identity. Likewise, the development of a robotic platform, known as MASHI, for the study of HRI is presented. Based on the service to be completed by the robot, improvements in the experimental robotic platform (structure, morphology, head, face, arms) were carried out in continuous cycles following the development of HRI experiences. This structure should be hold as simple as possible in order to make it `transparent' in the social HRI study. Next, the field study of human-robot social interaction with the MASHI robot with the role of exhibition guide in a cultural center is presented. Based on direct observation techniques, a study is made of the different spatial relationships that are generated when a robot interacts with a person or groups of people. Finally, a novel approach to represent the spatial relationships of HRI in a qualitative way is introduced for future experiences. In this concluding study, we analyze different spatial arrangements generated in a social scenario with a robot within the guide role. As a main conclusion, it can be stated that people follow social norms, in the form of spatial relationships, when interacting with a robot that provide a social service in a public space. Children, however, recurrently challenge these social norms, probably because they are constantly learning about the norms that regulate our coexistence. Spatial relationships are clearly reinforced when the role assigned to the robot is more explicit and understood by people. Spatial relationships can be affected by the characteristics of the environment, either by the available space or by the elements arranged in it, as well as by the number of people who inhabit it. Overall, this dissertation points out that the provided service, and its understanding from the user’s side, is more important that the robotic skills of the robotic platform in order to improve user experiences in public environments.Los robots se abren paso en entornos habitados por personas. Ya sea en entornos domésticos o públicos, los robots deben tener en cuenta ciertas normas y comportamientos sociales para convertirse en un robot social. Esta disertación se centra en el problema de cómo desarrollar una plataforma robótica para validar experiencias de interacción humano-robot en entornos realistas. Más específicamente, nos preocupamos por las interacciones sociales en grupos humano-robot en escenarios públicos, donde una gran variedad de personas puede converger. Nuestro objetivo final es el desarrollo de un robot social basado en ciertas teorías de comportamiento grupal y el uso del espacio, conocidas como relaciones espaciales. Los objetivos intermedios están relacionados con el diseño y desarrollo de las experiencias `en la naturaleza': cambios mínimos como sea posible en el escenario, definición de tareas sociales, desarrollo gradual de una plataforma robótica lo más transparente posible desde el lado robótico. Inicialmente, esta investigación presenta varios estudios preliminares de interacción humano-robot (HRI) con el robot REEM de PAL Robotics en el Museo de Ciencias CosmoCaixa de Barcelona. Basado en ciertas teorías sobre el uso del espacio como una forma de comunicación o interacción social, la tarea en este estudio con el robot comercial es como guía de museo, tanto cuando el grupo estaba en movimiento (es decir, cuando estaba siendo guiado) como cuando el grupo estaba en un lugar estático. Además, se llevó a cabo un segundo estudio de HRI con un robot REEM que realizaba la tarea de un asistente de profesor para analizar la percepción de la presencia e identidad social del robot. Asimismo, se presenta el desarrollo de una plataforma robótica, conocida como MASHI, para el estudio de la HRI. En función del servicio que debe completar el robot, las mejoras en la plataforma robótica experimental (estructura, morfología, cabeza, cara, brazos) se llevaron a cabo en ciclos continuos siguiendo el desarrollo de las experiencias de HRI. Esta estructura debe mantenerse lo más simple posible para que sea 'transparente' en el estudio de HRI social. A continuación, se presenta el estudio de campo de la interacción social humano-robot con el robot MASHI con el papel de guía de exposición en un centro cultural. Con base en técnicas de observación directa, se realiza un estudio de las diferentes relaciones espaciales que se generan cuando un robot interactúa con una persona o grupos de personas. Finalmente, se introduce un enfoque novedoso para representar las relaciones espaciales de la HRI de forma cualitativa para las experiencias futuras. En este estudio final, analizamos diferentes arreglos espaciales generados en un escenario social con un robot con el rol de guía. Como conclusión principal, se puede afirmar que las personas siguen normas sociales, en forma de relaciones espaciales, cuando interactúan con un robot que brinda un servicio social en un espacio público. Los niños, sin embargo, desafían recurrentemente estas normas sociales, probablemente porque están aprendiendo constantemente sobre las normas que regulan nuestra convivencia. Las relaciones espaciales se refuerzan claramente cuando el rol asignado al robot es más explícito y entendido por las personas. Las relaciones espaciales pueden verse afectadas por las características del entorno, ya sea por el espacio disponible o por los elementos dispuestos en él, así como por el número de personas que lo habitan. En general, esta disertación señala que el servicio prestado, y su comprensión del lado del usuario, es más importante que las habilidades robóticas de la plataforma robótica con el fin de mejorar las experiencias del usuario en entornos públicosPostprint (published version