The multi-modal interface of Robot-Era multi-robot services tailored for the elderly

Socially assistive robotic platforms are now a realistic option for the long-term care of ageing populations. Elderly users may benefit from many services provided by robots operating in different environments, such as providing assistance inside apartments, serving in shared facilities of buildings or guiding people outdoors. In this paper, we present the experience gained within the EU FP7 ROBOT-ERA project towards the objective of implementing easy-to-use and acceptable service robotic system for the elderly. In particular, we detail the user-centred design and the experimental evaluation in realistic environments of a web-based multi-modal user interface tailored for elderly users of near future multi-robot services. Experimental results demonstrate positive evaluation of usability and willingness to use by elderly users, especially those less experienced with technological devices who could benefit more from the adoption of robotic services. Further analyses showed how multi-modal modes of interaction support more flexible and natural elderly–robot interaction, make clear the benefits for the users and, therefore, increase its acceptability. Finally, we provide insights and lessons learned from the extensive experimentation, which, to the best of our knowledge, is one of the largest experimentation of a multi-robot multi-service system so far

Experimental evaluation of a multi-modal user interface for a robotic service

This paper reports the experimental evaluation of a Multi- Modal User Interface (MMUI) designed to enhance the user experience in terms of service usability and to increase acceptability of assistive robot systems by elderly users. The MMUI system offers users two main modalities to send commands: they are a GUI, usually running on the tablet attached to the robot, and a SUI, with a wearable microphone on the user. The study involved fifteen participants, aged between 70 and 89 years old, who were invited to interact with a robotic platform customized for providing every-day care and services to the elderly. The experimental task for the participants was to order a meal from three different menus using any interaction modality they liked. Quantitative and qualitative data analyses demonstrate a positive evaluation by users and show that the multi-modal means of interaction can help to make elderly-robot interaction more flexible and natural

Talking About Task Progress: Towards Integrating Task Planning and Dialog for Assistive Robotic Services

The use of service robots to assist ageing people in their own homes has the potential to allow people to maintain their independence, increasing their health and quality of life. In many assistive applications, robots perform tasks on people’s behalf that they are unable or unwilling to monitor directly. It is important that users be given useful and appropriate information about task progress. People being assisted in homes and other realworld environments are likely be engaged in other activities while they wait for a service, so information should also be presented in an appropriate, nonintrusive manner. This paper presents a human-robot interaction experiment investigatingwhat type of feedback people prefer in verbal updates by a service robot about distributed assistive services. People found feedback about time until task completion more useful than feedback about events in task progress or no feedback. We also discuss future research directions that involve giving non-expert users more input into the task planning process when delays or failures occur that necessitate replanning or modifying goals

Temporal Patterns in Multi-modal Social Interaction between Elderly Users and Service Robot

Social interaction, especially for older people living alone is a challenge currently facing human-robot interaction (HRI). User interfaces to manage service robots in home environments need to be tailored for older people. Multi-modal interfaces providing users with more than one communication option seem promising. There has been little research on user preference towards HRI interfaces; most studies have focused on utility and functionality of the interface. In this paper, we took both objective observations and participants’ opinions into account in studying older users with a robot partner. Our study was under the framework of the EU FP7 Robot-Era Project. The developed dual-modal robot interface offered older users options of speech or touch screen to perform tasks. Fifteen people aged from 70 to 89 years old, participated. We analyzed the spontaneous actions of the participants, including their attentional activities (eye contacts) and conversational activities, the temporal characteristics (timestamps, duration of events, event transitions) of these social behaviours, as well as questionnaires. This combination of data distinguishes it from other studies that focused on questionnaire ratings only. There were three main findings. First, the design of the Robot-Era interface was very acceptable for older users. Secondly, most older people used both speech and tablet to perform the food delivery service, with no difference in their preferences towards either. Thirdly, these older people had frequent and long-duration eye contact with the robot during their conversations, showing patience when expecting the robot to respond. They enjoyed the service. Overall, social engagement with the robot demonstrated by older people was no different from what might be expected towards a human partner. This study is an early attempt to reveal the social connections between human beings and a personal robot in real life. Our observations and findings should inspire new insights in HRI research and eventually contribute to next-generation intelligent robot developmen

A Review on Usability and User Experience of Assistive Social Robots for Older Persons

In the advancement of human-robot interaction technology, assistive social robots have been recognized as one of potential technologies that can provide physical and cognitive supports in older persons care. However, a major challenge faced by the designers is to develop an assistive social robot with prodigious usability and user experience for older persons who were known to have physical and cognitive limitations. A considerable number of published literatures was reporting on the technological design process of assistive social robots. However, only a small amount of attention has been paid to review the usability and user experience of the robots. The objective of this paper is to provide an overview of established researches in the literatures concerning usability and user experience issues faced by the older persons when interacting with assistive social robots. The authors searched relevant articles from the academic databases such as Google Scholar, Scopus and Web of Science as well as Google search for the publication period 2000 to 2021. Several search keywords were typed such as ‘older persons’ ‘elderly’, ‘senior citizens’, ‘assistive social robots’, ‘companion robots’, ‘personal robots’, ‘usability’ and ‘user experience’. This online search found a total of 215 articles which are related to assistive social robots in elderly care. Out of which, 54 articles identified as significant references, and they were examined thoroughly to prepare the main content of this paper. This paper reveals usability issues of 28 assistive social robots, and feedbacks of user experience based on 41 units of assistive social robots. Based on the research articles scrutinized, the authors concluded that the key elements in the design and development of assistive social robots to improve acceptance of older persons were determined by three factors: functionality, usability and users’ experience. Functionality refers to ability of robots to serve the older persons. Usability is ease of use of the robots. It is an indicator on how successful of interaction between the robots and the users. To improve usability, robot designers should consider the limitations of older persons such as vision, hearing, and cognition capabilities when interacting with the robots. User experience reflects to perceptions, preferences and behaviors of users that occur before, during and after use the robots. Combination of superior functionality and usability lead to a good user experience in using the robots which in the end achieves satisfaction of older persons

A Review on Usability and User Experience of Assistive Social Robots for Older Persons

In the advancement of human-robot interaction technology, assistive social robots have been recognized as one of potential technologies that can provide physical and cognitive supports in older persons care. However, a major challenge faced by the designers is to develop an assistive social robot with prodigious usability and user experience for older persons who were known to have physical and cognitive limitations. A considerable number of published literatures was reporting on the technological design process of assistive social robots. However, only a small amount of attention has been paid to review the usability and user experience of the robots. The objective of this paper is to provide an overview of established researches in the literatures concerning usability and user experience issues faced by the older persons when interacting with assistive social robots. The authors searched relevant articles from the academic databases such as Google Scholar, Scopus and Web of Science as well as Google search for the publication period 2000 to 2021. Several search keywords were typed such as ‘older persons’ ‘elderly’, ‘senior citizens’, ‘assistive social robots’, ‘companion robots’, ‘personal robots’, ‘usability’ and ‘user experience’. This online search found a total of 215 articles which are related to assistive social robots in elderly care. Out of which, 54 articles identified as significant references, and they were examined thoroughly to prepare the main content of this paper. This paper reveals usability issues of 28 assistive social robots, and feedbacks of user experience based on 41 units of assistive social robots. Based on the research articles scrutinized, the authors concluded that the key elements in the design and development of assistive social robots to improve acceptance of older persons were determined by three factors: functionality, usability and users’ experience. Functionality refers to ability of robots to serve the older persons. Usability is ease of use of the robots. It is an indicator on how successful of interaction between the robots and the users. To improve usability, robot designers should consider the limitations of older persons such as vision, hearing, and cognition capabilities when interacting with the robots. User experience reflects to perceptions, preferences and behaviors of users that occur before, during and after use the robots. Combination of superior functionality and usability lead to a good user experience in using the robots which in the end achieves satisfaction of older persons

COACHES Cooperative Autonomous Robots in Complex and Human Populated Environments

Public spaces in large cities are increasingly becoming complex and unwelcoming environments. Public spaces progressively become more hostile and unpleasant to use because of the overcrowding and complex information in signboards. It is in the interest of cities to make their public spaces easier to use, friendlier to visitors and safer to increasing elderly population and to citizens with disabilities. Meanwhile, we observe, in the last decade a tremendous progress in the development of robots in dynamic, complex and uncertain environments. The new challenge for the near future is to deploy a network of robots in public spaces to accomplish services that can help humans. Inspired by the aforementioned challenges, COACHES project addresses fundamental issues related to the design of a robust system of self-directed autonomous robots with high-level skills of environment modelling and scene understanding, distributed autonomous decision-making, short-term interacting with humans and robust and safe navigation in overcrowding spaces. To this end, COACHES will provide an integrated solution to new challenges on: (1) a knowledge-based representation of the environment, (2) human activities and needs estimation using Markov and Bayesian techniques, (3) distributed decision-making under uncertainty to collectively plan activities of assistance, guidance and delivery tasks using Decentralized Partially Observable Markov Decision Processes with efficient algorithms to improve their scalability and (4) a multi-modal and short-term human-robot interaction to exchange information and requests. COACHES project will provide a modular architecture to be integrated in real robots. We deploy COACHES at Caen city in a mall called “Rive de l’orne”. COACHES is a cooperative system consisting of ?xed cameras and the mobile robots. The ?xed cameras can do object detection, tracking and abnormal events detection (objects or behaviour). The robots combine these information with the ones perceived via their own sensor, to provide information through its multi-modal interface, guide people to their destinations, show tramway stations and transport goods for elderly people, etc.... The COACHES robots will use different modalities (speech and displayed information) to interact with the mall visitors, shopkeepers and mall managers. The project has enlisted an important an end-user (Caen la mer) providing the scenarios where the COACHES robots and systems will be deployed, and gather together universities with complementary competences from cognitive systems (SU), robust image/video processing (VUB, UNICAEN), and semantic scene analysis and understanding (VUB), Collective decision-making using decentralized partially observable Markov Decision Processes and multi-agent planning (UNICAEN, Sapienza), multi-modal and short-term human-robot interaction (Sapienza, UNICAEN

Supporting active and healthy aging with advanced robotics integrated in smart environment

The technological advances in the robotic and ICT fields represent an effective solution to address specific societal problems to support ageing and independent life. One of the key factors for these technologies is the integration of service robotics for optimising social services and improving quality of life of the elderly population. This chapter aims to underline the barriers of the state of the art, furthermore the authors present their concrete experiences to overcome these barriers gained at the RoboTown Living Lab of Scuola Superiore Sant'Anna within past and current projects. They analyse and discuss the results in order to give recommendations based on their experiences. Furthermore, this work highlights the trend of development from stand-alone solutions to cloud computing architecture, describing the future research directions

Robot-era project: Preliminary results on the system usability

The European project Robot-Era is an ambitious integrated project (FP7-ICT-2011.5.4), which objective is the development of advanced robotic services, integrated in intelligent environments, to provide independent living to older people. In order to guarantee the matching of the users� need and the demands, two loops of experimentation were conceived, in realistic and real setting. The aim of the paper is to described the methods applied and the main results coming from the first experimental loop, concerning the degree of usability of the interfaces and provide guidelines for testing socially assistive robots with older people. � Springer International Publishing Switzerland 2015

Adaptive modality selection algorithm in robot-assisted cognitive training

© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Interaction of socially assistive robots with users is based on social cues coming from different interaction modalities, such as speech or gestures. However, using all modalities at all times may be inefficient as it can overload the user with redundant information and increase the task completion time. Additionally, users may favor certain modalities over the other as a result of their disability or personal preference. In this paper, we propose an Adaptive Modality Selection (AMS) algorithm that chooses modalities depending on the state of the user and the environment, as well as user preferences. The variables that describe the environment and the user state are defined as resources, and we posit that modalities are successful if certain resources possess specific values during their use. Besides the resources, the proposed algorithm takes into account user preferences which it learns while interacting with users. We tested our algorithm in simulations, and we implemented it on a robotic system that provides cognitive training, specifically Sequential memory exercises. Experimental results show that it is possible to use only a subset of available modalities without compromising the interaction. Moreover, we see a trend for users to perform better when interacting with a system with implemented AMS algorithm.Peer ReviewedPostprint (author's final draft