The development of robots that are truly sociable requires understanding how human interactions can be applied to the interaction between humans and robots. A sociable robot must be able to interact with people taking into account aspects like verbal and non-verbal communications (emotions, postures, gestures). This work presents a social robot which main goal is to provide assistance to older people in carrying out their daily activities (through suggestions or reminders). In addition, the robot presents non-verbal communications like perceiving emotions and displaying human identifiable emotions in order to express empathy. A prototype of the robot is being tested in a daycare centre in the northern area of Portugal.This work is partially supported by the MINECO/FEDER TIN2015-65515-C4-
1-R and the FPI grant AP2013-01276 awarded to Jaime-Andres Rincon. Angelo
Costa thanks the FCT - Fundacao para a Ciencia e Tecnologia for the PostDoc grant SFRH/BPD/102696/2014. This work is supported by FCT within
the projects UID/CEC/00319/2013 and COMPETE: POCI-01-0145-FEDER007043

Carrascosa, Carlos

Costa, Ângelo Gonçalo Araújo Silva

Julian, Vicente

Martin, Alberto

Novais, Paulo

Rincon, Jaime A.

Information Technology and Nanotechnology

English

Universidade do Minho: RepositoriUM

EmIR: An Emotional Intelligent Robot AssistantJaime A. Rincon, Alberto Martin, Angelo Costa, Paulo Novais, Vicente Julian,and Carlos Carrascosa1 Universitat Politècnica de València. D. Sistemas Informáticos y Computaciónjrincon, vinglada, carrasco@dsic.upv.es2 Centro ALGORITMI, Escola de Engenharia, Universidade do Minho, Braga acosta,pjon@di.uminho.ptAbstract. The development of robots that are truly sociable requiresunderstanding how human interactions can be applied to the interactionbetween humans and robots. A sociable robot must be able to interactwith people taking into account aspects like verbal and non-verbal com-munications (emotions, postures, gestures). This work presents a socialrobot which main goal is to provide assistance to older people in car-rying out their daily activities (through suggestions or reminders). Inaddition, the robot presents non-verbal communications like perceivingemotions and displaying human identifiable emotions in order to expressempathy. A prototype of the robot is being tested in a daycare centre inthe northern area of Portugal.Keywords: Social Robots, Emotional Models, Ambient Assisted Living1 IntroductionGlobally, the elderly population is increasing, according to demographic projec-tions [1]. The OMS prdeicts that the amount of people aged over 60 is expectedto double between 2000 and 2050 [2]. As less developed countries start to evolve,this trend is onset immediately [2]. A common societal issue that emerges froma rapid elderly population growth is the exponential demand or care (medicaland otherwise).The common human resources needed are healthcare professionals, includingformal and informal caregivers. Which currently are in short supply, and there isno indication of increase of the caregivers numbers in the foreseeable future. Thelack of these professionals represents a serious issue. One way of dealing withthis issue is using technology that empowers people to overcome problems thatthey encounter on their daily lives. Various areas can be of help to the elderly,like Social Robotics [3], Virtual Assistants [4], Artificial Intelligence [5], etc.These areas try to solve these problems by introducing new technologies thatcan help older people living alone or in nursing homes. Some of these solutionsinclude voice assistants such as Amazon’s Alexa, Google Voice Assitant and Ap-ple’s HomePod. However, other emerging technologies such as assistant robotsand therapy robots are becoming increasingly popular. These devices integratespeech recognition, artificial vision, text chat and gestural interactions. Thesemechanisms help users to interact with these devices. However, in human in-teraction the way that emotion is expressed plays an important role, mainlybecause emotions are themselves a communication channel.In this paper we present a social robot, called EmIR, which main goal isto provide assistance to the elderly performing their daily activities, presentinghuman-like features like perceiving emotions of a group of people and displayinghuman identifiable emotions. Moreover, our social robot makes use of an e-Healthplatform (Cognitive Life Assistant - CLA) which integrates a persuasive modulethat makes use of argumentative techniques similar to reasoning procedures thatphysicians and caregivers use to recommend activities to patients.The rest of the paper is structured as follows. Section 2 analyzes previousworks in the area of assistants robots. Section 3 presents in detail the mainfunctionalities of the robot with particular emphasis on the emotion detectionand the user interaction. Section 4 describes the hardware employed to buildthe robot and it illustrates the functionalities of the robot through an example.Finally, the conclusions are presented in Section 5.2 Related WorkRecently, it is observable the increase on interest in assistive robots and theirdevelopment. Current sociological shift demands technological solutions that areable to interact with elderly people. Thus, the sudden increase of the number ofrobots available that have as goal solving this sociological issue. Furthermore,they are increasingly refined and advanced in terms of aspect and features, beingmore human-like in both fields.In recent years, we have seen a growing interest in robots, many of whichare available to people. These robots have nice looks, a powerful background ofartificial intelligence, navigation in complex environments, artificial vision, etc.Some of them, such as Pepper [6] and Romeo [7] from Aldebaran or Aido [8] orBuddy [8] from Frog Robotics, are presented as a new generation of home robots.One of the main applications of these robots is to assist older people in their dailylife. Usually, these robots are used as a form of therapy or health care. In therapyfield, we can find the Paro robot [9]. The Paro robot is an advanced interactiverobot developed by AIST, which offers the well-known benefits of animal therapyto be administered to patients in environments such as hospitals. This robot wasbuilt to be a companion to children and elderly, prompting a positive emotionalresponse due to its visual aspect. In the healthcare field we can find the Maburobot of Catalia Health, who learns over time about the personality, interests andtherapeutic challenges of each patient. This allows Mabu to create conversationsadapted to each patient. The structure of these conversations is based on well-known behavioral models of psychology to promote behavioral change.From these projects we have observed that while the physical aspect, and itsunderlying impact (as displayed by PARO), is very advanced and the target of acareful development, they lack in terms of functionality. The actions that makesone human and their emotions are not explored by these projects and we believethat after the initial impact of the visual aspect it is what makes them reallyhuman-like. We, as humans, strive for human empathy, and is what our projectaims for.3 Emotional Robot Assistant DescriptionIn this section we describe the proposed social robot, called EmIR (EmotionalIntelligent Robot). This robot has been developed in order to provide assistanceto the elderly to perform their daily activities. To validate the proposed robot, aprototype is being tested in a real scenario, in a daycare institution in the northof Portugal.The main features of the robot are the following: perceives the emotionsof each individual and also calculates the social emotion of a group of people;displays human identifiable emotions according to the emotional states of theusers; assists users recommending healthy activities and also, remembering whento do them; and finally, persuades users by giving them arguments to justifyrecommended actions. The robot was built using a lattepanda board3 and anArduino mega 2560 board4. These development boards are used to control thedifferent engines and also to detect emotions and to identify people. Figure 1shows a picture of the robot with its different elements.Fig. 1: Robot hardware.3 http://www.lattepanda.com/4 https://store.arduino.cc/arduino-mega-2560-rev3The functionalities of the proposed robot are divided into three main moduleswhich provides the services that our robot uses to perceive and interact with theenvironment. The first two modules are the Emotion Detection and the EmotionDisplay modules which are in charge of detecting, processing and displayingemotions. On the other hand, the other module is the User Interaction modulewhich has been implemented using the CLA framework [10]. The CLA frameworkimproves the user experience and gives a fluid visual interface to the robot.4 Application ExampleIn order to illustrate the behavior of the robot, this section presents an examplewhich describes the different processes made by the robot as an emotional-basedpersuasive recommender. In the proposed example, the robot will interact withhumans by identifying them, detecting their emotions and suggesting activities.Figure 2 shows the different processes followed by the robot for supportingthe described interaction.Fig. 2: A schema of the main processes followed by the robotIn a general view, the robot will follow the following steps:1. The way to initiate the interaction with the robot consists of saying the nameof the robot, which in our case is EmIR (Emotional Interactive Robot).Assoon as the robot hears its name it starts interacting with the caller(s).2. The first step is to take a picture through its webcam. This image is usedto identify the person and to track the face. If the robot does not know theperson, the robot initiates the following dialogue (as an example):– EmIR: I’m sorry, but I don’t know you.– EmIR: What’s your name?.– User: Hi, my name is Jaime.3. After this, the image is then introduced as input into a neural convolutionalnetwork that returns the emotional state. If the robot detects more thanone face in the same image, the process is repeated for each face. Once theemotional state of all the users are obtained, the robot will adapt its emotionin an attempt to show empathy. To do this the robot can move its eyebrowsand change the lights of its cheeks.4. Next, the robot makes use of its persuasive recommender system. It canrecommend the following items:– Events that promote active aging. The suggestion of events follows aset of configurations (age, likes, medical condition, etc.) that make themappropriate for each user.– Activities on the user calendar, e.g., medication reminders, medical ap-pointments.During this recommendation process, a dialogue with the user is startedwhere the user can accept or refuse the suggestions. If the user refuses a sug-gestion, the robot tries to suggest another activity or generates argumentsin order to persuade the user. In parallel, it keeps identifying the user’s emo-tion and producing visual cues (eyebrows, etc.) to enforce the informationdelivered.5. After this process, the robot goes to a waiting state until it is requested againby the user. In the meanwhile the robot presents practical information likethe current weather state or news, in an effort to interact with the user(s).5 Conclusions and future workThis paper has presented the EmIR robot, a low cost desktop robot devised tointeract with humans taking into account the emotion of the people it interactswith and even being able to express some emotion. The application area whereis assistance to an elderly community.In the interaction with people in this application area, the robot is able totalk with the persons it perceives and help those people by recommending newactivities along with giving reminders about scheduled activities.Although the robot is still in a prototype form, its design and abilities havebeen improved presenting human-like features like perceiving emotions of a groupof people and displaying human identifiable emotions. The functionalities of theproposed robot are divided into three main aspects. The first one is the EmotionDetection that allows the robot to estimate the emotional state of the peoplethat is in from of the robot. The second one is the Emotion Display, that allowsthe robot to express empathy with people according to the emotional statespreviously detected. The third and last one, is the User Interaction that allowsthe robot to enhance the user experience suggesting activities/events to the userbased on his/her profile and medical condition and trying to persuade the usersinto accepting the suggested activities/events.This work is being validated by workers and patients of a daycare centre in thenorthern area of Portugal. Specifically in the centre Centro Social Irmandade deS. Torcato. The validation is being performed through simple interactions withthe patients under the supervision of caregivers.As future work, we want to improve the detection of emotions by introducingspeech recognition as another input in the process of identifying emotional states.Another aspect to be introduced, and currently supported by CLA, is the directconnection with caregivers or doctors to share information on scheduled activitiesor events as well as the patient progress. Finally, we aim for the adaptation ofthe proposed system to people with disabilities who cannot use the touchscreen.6 AcknowledgementsThis work is partially supported by the MINECO/FEDER TIN2015-65515-C4-1-R and the FPI grant AP2013-01276 awarded to Jaime-Andres Rincon. AngeloCosta thanks the FCT - Fundacao para a Ciencia e Tecnologia for the Post-Doc grant SFRH/BPD/102696/2014. This work is supported by FCT withinthe projects UID/CEC/00319/2013 and COMPETE: POCI-01-0145-FEDER-007043.References1. United Nations, Department of Economic and Social Affairs, Population Division.World population ageing 2015. (ST/ESA/SER.A/390), 2015.2. United Nations, Department of Economic and Social Affairs, Population Division.World population prospects: The 2017 revision, key findings and advance tables.(ESA/P/WP/248), 2017.3. J. Broekens, M. Heerink, and H. Rosendal. Assistive social robots in elderly care:a review. Grontechnology, 8:94–103, 2009.4. Nicole C. Krämer. Social effects of virtual assistants. a review of empirical resultswith regard to communication. In Helmut Prendinger, James Lester, and MitsuruIshizuka, editors, Intelligent Virtual Agents, pages 507–508, Berlin, Heidelberg,2008. Springer Berlin Heidelberg.5. Eduardo Lupiani, Jose M. Juarez, Jose Palma, and Roque Marin. Monitoringelderly people at home with temporal case-based reasoning. Knowledge-BasedSystems, 134:116 – 134, 2017.6. Fumihide Tanaka, Kyosuke Isshiki, Fumiki Takahashi, Manabu Uekusa, RumikoSei, and Kaname Hayashi. Pepper learns together with children: Development ofan educational application. IEEE-RAS International Conference on HumanoidRobots, 2015-December:270–275, 2015.7. Giovanni Claudio, Fabien Spindler, and Francois Chaumette. Vision-based manip-ulation with the humanoid robot Romeo. In 2016 IEEE-RAS 16th InternationalConference on Humanoid Robots (Humanoids), pages 286–293. IEEE, nov 2016.8. Ester Martinez-Martin and Angel P. del Pobil. Personal Robot Assistants forElderly Care: An Overview, pages 77–91. Springer International Publishing, Cham,2018.9. K. Wada, T. Shibata, T. Saito, and K. Tanie. Effects of robot assisted activity toelderly people who stay at a health service facility for the aged. In 3 IEEE/RSJInternational Conference on Intelligent Robots and Systems (IROS), pages 2847–2852, 2003.10. Angelo Costa, José Carlos Castillo, Paulo Novais, Antonio Fernández-Caballero,and Ricardo Simões. Sensor-driven agenda for intelligent home care of the elderly.Expert Syst. Appl., 39(15):12192–12204, 2012.

EmIR: An emotional intelligent robot assistant

http://repositorium.sdum.uminho.pt/bitstream/1822/68777/1/2018%20EmIR%20An%20emotional%20intelligent%20robot%20assistantcomp.pdf

EmIR: An emotional intelligent robot assistant

Abstract

Similar works

Full text

Available Versions

Universidade do Minho: RepositoriUM