De Backere, Femke

De Pestel, Stijn

De Turck, Filip

Elprama, S

Jacobs, A

Jewell, C

Mahieu, Christof

Nelis, Jelle

Ongenae, Femke

Simoens, Pieter

English

Ghent University Academic Bibliography

Personalized robot interactions to intercept behavioraldisturbances of people with dementiaF. Ongenae, F. De Backere, J. Nelis, S. De Pestel, C. Mahieu, S. Elprama, C. Jewell, A.Jacobs, P. Simoens, and F. De TurckGhent University-iMinds. BelgiumFemke.Ongenae@intec.ugent.beAbstract. People with Dementia (PwD) exhibit Behavioral Disturbances (BD)that can be alleviated by personalized interactions, revisiting memories and pro-moting comfort and quality of life. However, caregivers are unable to spend a lotof time on these interactions. This work-in-progress poster details the design anddeployment of a semantic Internet of Robotic Things (IoRT) platform that enablespersonalized interactions of a robot with a PwD to reduce and intercept BDs.1 IntroductionAlong with the aging population, the number of people with dementia (PwD) is increas-ing. PwD exhibit behavioral disturbances (BD) likewandering,mood and sleep disordersand aggression. These BDs can be prevented by non-pharmaceutical interventions, likepersonal interaction, revisiting personal memories and promoting comfort and quality oflife. However, because of increased strain on healthcare resources, a dwindling numberof caregivers needs to care for an increasing number of elderly. This inhibits the stafffrom allocating a lot of time to these interactions.A robot could help to provide such person-centric care by interacting with the PwDto prevent and alleviate BDs. By audio-visual stimuli or by engaging communication,robots can elicit memories with associated positive feelings that have a calming effect onPwD. This improves the well-being of the PwD. It can also be used to distract the PwDuntil the staff arrives in acute situations. As manifestations of dementia and the stimulithey react to, vary widely amongst PwD, a personalized approach is required. This leadsto more enjoyable experiences for the PwD, encourages cooperation and engagementand prevents that the PwD loses interest in the long-term [1, 2].In the WONDER project1, we are developing an Internet of Robotic Things (IoRT)platform [6] to enable personalized robot interactions with PwD to reduce and interceptBDs. In an IoRT platform the robot is integrated in a smart environment outfitted with avariability of sensors and wearables that capture the current context. A semantic cloudback-end then analyzes this captured information and combines it with other contextinformation sources (e.g. profile of the PwD) to extract valuable knowledge about thecontext and activities of the persons active in it. This derived knowledge is then usedto steer the actions of the robot. The IoRT platform autonomously detects when a PwDexhibits a BD and determines which personalized interactions should occur between therobot and this PwD.1 https://www.iminds.be/en/projects/wonder2Fig. 1. The designed IoRT Platform2 IoRT Platform OverviewThe overall architecture of the designed IoRT platform is visualized in Fig. 1.Sensors are integrated into the care setting that monitor the context. The PwDare equipped with wearables to track their location and walking patterns. To hidethe different protocols and interfaces of the devices, the DYAMAND middleware isused [4]. It allows application developers to discover, request info and control devices ina transparent manner. DYAMAND maps the output of the sensors on a uniform modeland wraps into a JSON syntax. We have extended DYAMAND with a plugin that mapsits internal model on the Semantic Sensor Network (SSN) Ontology2, which allowsto link the measured data to each other and known background info, e.g., the sensorlocation and type of data it monitors. The plugin converts the data from the sensors intoJSON-LD, which is then pushed to aKafkaMessaging Bus running in the cloud. Thereit can be dynamically picked up by various services interested in this type of sensor data.To design services that detect BDs and steer the robot, MASSIF [5] is used. Itenables the seamless collaboration of modular reasoning services to facilitate complexdecision making. As common knowledge model, the Ambient-Aware Continuous CareOntologies (ACCIO)3 are used. TheContext Adapter picks up the JSON-LDmessagesfrom Kafka and converts them to OWL individuals that adhere to the ACCIO ontology.As this ontology extends the SSN ontology, this is a straight-forward conversion. Theseindividuals are pushed on the Semantic Communication Bus (SCB), which enablesa publish-subscribe mechanism based on high-level semantic concepts. The servicesexpress the data they interested in by defining semantic filter rules, i.e., OWL Classexpressions. Reasoning is used by the SCB to match incoming data on the filter rules and2 https://www.w3.org/TR/vocab-ssn/ 3 http://ibcnservices.github.io/Accio-Ontology/3decide to which services it should be forwarded. The services push their conclusions onthe SCB. As such, services can be chained together in a flexible and data-driven manner.Per type of BD, a BD Detection Service is made, which registers a filter rulewith the SCB indicating the types of data it needs, e.g., noise or location data. Twoservices have already been defined, i.e., Wandering and Yelling Detection Services.They incorporate an extension of the ACCIO ontology that models domain-specificknowledge about dementia, e.g., the profiles of the PwD (e.g, historical informationabout exhibited BDs, severity of dementia or relationships with other PwD) and thelayout and activity schedule of the care center. This info is extracted from databasesand mapped on the ontology using Ontop4. These services incorporate probabilisticpattern recognition algorithms to infer with a particular probability whether a BD wasdetected [3, 7], i.e., a PwD is yelling or wandering. To further assess the certainty ofthis detection and derive the risk level of the detected BD, the probabilistic output iscombined with semantic reasoning algorithms on the background knowledge. The risklevel indicates whether the behavior is disturbing other people or is harming the PwDhim/herself, e.g., he/she has been wandering for too long and is becoming tired. Thedetected BD, the confidence and the risk level are pushed back on the SCB. The use ofMASSIF allows that new services can easily be incorporated to detect other BDs or usealternative detection algorithms.The Task Assignment Service subscribes itself to detected BDs and current contextto determine the appropriate IoRT actuation strategy. Possible actions are alerting acaregiver, sending a robot or a combination of both. For example, when a BD is detectedwith low probability and risk level, the robot is sent to the scene to gather additional infothrough conversation and its camera. However, if yelling is detected in area where peopleare present that have a history of violent behavior, the staff is immediately alerted. Whenthe robot is sent, theRobot Task Selection Service decides which task this robot shouldperform. This decision algorithm adapts itself to the current context and the personalityof the PwD. For example, the robot should not make a lot of noise at night or a PwDmight respond better to a song than to conversation.The Task Executor is responsible for executing the assigned tasks. The Staff Com-munication sends the alerts to the caregivers. The Robot Controller splits up the robottasks into individual robotic actions and the Robot Manager decides to which robotthese commands should be sent. The latter keeps track of the charging schemes, ac-tivities and locations of the robots. The Path Planning Service finds an optimal pathto the PwD exhibiting the BD. The Robot Manager and the Path Planning Service usean Environment Model that encodes the floor plan of the care institution. The status,e.g., executing or finished, of the robot actions are communicated to MASSIF to detectwhether the BD has been alleviated or whether additional measures should be taken.3 IoRT Platform DeploymentThe platform is deployed in two care institutions in Flanders. The rooms of 26 elderlyare equipped with a GrovePi5 to measure environmental parameters, i.e., luminance,4 http://ontop.inf.unibz.it/ 5 http://www.dexterindustries.com/grovepi/4temperature, sound, loudness, movement and humidity. The Xetal Kinsei6 is used totrack the number of people in the room and perform localization. Doors are equippedwith enocean7 sensors to detect whether they are opened or closed. The common roomsare also equipped with GrovePi systems. Finally, the PwD wear a Xiami Mi Band8 totrack their movement. Their walking aids, e.g., wheelchair, are outfitted with a BLE tag.These wearables and tags are picked up by seven Raspberry Pi 3 gateways, enabled withBLE bluetooth, spread out across the institution. Each of the GrovePi and Raspberry Pigateways runs a DYAMAND instance to uniformly expose the data. The ZORA9, whichcombines the NAO with optimized software for healthcare settings, interacts with thePwD.The caregivers use a smartphone application to indicate when a PwD is exhibitinga BD. This labeled dataset will allow us to train and optimize the algorithms of theBD Detection Services. Decision tree workshops [8] have been organized together withthe staff members to derive which parameters, profile and context info are importantto take into account to assess the reliability of a BD detection, to decide who shouldbe alerted and which action should be performed by ZORA. These workshops shapedthe algorithms in the Task Assignment and Robot Task Selection Services. User testsare being performed to see how the PwD react to certain robot actions in particularsituations to further optimize these services.The next research step consists of using the collected dataset to optimize the BDDetection Services. Afterwards, the IoRT platform will be used in both institutions tosteer the robots and we will study the effect this has on the behavior of the PwD.Acknowledgment: This research was realized in the WONDER project, co-funded byVLAIO, iMinds, ZoraBots, Xetal, WZC De Vijvers and Weverbos.References1. A. Sabelli, et al.: A conversational robot in an elderly care center: an ethnographic study. In:HRI Conference. pp. 37–44 (2011)2. Dautenhahn, K.: Robots we like to live with? a developmental perspective on a personalized,life-long robot companion. In: ROMANWorkshop. pp. 17–22 (2004)3. F. Palumbo, et al.: Multisensor data fusion for activity recognition based on reservoir comput-ing. EvAAL pp. 24–35 (2013)4. J. Nelis, et al.: Dyamand: dynamic, adaptive management of networks and devices. In: IEEELCN Conference. pp. 192–195 (2012)5. P. Bonte, et al.: The massif platform: a modular & semantic platform for the development offlexible iot services. KAIS (2016)6. P. Simoens, et al.: Internet of robotic things: Context-aware and personalized interventions ofassistive social robots. In: IEEE CloudNet conference (2016)7. Q. Lin, et al.: Managing elders' wandering behavior using sensors-based solutions: a survey.IJGE 8(2), 49–55 (2014)8. S. Elprama, et al.: Opportunities to use robots to alleviate behavioral disturbances of nursinghome residents. In: ISG Conference (2016)6 http://xetal.net/7 https://www.enocean.com/en/8 http://www.mi.com/sg/miband/9 http://zorarobotics.be/

Personalized robot interactions to intercept behavioral disturbances of people with dementia

https://biblio.ugent.be/publication/8500131/file/8500132

Personalized robot interactions to intercept behavioral disturbances of people with dementia

Abstract

Similar works

Full text

Available Versions

Ghent University Academic Bibliography