Recreating living experiences from past memories through virtual worlds for people with dementia

This paper describes a study aimed to understand the use of 3D virtual world (VW) technology to support life engagement for people with dementia in long-term care. Three versions of VW prototypes (reminiscence room, virtual tour and gardening) utilising gestured-base interaction were developed iteratively. These prototypes were tested with older residents (80+) with dementia in care homes and their caregivers. Data collection was based on observations of how the residents and care staff interacted collaboratively with the VW. We discussed in depth the use of VWs in stimulating past memories and how this technology could help enhance their sense of self through various means. We also highlighted key approaches in designing VWs to sustain attention, create ludic experiences and facilitate interaction for older people with dementia

Multimodal Signal Processing and Learning Aspects of Human-Robot Interaction for an Assistive Bathing Robot

We explore new aspects of assistive living on smart human-robot interaction (HRI) that involve automatic recognition and online validation of speech and gestures in a natural interface, providing social features for HRI. We introduce a whole framework and resources of a real-life scenario for elderly subjects supported by an assistive bathing robot, addressing health and hygiene care issues. We contribute a new dataset and a suite of tools used for data acquisition and a state-of-the-art pipeline for multimodal learning within the framework of the I-Support bathing robot, with emphasis on audio and RGB-D visual streams. We consider privacy issues by evaluating the depth visual stream along with the RGB, using Kinect sensors. The audio-gestural recognition task on this new dataset yields up to 84.5%, while the online validation of the I-Support system on elderly users accomplishes up to 84% when the two modalities are fused together. The results are promising enough to support further research in the area of multimodal recognition for assistive social HRI, considering the difficulties of the specific task. Upon acceptance of the paper part of the data will be publicly available

Design of VR app applied to cognitive training

L’objectiu principal d’aquest projecte és el disseny d’una aplicació de realitat virtual per millorar el tractament dels pacients amb deteriorament cognitiu lleu, així com estudiar els possibles avantatges que aquesta tecnologia pot proporcionar en aquest camp. Es va escollir la realitat virtual perquè permet augmentar la sensació d’immersió pel que fa a les tecnologies actuals. Actualment la realitat virtual s’està utilitzant amb aquest tipus de tractament i està aconseguint gran resultats amb els pacients. A més, mitjançant l’ús d’aquesta tècnica d’immersió visual, s’espera que ajudi a millorar la capacitat dels pacients davant nous problemes, com pot ser la iniciació a la realitat virtual, una qüestió fonamental que ajuda a la millora dels pacients que es troben en les primeres etapes de la malaltia. L’aplicació consisteix en un entorn de supermercat virtual on el pacient pot realitzar diverses proves. En aquesta hi haurà diferents nivells amb diverses complexitats, sempre després d’haver realitzat un tutorial previ. L’aplicació s’ha realitzat en dues fases diferents: primer es va crear el guió, amb col·laboració amb la unitat d’Alzheimer de l’Hospital Clínic. Els nivells de l’aplicació es van definir aquí. El següent va ser la realització de l’aplicació amb col·laboració amb la companyia Vysion 360. Per a la seva utilització per la unitat d’Alzheimer de l’Hospital Clínic, l’aplicació tenia que complir diferents criteris. En primer lloc, els nivells de dificultat tenen que ser suficients per realitzar un tractament a llarg termini. En segon lloc, per crear una bona experiència de immersió, l’entorn creat té que ser el més realista possible. Finalment, s’ha creat una base de dades local per guardar la informació de totes les sessions, utilitzat posteriorment en l’anàlisi de evolució dels pacients. Amb aquesta aplicació, s’espera que els resultats en els pacients amb deteriorament cognitiu lleu milloren respecte a les tècniques anteriors. Especialment gràcies a la gran experiència d’immersió aconseguida amb la realitat virtual, la qual ajuda a la concentració dels pacients durant el tractament

An Internet- and Kinect-Based Multiple Sclerosis Fitness Intervention Training With Pilates Exercises: Development and Usability Study

background: balance impairments are common in people with multiple sclerosis (MS), with reduced ability to maintain position and delayed responses to postural adjustments. Pilates is a popular alternative method for balance training that may reduce the rapid worsening of symptoms and the increased risk of secondary conditions (eg, depression) that are frequently associated with physical inactivity.objective: In this paper, we aimed to describe the design, development, and usability testing of MS Fitness Intervention Training (MS-FIT), a Kinect-based tool implementing Pilates exercises customized for MS. methods: MS-FIT has been developed using a user-centered design approach (design, prototype, user feedback, and analysis) to gain the target user's perspective. a team composed of 1 physical therapist, 2 game programmers, and 1 game designer developed the first version of MS-FIT that integrated the knowledge and experience of the team with MS literature findings related to pilates exercises and balance interventions based on exergames. MS-FIT, developed by using the Unity 3D (Unity Technologies) game engine software with kinect Sensor V2 for Windows, implements exercises for breathing, posture, and balance. Feedback from an Italian panel of experts in MS rehabilitation (neurologists, physiatrists, physical therapists, 1 statistician, and 1 bioengineer) and people with MS was collected to customize the tool for use in MS. The context of MS-FIT is traveling around the world to visit some of the most important cities to learn the aspects of their culture through pictures and stories. At each stay of the travel, the avatar of a Pilates teacher shows the user the exercises to be performed. Overall, 9 people with MS (n=4, 44% women; mean age 42.89, SD 11.97 years; mean disease duration 10.19, SD 9.18 years; Expanded Disability Status Scale score 3.17, SD 0.75) were involved in 3 outpatient user test sessions of 30 minutes; MS-FIT's usability was assessed through an ad hoc questionnaire (maximum value=5; higher the score, higher the usability) evaluating easiness to use, playability, enjoyment, satisfaction, and acceptance.Results: A user-centered design approach was used to develop an accessible and challenging tool for balance training. all people with MS (9/9, 100%) completed the user test sessions and answered the ad hoc questionnaire. the average score on each item ranged from 3.78 (SD 0.67) to 4.33 (SD 1.00), which indicated a high usability level. The feedback and suggestions provided by 64% (9/14) of people with MS and 36% (5/14) of therapists involved in the user test were implemented to refine the first prototype to release MS-FIT 2.0. Conclusions: The participants reported that MS-FIT was a usable tool. It is a promising system for enhancing the motivation and engagement of people with MS in performing exercise with the aim of improving their physical status