LONG-REMI : an AI-Based Technological Application to Promote Healthy Mental Longevity Grounded in Reminiscence Therapy

Altres ajuts: Fundación General CSIC (0551_PSL_6_E POCTEP)Reminiscence therapy (RT) consists of thinking about one's own experiences through the presentation of memory-facilitating stimuli, and it has as its fundamental axis the activation of emotions. An innovative way of offering RT involves the use of technology-assisted applications, which must also satisfy the needs of the user. This study aimed to develop an AI-based computer application that recreates RT in a personalized way, meeting the characteristics of RT guided by a therapist or a caregiver. The material guiding RT focuses on intangible cultural heritage. The application incorporates facial expression analysis and reinforcement learning techniques, with the aim of identifying the user's emotions and, with them, guiding the computer system that emulates RT dynamically and in real time. A pilot study was carried out at five senior centers in Barcelona and Portugal. The results obtained are very positive, showing high user satisfaction. Moreover, the results indicate that the high frequency of positive emotions increased in the participants at the end of the intervention, while the low frequencies of negative emotions were maintained at the end of the intervention

LONG-REMI: An AI-based technological application to promote healthy mental longevity grounded in reminiscence therapy

Reminiscence therapy (RT) consists of thinking about one’s own experiences through the presentation of memory-facilitating stimuli, and it has as its fundamental axis the activation of emotions. An innovative way of offering RT involves the use of technology-assisted applications, which must also satisfy the needs of the user. This study aimed to develop an AI-based computer application that recreates RT in a personalized way, meeting the characteristics of RT guided by a therapist or a caregiver. The material guiding RT focuses on intangible cultural heritage. The application incorporates facial expression analysis and reinforcement learning techniques, with the aim of identifying the user’s emotions and, with them, guiding the computer system that emulates RT dynamically and in real time. A pilot study was carried out at five senior centers in Barcelona and Portugal. The results obtained are very positive, showing high user satisfaction. Moreover, the results indicate that the high frequency of positive emotions increased in the participants at the end of the intervention, while the low frequencies of negative emotions were maintained at the end of the intervention.This research was supported by the Fundación General CSIC (FGCSIC), the Programa para una Sociedad Longeva (0551_PSL_6_E POCTEP), and the Fondo Europeo de Desarrollo Regional (FEDER).Peer ReviewedPostprint (published version

MemoriEase at the NTCIR-17 Lifelog-5 Task

We present the MemoriEase retrieval system used for our participation in the NTCIR Lifelog-5 Task. We report our method to address the lifelog retrieval problem and discuss our official results of the MemoriEase at Lifelog-5 task. We originally introduced the MemoriEase system for the Lifelog Search Challenge (LSC) as an interactive lifelog retrieval system. We have modified it to an automatic retrieval system to address the NTCIR Lifelog-5 Task. We propose the BLIP-2 model as the core embedding model to retrieve lifelog images from textual queries. The open-sourced Elasticsearch search engine serves as the main engine in the MemoriEase system. Some pre-processing and post-processing techniques are applied to adapt this system to an automatic version and improve the accuracy of retrieval results. Finally, we discuss the results of the system on the task, some limitations of the system, and lessons learned from participating in the Lifelog-5 task for further improvements for the system in the future

Temporal multimodal video and lifelog retrieval

The past decades have seen exponential growth of both consumption and production of data, with multimedia such as images and videos contributing significantly to said growth. The widespread proliferation of smartphones has provided everyday users with the ability to consume and produce such content easily. As the complexity and diversity of multimedia data has grown, so has the need for more complex retrieval models which address the information needs of users. Finding relevant multimedia content is central in many scenarios, from internet search engines and medical retrieval to querying one's personal multimedia archive, also called lifelog. Traditional retrieval models have often focused on queries targeting small units of retrieval, yet users usually remember temporal context and expect results to include this. However, there is little research into enabling these information needs in interactive multimedia retrieval. In this thesis, we aim to close this research gap by making several contributions to multimedia retrieval with a focus on two scenarios, namely video and lifelog retrieval. We provide a retrieval model for complex information needs with temporal components, including a data model for multimedia retrieval, a query model for complex information needs, and a modular and adaptable query execution model which includes novel algorithms for result fusion. The concepts and models are implemented in vitrivr, an open-source multimodal multimedia retrieval system, which covers all aspects from extraction to query formulation and browsing. vitrivr has proven its usefulness in evaluation campaigns and is now used in two large-scale interdisciplinary research projects. We show the feasibility and effectiveness of our contributions in two ways: firstly, through results from user-centric evaluations which pit different user-system combinations against one another. Secondly, we perform a system-centric evaluation by creating a new dataset for temporal information needs in video and lifelog retrieval with which we quantitatively evaluate our models. The results show significant benefits for systems that enable users to specify more complex information needs with temporal components. Participation in interactive retrieval evaluation campaigns over multiple years provides insight into possible future developments and challenges of such campaigns

Automatic reminiscence therapy for dementia

With people living longer than ever, the number of cases with dementia such as Alzheimer's disease increases steadily. It affects more than 46 million people worldwide, and it is estimated that in 2050 more than 100 million will be affected. While there are no effective treatments for these terminal diseases, therapies such as reminiscence, that stimulate memories from the past are recommended. Currently, reminiscence therapy takes place in care homes and is guided by a therapist or a carer. In this work, we present an AI-based solution to automate the reminiscence therapy. This consists of a dialogue system that uses photos of the users as input to generate questions about their life. Overall, this paper presents how reminiscence therapy can be automated by using deep learning, and deployed to smartphones and laptops, making the therapy more accessible to every person affected by dementia.Peer Reviewe