Identification of the presence of ischaemic stroke lesions by means of texture analysis on brain magnetic resonance images

Study funding This work was funded by the Row Fogo Charitable Trust (MVH, VGC) grant no. BRO-D.FID3668413, and the Wellcome Trust (patient recruitment, scanning, primary study Ref No. 088134/Z/09). The study was conducted independently of the funders who do not hold the data and did not participate in the study design or analyses. The Lothian Birth Cohort 1936 is funded by Age UK (Disconnected Mind grant) and the Medical Research Council (MRC; MR/M01311/1, G1001245, 82800), and the latter supported BSA. IJD was supported by the Centre for Cognitive Ageing and Cognitive Epidemiology, which is funded by the MRC and the Biotechnology and Biological Sciences Research Council (MR/K026992/1). David Moratal acknowledges financial support from the Spanish Ministerio de Economía y Competitividad (MINECO) and FEDER funds under Grant BFU2015-64380-C2-2-R, and from the Conselleria d'Educació, Investigació, Cultura i Esport, Generalitat Valenciana (grants AEST/2017/013 and AEST/2018/021). Rafael Ortiz-Ramón was supported by grant ACIF/2015/078 and grant BEFPI/2017/004 from the Conselleria d’Educació, Investigació, Cultura i Esport of the Valencian Community (Spain).Peer reviewedPublisher PD

Hippocampal representations for deep learning on Alzheimer’s disease

Deep learning offers a powerful approach for analyzing hippocampal changes in Alzheimer’s disease (AD) without relying on handcrafted features. Nevertheless, an input format needs to be selected to pass the image information to the neural network, which has wide ramifications for the analysis, but has not been evaluated yet. We compare five hippocampal representations (and their respective tailored network architectures) that span from raw images to geometric representations like meshes and point clouds. We performed a thorough evaluation for the prediction of AD diagnosis and time-to-dementia prediction with experiments on an independent test dataset. In addition, we evaluated the ease of interpretability for each representation–network pair. Our results show that choosing an appropriate representation of the hippocampus for predicting Alzheimer’s disease with deep learning is crucial, since it impacts performance and ease of interpretation

Geometric deep learning for Alzheimer's disease analysis

Alzheimer’s Disease (AD) represents between 50-70% of the cases of dementia, which translates in around 25-35 million people affected by this disease. During its development, patients suffering from AD experience an irreversible cognitive decline, which limits their autonomy on their daily lives. While many of the causes of AD are still unknown, researchers have noticed a abnormal amyloid deposition and neurofibrillary tangles that will start affecting the short-term memory of the patient, together with other cognitive functions. In fact, these pathophysiological changes start taking place even before the patient experiences the first symptoms. One of the structures that is first affected by the disease is the hippocampus. During the development of AD, this part of the brain experiences an irregular deformation that affects its capabilities of forming new memories. Therefore, many clinical work has set a focus on studying this structure and its evolution along the disease. Identifying the changes it suffers can help us understand better the causes of the patient's cognitive decline. Given the complexity that characterizes AD, identifying patterns during its development is still a cumbersome task for physicians.Thus, aiding the diagnosis and prognosis of the disease using Deep Learning methods can be highly beneficial, as seen for other medical applications. In particular, if the focus is set on single structures (e.g. the hippocampus) Geometric Deep Learning offers a set of models that are best suited for 3D shape representations. We believe these methods can help doctors identify abnormalities in the structure that can lead to AD in the future. In this work, we first study the capabilities of current Geometric Deep Learning methods in diagnosing patients suffering from AD, by only looking at the hippocampus. We start by studying one of the simplest 3d representations, point clouds. We continue by comparing this representation to other non-Euclidean representations, such as meshes, and also Euclidean ones (e.g. 3d masks). We observe that meshes are one of the optimal ways of representing 3d structures for capturing fine-grained changes, but they carry additional pre-processing steps that Euclidean representations do not require. Finally, once we have confirmed that Geometric Deep Learning, particularly mesh neural networks, can properly capture the effects of AD on the hippocampus, we expand their application to longitudinal analysis of the structure. We propose a new temporal model based on Spiral Resnet and Transformers that sets a new state-of-the-art for the task of predicting longitudinal trajectories of the hippocampus. We also evaluated the effect that imputing missing longitudinal data has on detecting subjects that are developping to AD. Our experiments show an increase of a 3% in distinguishing between converting and stable trajectories

Real-time human action and gesture recognition using skeleton joints information towards medical applications

Des efforts importants ont été faits pour améliorer la précision de la détection des actions humaines à l’aide des articulations du squelette. Déterminer les actions dans un environnement bruyant reste une tâche difficile, car les coordonnées cartésiennes des articulations du squelette fournies par la caméra de détection à profondeur dépendent de la position de la caméra et de la position du squelette. Dans certaines applications d’interaction homme-machine, la position du squelette et la position de la caméra ne cessent de changer. La méthode proposée recommande d’utiliser des valeurs de position relatives plutôt que des valeurs de coordonnées cartésiennes réelles. Les récents progrès des réseaux de neurones à convolution (RNC) nous aident à obtenir une plus grande précision de prédiction en utilisant des entrées sous forme d’images. Pour représenter les articulations du squelette sous forme d’image, nous devons représenter les informations du squelette sous forme de matrice avec une hauteur et une largeur égale. Le nombre d’articulations du squelette fournit par certaines caméras de détection à profondeur est limité, et nous devons dépendre des valeurs de position relatives pour avoir une représentation matricielle des articulations du squelette. Avec la nouvelle représentation des articulations du squelette et le jeu de données MSR, nous pouvons obtenir des performances semblables à celles de l’état de l’art. Nous avons utilisé le décalage d’image au lieu de l’interpolation entre les images, ce qui nous aide également à obtenir des performances similaires à celle de l’état de l’art.There have been significant efforts in the direction of improving accuracy in detecting human action using skeleton joints. Recognizing human activities in a noisy environment is still challenging since the cartesian coordinate of the skeleton joints provided by depth camera depends on camera position and skeleton position. In a few of the human-computer interaction applications, skeleton position, and camera position keep changing. The proposed method recommends using relative positional values instead of actual cartesian coordinate values. Recent advancements in CNN help us to achieve higher prediction accuracy using input in image format. To represent skeleton joints in image format, we need to represent skeleton information in matrix form with equal height and width. With some depth cameras, the number of skeleton joints provided is limited, and we need to depend on relative positional values to have a matrix representation of skeleton joints. We can show the state-of-the-art prediction accuracy on MSR data with the help of the new representation of skeleton joints. We have used frames shifting instead of interpolation between frames, which helps us achieve state-of-the-art performance

An affective computing and image retrieval approach to support diversified and emotion-aware reminiscence therapy sessions

A demência é uma das principais causas de dependência e incapacidade entre as pessoas idosas em todo o mundo. A terapia de reminiscência é uma terapia não farmacológica comummente utilizada nos cuidados com demência devido ao seu valor terapêutico para as pessoas com demência. Esta terapia é útil para criar uma comunicação envolvente entre pessoas com demência e o resto do mundo, utilizando as capacidades preservadas da memória a longo prazo, em vez de enfatizar as limitações existentes por forma a aliviar a experiência de fracasso e isolamento social. As soluções tecnológicas de assistência existentes melhoram a terapia de reminiscência ao proporcionar uma experiência mais envolvente para todos os participantes (pessoas com demência, familiares e clínicos), mas não estão livres de lacunas: a) os dados multimédia utilizados permanecem inalterados ao longo das sessões, e há uma falta de personalização para cada pessoa com demência; b) não têm em conta as emoções transmitidas pelos dados multimédia utilizados nem as reacções emocionais da pessoa com demência aos dados multimédia apresentados; c) a perspectiva dos cuidadores ainda não foi totalmente tida em consideração. Para superar estes desafios, seguimos uma abordagem de concepção centrada no utilizador através de inquéritos mundiais, entrevistas de seguimento, e grupos de discussão com cuidadores formais e informais para informar a concepção de soluções tecnológicas no âmbito dos cuidados de demência. Para cumprir com os requisitos identificados, propomos novos métodos que facilitam a inclusão de emoções no loop durante a terapia de reminiscência para personalizar e diversificar o conteúdo das sessões ao longo do tempo. As contribuições desta tese incluem: a) um conjunto de requisitos funcionais validados recolhidos com os cuidadores formais e informais, os resultados esperados com o cumprimento de cada requisito, e um modelo de arquitectura para o desenvolvimento de soluções tecnológicas de assistência para cuidados de demência; b) uma abordagem end-to-end para identificar automaticamente múltiplas informações emocionais transmitidas por imagens; c) uma abordagem para reduzir a quantidade de imagens que precisam ser anotadas pelas pessoas sem comprometer o desempenho dos modelos de reconhecimento; d) uma técnica de fusão tardia interpretável que combina dinamicamente múltiplos sistemas de recuperação de imagens com base em conteúdo para procurar eficazmente por imagens semelhantes para diversificar e personalizar o conjunto de imagens disponíveis para serem utilizadas nas sessões.Dementia is one of the major causes of dependency and disability among elderly subjects worldwide. Reminiscence therapy is an inexpensive non-pharmacological therapy commonly used within dementia care due to its therapeutic value for people with dementia. This therapy is useful to create engaging communication between people with dementia and the rest of the world by using the preserved abilities of long-term memory rather than emphasizing the existing impairments to alleviate the experience of failure and social isolation. Current assistive technological solutions improve reminiscence therapy by providing a more lively and engaging experience to all participants (people with dementia, family members, and clinicians), but they are not free of drawbacks: a) the multimedia data used remains unchanged throughout sessions, and there is a lack of customization for each person with dementia; b) they do not take into account the emotions conveyed by the multimedia data used nor the person with dementia’s emotional reactions to the multimedia presented; c) the caregivers’ perspective have not been fully taken into account yet. To overcome these challenges, we followed a usercentered design approach through worldwide surveys, follow-up interviews, and focus groups with formal and informal caregivers to inform the design of technological solutions within dementia care. To fulfil the requirements identified, we propose novel methods that facilitate the inclusion of emotions in the loop during reminiscence therapy to personalize and diversify the content of the sessions over time. Contributions from this thesis include: a) a set of validated functional requirements gathered from formal and informal caregivers, the expected outcomes with the fulfillment of each requirement, and an architecture’s template for the development of assistive technology solutions for dementia care; b) an end-to-end approach to automatically identify multiple emotional information conveyed by images; c) an approach to reduce the amount of images that need to be annotated by humans without compromising the recognition models’ performance; d) an interpretable late-fusion technique that dynamically combines multiple content-based image retrieval systems to effectively search for similar images to diversify and personalize the pool of images available to be used in sessions

Recognition of Activities of Daily Living with Egocentric Vision: A Review.

Video-based recognition of activities of daily living (ADLs) is being used in ambient assisted living systems in order to support the independent living of older people. However, current systems based on cameras located in the environment present a number of problems, such as occlusions and a limited field of view. Recently, wearable cameras have begun to be exploited. This paper presents a review of the state of the art of egocentric vision systems for the recognition of ADLs following a hierarchical structure: motion, action and activity levels, where each level provides higher semantic information and involves a longer time frame. The current egocentric vision literature suggests that ADLs recognition is mainly driven by the objects present in the scene, especially those associated with specific tasks. However, although object-based approaches have proven popular, object recognition remains a challenge due to the intra-class variations found in unconstrained scenarios. As a consequence, the performance of current systems is far from satisfactory