Automatic Assessment of the Type and Intensity of Agitated Hand Movements

With increasing numbers of people living with dementia, there is growing interest in the automatic monitoring of agitation. Current assessments rely on carer observations within a framework of behavioural scales. Automatic monitoring of agitation can supplement existing assessments, providing carers and clinicians with a greater understanding of the causes and extent of agitation. Despite agitation frequently manifesting in repetitive hand movements, the automatic assessment of repetitive hand movements remains a sparsely researched field. Monitoring hand movements is problematic due to the subtle differences between different types of hand movements and variations in how they can be carried out; the lack of training data creates additional challenges. This paper proposes a novel approach to assess the type and intensity of repetitive hand movements using skeletal model data derived from video. We introduce a video-based dataset of five repetitive hand movements symptomatic of agitation. Using skeletal keypoint locations extracted from video, we demonstrate a system to recognise repetitive hand movements using discriminative poses. By first learning characteristics of the movement, our system can accurately identify changes in the intensity of repetitive movements. Wide inter-subject variation in agitated behaviours suggests the benefit of personalising the recognition model with some end-user information. Our results suggest that data captured using a single conventional RGB video camera can be used to automatically monitor agitated hand movements of sedentary patients

Recognition physical activities with optimal number of wearable sensors using data mining algorithms and deep belief network

© 2017 IEEE. Daily physical activities monitoring is benefiting the health care field in several ways, in particular with the development of the wearable sensors. This paper adopts effective ways to calculate the optimal number of the necessary sensors and to build a reliable and a high accuracy monitoring system. Three data mining algorithms, namely Decision Tree, Random Forest and PART Algorithm, have been applied for the sensors selection process. Furthermore, the deep belief network (DBN) has been investigated to recognise 33 physical activities effectively. The results indicated that the proposed method is reliable with an overall accuracy of 96.52% and the number of sensors is minimised from nine to six sensors

Détection des moments d'agitation chez les enfants autistes à l'aide de données physiologiques et d'accélération

Les enfants autistes présentent souvent des comportements agités qui peuvent avoir un impact négatif sur leur qualité de vie, ce qui est source de stress pour les familles et les soignants. La détection et la prévention de ces comportements sont des tâches fastidieuses et difficiles, c'est pourquoi le besoin de techniques d'assistance technologique se fait de plus en plus sentir. Ce mémoire vise à développer un modèle automatique pour détecter les comportements agités chez les enfants autistes en se basant sur des signaux physiologiques et d'accélération. Pour ce faire, la première étape de ce projet consiste à collecter des données auprès d'un enfant de sexe masculin âgé de neuf ans. Le dispositif utilisé s'appelle Empatica E4. Il s'agit d'un bracelet utilisé par les cliniciens pour recueillir différents types de signaux, notamment le débit sanguin, la fréquence cardiaque, l'activité électrodermale et l'accélération en trois dimensions. Par rapport aux travaux précédents, la collecte de données dans ce projet est effectuée pendant les activités quotidiennes de l'enfant. Malheureusement, l'ensemble des données collectées souffre d'un déséquilibre, avec moins de comportements agités que de comportements normaux. Après la collecte des signaux, des schémas de traitement ont été établis pour chaque signal, comprenant le filtrage et l'extraction des descripteurs. Comme les données ont été collectées dans un environnement de vie quotidienne et sous aucun protocole expérimental, la plupart des enregistrements ont été affecté par le bruit et les artefacts extérieurs. Nous avons examiné un large éventail de descripteurs pour chaque signal par rapport aux études précédentes, y compris des descripteurs dans le domaine temporel (statistique, …) et fréquentiel (amplitudes de la puissance spectrale, …). Les descripteurs extraits sont ensuite utilisés pour entraîner différents algorithmes d'apprentissage automatique afin de détecter les comportements agités. Ces algorithmes comprennent la Machine à Vecteurs de Support, la Forêt d'Arbres Décisionnels, l'Amplification du Gradient Extrême et le classificateur TabNet. Les résultats obtenus dans le cadre de ce projet sont prometteurs en fonction de précision, rappel, justesse,... En effet, le Boosting du Gradient Extrême a obtenu les meilleures performances en termes de métriques d'évaluation utilisées. De plus, notre approche a obtenu de meilleurs résultats qu'une étude similaire antérieure, même si leurs données étaient collectées au laboratoire. Ce travail contribue au développement des techniques automatisées pour aider les soignants à détecter et à prévenir les comportements agités des enfants autistes, ce qui permet d'améliorer la qualité de vie de ces enfants et de leurs familles

An affective computing and image retrieval approach to support diversified and emotion-aware reminiscence therapy sessions

A demência é uma das principais causas de dependência e incapacidade entre as pessoas idosas em todo o mundo. A terapia de reminiscência é uma terapia não farmacológica comummente utilizada nos cuidados com demência devido ao seu valor terapêutico para as pessoas com demência. Esta terapia é útil para criar uma comunicação envolvente entre pessoas com demência e o resto do mundo, utilizando as capacidades preservadas da memória a longo prazo, em vez de enfatizar as limitações existentes por forma a aliviar a experiência de fracasso e isolamento social. As soluções tecnológicas de assistência existentes melhoram a terapia de reminiscência ao proporcionar uma experiência mais envolvente para todos os participantes (pessoas com demência, familiares e clínicos), mas não estão livres de lacunas: a) os dados multimédia utilizados permanecem inalterados ao longo das sessões, e há uma falta de personalização para cada pessoa com demência; b) não têm em conta as emoções transmitidas pelos dados multimédia utilizados nem as reacções emocionais da pessoa com demência aos dados multimédia apresentados; c) a perspectiva dos cuidadores ainda não foi totalmente tida em consideração. Para superar estes desafios, seguimos uma abordagem de concepção centrada no utilizador através de inquéritos mundiais, entrevistas de seguimento, e grupos de discussão com cuidadores formais e informais para informar a concepção de soluções tecnológicas no âmbito dos cuidados de demência. Para cumprir com os requisitos identificados, propomos novos métodos que facilitam a inclusão de emoções no loop durante a terapia de reminiscência para personalizar e diversificar o conteúdo das sessões ao longo do tempo. As contribuições desta tese incluem: a) um conjunto de requisitos funcionais validados recolhidos com os cuidadores formais e informais, os resultados esperados com o cumprimento de cada requisito, e um modelo de arquitectura para o desenvolvimento de soluções tecnológicas de assistência para cuidados de demência; b) uma abordagem end-to-end para identificar automaticamente múltiplas informações emocionais transmitidas por imagens; c) uma abordagem para reduzir a quantidade de imagens que precisam ser anotadas pelas pessoas sem comprometer o desempenho dos modelos de reconhecimento; d) uma técnica de fusão tardia interpretável que combina dinamicamente múltiplos sistemas de recuperação de imagens com base em conteúdo para procurar eficazmente por imagens semelhantes para diversificar e personalizar o conjunto de imagens disponíveis para serem utilizadas nas sessões.Dementia is one of the major causes of dependency and disability among elderly subjects worldwide. Reminiscence therapy is an inexpensive non-pharmacological therapy commonly used within dementia care due to its therapeutic value for people with dementia. This therapy is useful to create engaging communication between people with dementia and the rest of the world by using the preserved abilities of long-term memory rather than emphasizing the existing impairments to alleviate the experience of failure and social isolation. Current assistive technological solutions improve reminiscence therapy by providing a more lively and engaging experience to all participants (people with dementia, family members, and clinicians), but they are not free of drawbacks: a) the multimedia data used remains unchanged throughout sessions, and there is a lack of customization for each person with dementia; b) they do not take into account the emotions conveyed by the multimedia data used nor the person with dementia’s emotional reactions to the multimedia presented; c) the caregivers’ perspective have not been fully taken into account yet. To overcome these challenges, we followed a usercentered design approach through worldwide surveys, follow-up interviews, and focus groups with formal and informal caregivers to inform the design of technological solutions within dementia care. To fulfil the requirements identified, we propose novel methods that facilitate the inclusion of emotions in the loop during reminiscence therapy to personalize and diversify the content of the sessions over time. Contributions from this thesis include: a) a set of validated functional requirements gathered from formal and informal caregivers, the expected outcomes with the fulfillment of each requirement, and an architecture’s template for the development of assistive technology solutions for dementia care; b) an end-to-end approach to automatically identify multiple emotional information conveyed by images; c) an approach to reduce the amount of images that need to be annotated by humans without compromising the recognition models’ performance; d) an interpretable late-fusion technique that dynamically combines multiple content-based image retrieval systems to effectively search for similar images to diversify and personalize the pool of images available to be used in sessions

Deep Multi-Model Fusion for Human Activity Recognition Using Evolutionary Algorithms

Machine recognition of the human activities is an active research area in computer vision. In previous study, either one or two types of modalities have been used to handle this task. However, the grouping of maximum information improves the recognition accuracy of human activities. Therefore, this paper proposes an automatic human activity recognition system through deep fusion of multi-streams along with decision-level score optimization using evolutionary algorithms on RGB, depth maps and 3d skeleton joint information. Our proposed approach works in three phases, 1) space-time activity learning using two 3D Convolutional Neural Network (3DCNN) and a Long Sort Term Memory (LSTM) network from RGB, Depth and skeleton joint positions 2) Training of SVM using the activities learned from previous phase for each model and score generation using trained SVM 3) Score fusion and optimization using two Evolutionary algorithm such as Genetic algorithm (GA) and Particle Swarm Optimization (PSO) algorithm. The proposed approach is validated on two 3D challenging datasets, MSRDailyActivity3D and UTKinectAction3D. Experiments on these two datasets achieved 85.94% and 96.5% accuracies, respectively. The experimental results show the usefulness of the proposed representation. Furthermore, the fusion of different modalities improves recognition accuracies rather than using one or two types of information and obtains the state-of-art results

A review on deep-learning-based cyberbullying detection

Bullying is described as an undesirable behavior by others that harms an individual physically, mentally, or socially. Cyberbullying is a virtual form (e.g., textual or image) of bullying or harassment, also known as online bullying. Cyberbullying detection is a pressing need in today’s world, as the prevalence of cyberbullying is continually growing, resulting in mental health issues. Conventional machine learning models were previously used to identify cyberbullying. However, current research demonstrates that deep learning surpasses traditional machine learning algorithms in identifying cyberbullying for several reasons, including handling extensive data, efficiently classifying text and images, extracting features automatically through hidden layers, and many others. This paper reviews the existing surveys and identifies the gaps in those studies. We also present a deep-learning-based defense ecosystem for cyberbullying detection, including data representation techniques and different deep-learning-based models and frameworks. We have critically analyzed the existing DL-based cyberbullying detection techniques and identified their significant contributions and the future research directions they have presented. We have also summarized the datasets being used, including the DL architecture being used and the tasks that are accomplished for each dataset. Finally, several challenges faced by the existing researchers and the open issues to be addressed in the future have been presented

Toward Sensor-Based Early Diagnosis of Cognitive Impairment of Elderly Adults in Smart-Home Environments using Poisson Process Models

Emerging sensor-based assessment in combination with machine learning methodologies provide the potential to revolutionize current practices of (early) diagnosis of dementia. The goal of this research is to detect cognitive impairment in elderly adults using sensor-based measures. Longitudinal time-series data of sensor signals are analyzed with advanced computational models and supervised machine learning algorithms to identify individuals with cognitive impairment. This research further designs novel computational models using Poisson Processes that can model subtle temporal changes in sensor-based measurements, therefore have the potential to provide more reliable descriptors of cognitive impairments compared to aggregate time-series measures. Our results indicate that the proposed approach can effectively distinguish between dementia and MCI based on the sensor features yielded by the Poisson Process. Sensor-based assessment that relies on the non-homogeneous Poisson Process is further found to be effective in differentiating between adults with dementia and healthy adults, and depicts better performance compared to expert-based assessment. Findings from this research have the potential to help detect the early onset of cognitive impairment for elderly adults, and demonstrate the ability of advanced computational models and machine learning techniques to predict one’s cognitive health, thus, contributing toward advancing aging-in-place

Towards developing a reliable medical device for automated epileptic seizure detection in the ICU

Abstract. Epilepsy is a prevalent neurological disorder that affects millions of people globally, and its diagnosis typically involves laborious manual inspection of electroencephalography (EEG) data. Automated detection of epileptic seizures in EEG signals could potentially improve diagnostic accuracy and reduce diagnosis time, but there should be special attention to the number of false alarms to reduce unnecessary treatments and costs. This research presents a study on the use of machine learning techniques for EEG seizure detection with the aim of investigating the effectiveness of different algorithms in terms of high sensitivity and low false alarm rates for feature extraction, selection, pre-processing, classification, and post-processing in designing a medical device for detecting seizure activity in EEG data. The current state-of-the-art methods which are validated clinically using large amounts of data are introduced. The study focuses on finding potential machine learning methods, considering KNN, SVM, decision trees and, Random forests, and compares their performance on the task of seizure detection using features introduced in the literature. Also using ensemble methods namely, bootstrapping and majority voting techniques we achieved a sensitivity of 0.80 and FAR/h of 2.10, accuracy of 97.1% and specificity of 98.2%. Overall, the findings of this study can be useful for developing more accurate and efficient algorithms for EEG seizure detection medical device, which can contribute to the early diagnosis and treatment of epilepsy in the intensive care unit for critically ill patients