Sleep analysis for elderly care using a low-resolution visual sensor network

Nearly half of the senior citizens report difficulty initiating and maintaining sleep. Frequent visits to the bathroom in the middle of the night is considered as one of the major reasons for sleep disorder. This leads to serious diseases such as depression and diabetes. In this paper, we propose to use a network of cheap low-resolution visual sensors (30 x 30 pixels) for long-term activity analysis of a senior citizen in a service flat. The main focus of our research is on elderly behaviour analysis to detect health deterioration. Specifically, this paper treats the analysis of sleep patterns. Firstly, motion patterns are detected. Then, a rule-based approach on the motion patterns is proposed to determine the wake up time and sleep time. The nightly bathroom visit is identified using a classification-based model. In our evaluation, we performed experiments on 10 months of real-life data. The ground truth is collected from the diaries in which the senior citizen wrote down his sleep time and wake up time. The results show accurate extraction of the sleep durations with an overall Mean Absolute Error (MAE) of 22.91 min and Spearman correlation coefficient of 0.69. Finally, the nightly bathroom visits analysis indicate sleep disorder in several nights

A Lifelogging Platform Towards Detecting Negative Emotions in Everyday Life using Wearable Devices

Repeated experiences of negative emotions, such as stress, anger or anxiety, can have long-term consequences for health. These episodes of negative emotion can be associated with inflammatory changes in the body, which are clinically relevant for the development of disease in the long-term. However, the development of effective coping strategies can mediate this causal chain. The proliferation of ubiquitous and unobtrusive sensor technology supports an increased awareness of those physiological states associated with negative emotion and supports the development of effective coping strategies. Smartphone and wearable devices utilise multiple on-board sensors that are capable of capturing daily behaviours in a permanent and comprehensive manner, which can be used as the basis for self-reflection and insight. However, there are a number of inherent challenges in this application, including unobtrusive monitoring, data processing, and analysis. This paper posits a mobile lifelogging platform that utilises wearable technology to monitor and classify levels of stress. A pilot study has been undertaken with six participants, who completed up to ten days of data collection. During this time, they wore a wearable device on the wrist during waking hours to collect instances of heart rate (HR) and Galvanic Skin Resistance (GSR). Preliminary data analysis was undertaken using three supervised machine learning algorithms: Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA) and Decision Tree (DT). An accuracy of 70% was achieved using the Decision Tree algorithm

Mobile phones as medical devices in mental disorder treatment: an overview

Mental disorders can have a significant, negative impact on sufferers’ lives, as well as on their friends and family, healthcare systems and other parts of society. Approximately 25 % of all people in Europe and the USA experience a mental disorder at least once in their lifetime. Currently, monitoring mental disorders relies on subjective clinical self-reporting rating scales, which were developed more than 50 years ago. In this paper, we discuss how mobile phones can support the treatment of mental disorders by (1) implementing human–computer interfaces to support therapy and (2) collecting relevant data from patients’ daily lives to monitor the current state and development of their mental disorders. Concerning the first point, we review various systems that utilize mobile phones for the treatment of mental disorders. We also evaluate how their core design features and dimensions can be applied in other, similar systems. Concerning the second point, we highlight the feasibility of using mobile phones to collect comprehensive data including voice data, motion and location information. Data mining methods are also reviewed and discussed. Based on the presented studies, we summarize advantages and drawbacks of the most promising mobile phone technologies for detecting mood disorders like depression or bipolar disorder. Finally, we discuss practical implementation details, legal issues and business models for the introduction of mobile phones as medical devices

Detecting Negative Emotions During Real-Life Driving via Dynamically Labelled Physiological Data

Driving is an activity that can induce significant levels of negative emotion, such as stress and anger. These negative emotions occur naturally in everyday life, but frequent episodes can be detrimental to cardiovascular health in the long term. The development of monitoring systems to detect negative emotions often rely on labels derived from subjective self-report. However, this approach is burdensome, intrusive, low fidelity (i.e. scales are administered infrequently) and places huge reliance on the veracity of subjective self-report. This paper explores an alternative approach that provides greater fidelity by using psychophysiological data (e.g. heart rate) to dynamically label data derived from the driving task (e.g. speed, road type). A number of different techniques for generating labels for machine learning were compared: 1) deriving labels from subjective self-report and 2) labelling data via psychophysiological activity (e.g. heart rate (HR), pulse transit time (PTT), etc.) to create dynamic labels of high vs. low anxiety for each participant. The classification accuracy associated with both labelling techniques was evaluated using Linear Discriminant Analysis (LDA) and Support Vector Machines (SVM). Results indicated that classification of driving data using subjective labelled data (1) achieved a maximum AUC of 73%, whilst the labels derived from psychophysiological data (2) achieved equivalent performance of 74%. Whilst classification performance was similar, labelling driving data via psychophysiology offers a number of advantages over self-reports, e.g. implicit, dynamic, objective, high fidelity

PUSHING THE BOUNDARIES OF CONSUMER GRADE WEARABLE DEVICES IN HEALTH CARE FOR OLDER ADULTS

Background: The proliferation of wearable and mobile devices in recent years has led to the generation of unprecedented amounts of health-related data. Together with the growing population of older adults in Canada, the increasing adoption of these technologies created a momentous opportunity to improve the way we deliver, access, and interact with the health care system. Many have recognized the opportunity, yet there is a lack of evidence on how these devices and the growing size of health data can be used to transform health care and benefit us. In Chapter 2, a review of the literature was presented to identify the current evidence of wearable technology and gaps that exist in aging research. Based on the literature review, one promising way to use wearable devices is to assess frailty, which can contribute to improving care and enhancing aging-in-place. Chapter 3 summarizes key concepts related to wearable devices including mobile health, patient-generated health data, big data, predictive algorithms, machine learning, and artificial intelligence. While in-depth mathematical representation of these big data analytics is outside the scope of this dissertation, this chapter provides foundational information along with examples found in health care settings. Objective: The overall aim of this dissertation was to investigate possible use of consumer-grade wearable devices and the patient-generated health data to improve the health of older adults. Methods: This thesis is presented as three individual studies included in Chapters 4 to 6. Study 1 aimed to investigate use of wearable devices to predict and find associations with frailty for community-dwelling older adults receiving home care service. Participants were asked to wear wearable device for 8 days in their home environment and no supervision was provided. Frailty level was assessed using the Fried Frailty Index. Other variables were collected including Charlson Comorbidity Index, independence using the Katz Index, and home care service utilization level. A sequential stepwise feature selection method was used to determine variables that are fitted in multiple variable logistic regression model to predict frailty. Study 2 extended the investigation of possible use of wearable devices for understanding frailty by examining the relationship between wearable device data and frailty progression among critical illness survivors from an intensive care unit at Kingston General Hospital. Participants were assessed for frailty using the Clinical Frailty Scale three times; at admission, at hospital discharge, and at 4-weeks post-hospital discharge. The changes in frailty level between the three time points were used to identify association with wearable device data that was collected for 4 weeks post-hospital discharge. Demonstrating evidence for wearable devices and patient-generated health data in research does not guarantee its use in real life. In Study 3, a mixed method study was conducted to explore clinicians’ and older adults’ perceptions of patient-generated health data. Focus group interviews were conducted with older adults and health care providers from the Greater Toronto Area and the Kitchener-Waterloo region. A questionnaire that aimed to explore perceived usefulness of a range of different patient-generated health data was embedded in the study design. Focus group interviews were transcribed verbatim. Line by line coding was conducted on all interviews followed by thematic analysis. Results: Results from Study 1 indicate data generated from wearable devices are closely linked to frailty level. Results showed a significant difference between frail and non-frail participants in age (p<0.01), home care service utilization (p=0.012), daily step count (p=0.04), total sleep time (p=0.010), and deep sleep time (p<0.01). Total sleep time (r=0.41, p=0.012) and deep sleep time (r=0.53, p<0.01) were associated with frailty level. A receiver operating characteristics area under the curve of 0.90 was achieved using deep sleep time, sleep quality, age, and education level (Hosmer-Lemeshow p=0.88), demonstrating that data from wearable devices can augment the demographic and conventional clinical data in predicting frailty status. Results from Study 2 demonstrated that frailty level increases significantly following a critical illness (p=0.02). Frail survivors had significantly lower daily step counts (p=0.02). Daily step count (r=-0.72, p=0.04) and mean heart rate (r=-0.72, p=0.046) were strongly correlated with frailty level at admission and discharge. Mean standard deviation of heart rate was correlated with the change in frailty status from admission to 4-week follow-up (r=0.78, p<0.05). The results demonstrated a relationship between the worsening of frailty due to critical illness and the pattern of increasing step count (r=0.65, p=0.03) and heart rate (r=0.62, p=0.03) over the 4-week observation period. Results from Study 3 provided an understanding of what older adults and clinicians considered barriers to using patient-generated health data in their care and clinical settings. Four main themes were identified from the focus group interviews: influence of patient-generated health data on patient-provider trust; reliability of patient-generated health data; meaningful use of patient-generated health data and decision support system; and perceived clinical benefits and intrusiveness of patient-generated health data. Results from the questionnaire and focus group interviews demonstrated that older adults and clinicians perceived blood glucose, step count, physical activity, sleep, blood pressure, and stress level as the most useful data for managing their health and delivering high quality care. Discussion: This dissertation provides evidence for using consumer-grade wearable device to assess, monitor, and predict frailty for older adults who receive home care or survived critical illness. The possibility of using a wearable device to assess frailty can enable health care providers to obtain frailty information in a timely manner, which is challenging to acquire otherwise due to a lack of appropriate tools in primary care, ambulatory care, home and community care, critical illness care, and other sectors. There was a distinct relationship between failure to recover frailty level from critical illness and the pattern of daily step count and heart rate. This can enable early detection of critical illness survivors who may not return to pre-critical illness level. It can provide guidance to identify those who may benefit the most from follow-up visits and elevated treatment. To ensure the benefits of patient-generated health data are realized, it must be integrated into health care. There are technical challenges that prevent such integration and discussion around policies and regulations must begin to make progress. Conclusion: This dissertation demonstrated use of wearable devices to assess frailty and identified factors that can hinder the integration of patient-generated health data into health care. It opened a possibility of assessing frailty, expanding the boundaries of current use of consumer-grade wearable devices

personal stress level clustering and decision level smoothing to enhance the performance of ambulatory stress detection with smartwatches

n/

Evaluating the Reproducibility of Physiological Stress Detection Models

Recent advances in wearable sensor technologies have led to a variety of approaches for detecting physiological stress. Even with over a decade of research in the domain, there still exist many significant challenges, including a near-total lack of reproducibility across studies. Researchers often use some physiological sensors (custom-made or off-the-shelf), conduct a study to collect data, and build machine-learning models to detect stress. There is little effort to test the applicability of the model with similar physiological data collected from different devices, or the efficacy of the model on data collected from different studies, populations, or demographics. This paper takes the first step towards testing reproducibility and validity of methods and machine-learning models for stress detection. To this end, we analyzed data from 90 participants, from four independent controlled studies, using two different types of sensors, with different study protocols and research goals. We started by evaluating the performance of models built using data from one study and tested on data from other studies. Next, we evaluated new methods to improve the performance of stress-detection models and found that our methods led to a consistent increase in performance across all studies, irrespective of the device type, sensor type, or the type of stressor. Finally, we developed and evaluated a clustering approach to determine the stressed/not-stressed classification when applying models on data from different studies, and found that our approach performed better than selecting a threshold based on training data. This paper\u27s thorough exploration of reproducibility in a controlled environment provides a critical foundation for deeper study of such methods, and is a prerequisite for tackling reproducibility in free-living conditions

Use of Smart Technology Tools for Supporting Public Health Surveillance: From Development of a Mobile Health Platform to Application in Stress Prediction

BACKGROUND Traditional public health data collection methods are typically based on self-reported data and may be subject to limitations such as biases, delays between collection and reporting, costs, and logistics. These may affect the quality of collected information and the ability of public health agencies to monitor and improve the health of populations. An alternative may be the use of personal, off-the-shelf smart devices (e.g., smartphones and smartwatches) as additional data collection tools. These devices can collect passive, continuous, real-time and objective health-related data, mitigating some of the limitations of self-reported information. The novel data types can then be used to further study and predict a condition in a population through advanced analytics. In this context, this thesis’ goal is to investigate new ways to support public health through the use of consumer-level smart technologies as complementary survey, monitoring and analyses tools, with a focus on perceived stress. To this end, a mobile health platform (MHP) that collects data from devices connected to Apple Health was developed and tested in a pilot study collecting self-reported and objective stress-related information, and a number of Machine Learning (ML) models were developed based on these data to monitor and predict the stress levels of participants. METHODS The mobile platform was created for iOS using the XCode software, allowing users to self-report their stress levels based on the stress subscale of the Depression, Anxiety and Stress Scale (DASS-21) as well as a single-item LIKERT-based scale. The platform also collects objective data from sensors that integrate with Apple Health, one of the most popular mobile health data repositories. A pilot study with 45 participants was conducted that uses the platform to collects stress self-reports and variables associated with stress from Apple Health, including heart rate, heart rate variability, ECG, sleep, blood pressure, weight, temperature, and steps. To this end, participants were given an iPhone with the platform installed as well as an Apple Watch, Withings Sleep, Withings Thermos, Withings BPM Connect, Withings Body+, and an Empatica E4 (the only device that does not connect to Apple Health but included due to its wide use in research). Participants were instructed to take device measurements and self-report stress levels 6 times per day for 14 days. Several experiments were conducted involving the development of ML models to predict stress based on the data, using Random Forests and Support Vector Machines. In each experiment, different subsets of the data from the full sample of 45 participants were used. 3 approaches to model development were followed: a) creating generalized models with all data; b) a hybrid approach using 80% of participants to train and 20% to test the model c) creating individualized user-specific models for each participant. In addition, statistical analyses of the data – specifically Spearman correlation and repeated measures ANOVA – were conducted. RESULTS Statistical analyses did not find significant differences between groups and only weak significant correlations. Among the Machine Learning models, the approach of using generalized models performed well, with f1-macro scores above 60% for several of the samples and features investigated. User-specific models also showed promise, with 82% achieving accuracies higher than 60% (the bottom limit of the state-of-the-art). While the hybrid approach had lower f1-macro scores, suggesting the models could not predict the two classes well, the accuracy of several of these models was in line with the state-of-the-art. Apple Watch sleep features, as well as weight, blood pressure, and temperature, were shown to be important in building the models. DISCUSSION AND CONCLUSION ML-based models built with data collected from the MHP in real-life conditions were able to predict stress with results often in line with state-of-the- art, showing that smart technology data can be a promising tool to support public health surveillance. In particular, the approaches of creating models for each participant or one generalized model were successful, although more validation is needed in future studies (e.g., with more purposeful sampling) for increased generalizability and validity on the use of these technologies in the real-world. The hybrid approach had good accuracy but lower f1-scores, indicating results could potentially be improved (e.g., possibly with less missing or noisy data, collected in more controlled conditions). For feature selection, important features included sleep data as well as weight, blood pressure and temperature from mobile and wearable devices. In summary, this study indicates that a platform such as the MHP, collecting data from smart technologies, could potentially be a novel tool to complement population-level public health data collection. The predictive stress modelling might be used to monitor stress levels in a population and provide personalized interventions. Although more validation may be needed, this work represents a step in this direction

INNOVATING CONTROL AND EMOTIONAL EXPRESSIVE MODALITIES OF USER INTERFACES FOR PEOPLE WITH LOCKED-IN SYNDROME

Patients with Lock-In-Syndrome (LIS) lost their ability to control any body part beside their eyes. Current solutions mainly use eye-tracking cameras to track patients' gaze as system input. However, despite the fact that interface design greatly impacts user experience, only a few guidelines have been were proposed so far to insure an easy, quick, fluid and non-tiresome computer system for these patients. On the other hand, the emergence of dedicated computer software has been greatly increasing the patients' capabilities, but there is still a great need for improvements as existing systems still present low usability and limited capabilities. Most interfaces designed for LIS patients aim at providing internet browsing or communication abilities. State of the art augmentative and alternative communication systems mainly focus on sentences communication without considering the need for emotional expression inextricable from human communication. This thesis aims at exploring new system control and expressive modalities for people with LIS. Firstly, existing gaze-based web-browsing interfaces were investigated. Page analysis and high mental workload appeared as recurring issues with common systems. To address this issue, a novel user interface was designed and evaluated against a commercial system. The results suggested that it is easier to learn and to use, quicker, more satisfying, less frustrating, less tiring and less prone to error. Mental workload was greatly diminished with this system. Other types of system control for LIS patients were then investigated. It was found that galvanic skin response may be used as system input and that stress related bio-feedback helped lowering mental workload during stressful tasks. Improving communication was one of the main goal of this research and in particular emotional communication. A system including a gaze-controlled emotional voice synthesis and a personal emotional avatar was developed with this purpose. Assessment of the proposed system highlighted the enhanced capability to have dialogs more similar to normal ones, to express and to identify emotions. Enabling emotion communication in parallel to sentences was found to help with the conversation. Automatic emotion detection seemed to be the next step toward improving emotional communication. Several studies established that physiological signals relate to emotions. The ability to use physiological signals sensors with LIS patients and their non-invasiveness made them an ideal candidate for this study. One of the main difficulties of emotion detection is the collection of high intensity affect-related data. Studies in this field are currently mostly limited to laboratory investigations, using laboratory-induced emotions, and are rarely adapted for real-life applications. A virtual reality emotion elicitation technique based on appraisal theories was proposed here in order to study physiological signals of high intensity emotions in a real-life-like environment. While this solution successfully elicited positive and negative emotions, it did not elicit the desired emotions for all subject and was therefore, not appropriate for the goals of this research. Collecting emotions in the wild appeared as the best methodology toward emotion detection for real-life applications. The state of the art in the field was therefore reviewed and assessed using a specifically designed method for evaluating datasets collected for emotion recognition in real-life applications. The proposed evaluation method provides guidelines for future researcher in the field. Based on the research findings, a mobile application was developed for physiological and emotional data collection in the wild. Based on appraisal theory, this application provides guidance to users to provide valuable emotion labelling and help them differentiate moods from emotions. A sample dataset collected using this application was compared to one collected using a paper-based preliminary study. The dataset collected using the mobile application was found to provide a more valuable dataset with data consistent with literature. This mobile application was used to create an open-source affect-related physiological signals database. While the path toward emotion detection usable in real-life application is still long, we hope that the tools provided to the research community will represent a step toward achieving this goal in the future. Automatically detecting emotion could not only be used for LIS patients to communicate but also for total-LIS patients who have lost their ability to move their eyes. Indeed, giving the ability to family and caregiver to visualize and therefore understand the patients' emotional state could greatly improve their quality of life. This research provided tools to LIS patients and the scientific community to improve augmentative and alternative communication, technologies with better interfaces, emotion expression capabilities and real-life emotion detection. Emotion recognition methods for real-life applications could not only enhance health care but also robotics, domotics and many other fields of study. A complete system fully gaze-controlled was made available open-source with all the developed solutions for LIS patients. This is expected to enhance their daily lives by improving their communication and by facilitating the development of novel assistive systems capabilities