Autonomic Nervous System Dynamics for Mood and Emotional-State Recognition: Wearable Systems, Modeling, and Advanced Biosignal Processing

This thesis aims at investigating how electrophysiological signals related to the autonomic nervous system (ANS) dynamics could be source of reliable and effective markers for mood state recognition and assessment of emotional responses. In-depth methodological and applicative studies of biosignals such as electrocardiogram, electrodermal response, and respiration activity along with information coming from the eyes (gaze points and pupil size variation) were performed. Supported by the current literature, I found that nonlinear signal processing techniques play a crucial role in understanding the underlying ANS physiology and provide important quantifiers of cardiovascular control dynamics with prognostic value in both healthy subjects and patients. Two main applicative scenarios were identified: the former includes a group of healthy subjects who was presented with sets of images gathered from the International Affective Picture System hav- ing five levels of arousal and five levels of valence, including both a neutral reference level. The latter was constituted by bipolar patients who were followed for a period of 90 days during which psychophysical evaluations were performed. In both datasets, standard signal processing techniques as well as nonlinear measures have been taken into account to automatically and accurately recognize the elicited levels of arousal and valence and mood states, respectively. A novel probabilistic approach based on the point-process theory was also successfully applied in order to model and characterize the instantaneous ANS nonlinear dynamics in both healthy subjects and bipolar patients. According to the reported evidences on ANS complex behavior, experimental results demonstrate that an accurate characterization of the elicited affective levels and mood states is viable only when non- linear information are retained. Moreover, I demonstrate that the instantaneous ANS assessment is effective in both healthy subjects and patients. Besides mathematics and signal processing, this thesis also contributes to pragmatic issues such as emotional and mood state mod- eling, elicitation, and noninvasive ANS monitoring. Throughout the dissertation, a critical review on the current state-of-the-art is reported leading to the description of dedicated experimental protocols, reliable mood models, and novel wearable systems able to perform ANS monitoring in a naturalistic environment

Inhomogeneous Point-Processes to Instantaneously Assess Affective Haptic Perception through Heartbeat Dynamics Information

This study proposes the application of a comprehensive signal processing framework, based on inhomogeneous point-process models of heartbeat dynamics, to instantaneously assess affective haptic perception using electrocardiogram-derived information exclusively. The framework relies on inverse-Gaussian point-processes with Laguerre expansion of the nonlinear Wiener-Volterra kernels, accounting for the long-term information given by the past heartbeat events. Up to cubic-order nonlinearities allow for an instantaneous estimation of the dynamic spectrum and bispectrum of the considered cardiovascular dynamics, as well as for instantaneous measures of complexity, through Lyapunov exponents and entropy. Short-term caress-like stimuli were administered for 4.3?25?seconds on the forearms of 32 healthy volunteers (16 females) through a wearable haptic device, by selectively superimposing two levels of force, 2?N and 6?N, and two levels of velocity, 9.4?mm/s and 65?mm/s. Results demonstrated that our instantaneous linear and nonlinear features were able to finely characterize the affective haptic perception, with a recognition accuracy of 69.79% along the force dimension, and 81.25% along the velocity dimension

Wearable artificial intelligence for anxiety and depression: A scoping review

Background: Anxiety and depression are the most common mental disorders worldwide. Owing to the lack of psychiatrists around the world, the incorporation of AI and wearable devices (wearable artificial intelligence (AI)) have been exploited to provide mental health services. Objective: The current review aimed to explore the features of wearable AI used for anxiety and depression to identify application areas and open research issues. Methods: We searched 8 electronic databases (MEDLINE, PsycINFO, EMBASE, CINAHL, IEEE Xplore, ACM Digital Library, Scopus, and Google Scholar). Then, we checked studies that cited the included studies, and screened studies that were cited by the included studies. Study selection and data extraction were carried out by two reviewers independently. The extracted data were aggregated and summarized using the narrative synthesis. Results: Of the 1203 citations identified, 69 studies were included in this review. About two thirds of the studies used wearable AI for depression while the remaining studies used it for anxiety. The most frequent application of wearable AI was diagnosing anxiety and depression while no studies used it for treatment purposes. The majority of studies targeted individuals between the ages of 18 and 65. The most common wearable devices used in the studies were Actiwatch AW4. The wrist-worn devices were most common in the studies. The most commonly used data for model development were physical activity data, sleep data, and heart rate data. The most frequently used dataset from open sources was Depresjon. The most commonly used algorithms were Random Forest (RF) and Support Vector Machine (SVM). Conclusions: Wearable AI can offer great promise in providing mental health services related to anxiety and depression. Wearable AI can be used by individuals as a pre-screening assessment of anxiety and depression. Further reviews are needed to statistically synthesize studies’ results related to the performance and effectiveness of wearable AI. Given its potential, tech companies should invest more in wearable AI for treatment purposes for anxiety and depression

Revealing Real-Time Emotional Responses: a Personalized Assessment based on Heartbeat Dynamics

Emotion recognition through computational modeling and analysis of physiological signals has been widely investigated in the last decade. Most of the proposed emotion recognition systems require relatively long-time series of multivariate records and do not provide accurate real-time characterizations using short-time series. To overcome these limitations, we propose a novel personalized probabilistic framework able to characterize the emotional state of a subject through the analysis of heartbeat dynamics exclusively. The study includes thirty subjects presented with a set of standardized images gathered from the international affective picture system, alternating levels of arousal and valence. Due to the intrinsic nonlinearity and nonstationarity of the RR interval series, a specific point-process model was devised for instantaneous identification considering autoregressive nonlinearities up to the third-order according to the Wiener-Volterra representation, thus tracking very fast stimulus-response changes. Features from the instantaneous spectrum and bispectrum, as well as the dominant Lyapunov exponent, were extracted and considered as input features to a support vector machine for classification. Results, estimating emotions each 10 seconds, achieve an overall accuracy in recognizing four emotional states based on the circumplex model of affect of 79.29%, with 79.15% on the valence axis, and 83.55% on the arousal axis

Beyond mobile apps: a survey of technologies for mental well-being

Mental health problems are on the rise globally and strain national health systems worldwide. Mental disorders are closely associated with fear of stigma, structural barriers such as financial burden, and lack of available services and resources which often prohibit the delivery of frequent clinical advice and monitoring. Technologies for mental well-being exhibit a range of attractive properties, which facilitate the delivery of state-of-the-art clinical monitoring. This review article provides an overview of traditional techniques followed by their technological alternatives, sensing devices, behaviour changing tools, and feedback interfaces. The challenges presented by these technologies are then discussed with data collection, privacy, and battery life being some of the key issues which need to be carefully considered for the successful deployment of mental health toolkits. Finally, the opportunities this growing research area presents are discussed including the use of portable tangible interfaces combining sensing and feedback technologies. Capitalising on the data these ubiquitous devices can record, state of the art machine learning algorithms can lead to the development of robust clinical decision support tools towards diagnosis and improvement of mental well-being delivery in real-time

Quantitative Analysis of the Effect of an Ectopic Beat on the Heart Rate Variability in the Resting Condition

The purpose of this study is to quantitatively analyze the effect of an ectopic beat on heart rate variability (HRV) in the time domain, frequency domain, and in a non-linear analysis. A quantitative analysis was carried out by generating artificial ectopic beats that probabilistically contained a missed beat or a false-detected beat, and the statistical significance was evaluated though a comparison with an ectopic-free HRV by increasing the ratio of the ectopic beat in 0.1% increments from 0 to 50%. The effect of the interpolation on the ectopic HRV was also investigated by applying nearest-neighbor interpolation, linear interpolation, and cubic spline interpolation. The results confirmed a statistically significant difference (P < 0.05) even in the less-than-1% ectopic HRV in every domain. When interpolation was applied, there were differences according to the interpolation method used, but statistical significance was secured for an ectopic beat ratio from 1 to 2% to several tens of a percent. In the effect, linear interpolation, and spline interpolation were confirmed to have a higher effect on the high-frequency related HRV variables, and nearest-neighbor interpolation had a higher effect on low-frequency related variables

Non-invasive wearable sensing systems for continuous health monitoring and long-term behavior modeling

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, February 2006.Includes bibliographical references (p. 212-228).Deploying new healthcare technologies for proactive health and elder care will become a major priority over the next decade, as medical care systems worldwide become strained by the aging populations. This thesis presents LiveNet, a distributed mobile system based on low-cost commodity hardware that can be deployed for a variety of healthcare applications. LiveNet embodies a flexible infrastructure platform intended for long-term ambulatory health monitoring with real-time data streaming and context classification capabilities. Using LiveNet, we are able to continuously monitor a wide range of physiological signals together with the user's activity and context, to develop a personalized, data-rich health profile of a user over time. Most clinical sensing technologies that exist have focused on accuracy and reliability, at the expense of cost-effectiveness, burden on the patient, and portability. Future proactive health technologies, on the other hand, must be affordable, unobtrusive, and non-invasive if the general population is going to adopt them.(cont.) In this thesis, we focus on the potential of using features derived from minimally invasive physiological and contextual sensors such as motion, speech, heart rate, skin conductance, and temperature/heat flux that can be used in combination with mobile technology to create powerful context-aware systems that are transparent to the user. In many cases, these non-invasive sensing technologies can completely replace more invasive diagnostic sensing for applications in long-term monitoring, behavior and physiology trending, and real-time proactive feedback and alert systems. Non-invasive sensing technologies are particularly important in ambulatory and continuous monitoring applications, where more cumbersome sensing equipment that is typically found in medical and clinical research settings is not usable. The research in this thesis demonstrates that it is possible to use simple non-invasive physiological and contextual sensing using the LiveNet system to accurately classify a variety of physiological conditions. We demonstrate that non-invasive sensing can be correlated to a variety of important physiological and behavioral phenomenon, and thus can serve as substitutes to more invasive and unwieldy forms of medical monitoring devices while still providing a high level of diagnostic power.(cont.) From this foundation, the LiveNet system is deployed in a number of studies to quantify physiological and contextual state. First, a number of classifiers for important health and general contextual cues such as activity state and stress level are developed from basic non-invasive physiological sensing. We then demonstrate that the LiveNet system can be used to develop systems that can classify clinically significant physiological and pathological conditions and that are robust in the presence of noise, motion artifacts, and other adverse conditions found in real-world situations. This is highlighted in a cold exposure and core body temperature study in collaboration with the U.S. Army Research Institute of Environmental Medicine. In this study, we show that it is possible to develop real-time implementations of these classifiers for proactive health monitors that can provide instantaneous feedback relevant in soldier monitoring applications. This thesis also demonstrates that the LiveNet platform can be used for long-term continuous monitoring applications to study physiological trends that vary slowly with time.(cont.) In a clinical study with the Psychiatry Department at the Massachusetts General Hospital, the LiveNet platform is used to continuously monitor clinically depressed patients during their stays on an in-patient ward for treatment. We show that we can accurately correlate physiology and behavior to depression state, as well as to track changes in depression state over time through the course of treatment. This study demonstrates how long-term physiology and behavioral changes can be captured to objectively measure medical treatment and medication efficacy. In another long-term monitoring study, the LiveNet platform is used to collect data on people's everyday behavior as they go through daily life. By collecting long-term behavioral data, we demonstrate the possibility of modeling and predicting high-level behavior using simple physiologic and contextual information derived solely from ambulatory mobile sensing technology.by Michael Sung.Ph.D