Skin Admittance Measurement for Emotion Recognition: A Study over Frequency Sweep

The electrodermal activity (EDA) is a reliable physiological signal for monitoring the sympathetic nervous system. Several studies have demonstrated that EDA can be a source of effective markers for the assessment of emotional states in humans. There are two main methods for measuring EDA: endosomatic (internal electrical source) and exosomatic (external electrical source). Even though the exosomatic approach is the most widely used, differences between alternating current (AC) and direct current (DC) methods and their implication in the emotional assessment field have not yet been deeply investigated. This paper aims at investigating how the admittance contribution of EDA, studied at different frequency sources, affects the EDA statistical power in inferring on the subject?s arousing level (neutral or aroused). To this extent, 40 healthy subjects underwent visual affective elicitations, including neutral and arousing levels, while EDA was gathered through DC and AC sources from 0 to 1 kHz. Results concern the accuracy of an automatic, EDA feature-based arousal recognition system for each frequency source. We show how the frequency of the external electrical source affects the accuracy of arousal recognition. This suggests a role of skin susceptance in the study of affective stimuli through electrodermal response

Muscle fatigue assessment through electrodermal activity analysis during isometric contraction

We studied the effects of muscle fatigue on the Autonomic Nervous System (ANS) dynamics. Specifically, we monitored the electrodermal activity (EDA) on 32 healthy subjects performing isometric biceps contraction. As assessed by means of an electromyography (EMG) analysis, 15 subjects showed muscle fatigue and 17 did not. EDA signals were analyzed using the recently proposed cvxEDA model in order to decompose them into their phasic and tonic components and extract effective features to study ANS dynamics. A statistical comparison between the two groups of subjects was performed. Results revealed that relevant phasic EDA features significantly increased in the fatigued group. Moreover, a pattern recognition system was applied to the EDA dataset in order to automatically discriminate between fatigued and non-fatigued subjects. The proposed leave-one-subject-out KNN classifier showed an accuracy of 75.69%. These results suggest the use of EDA as correlate of muscle fatigue, providing integrative information to the standard indices extracted from the EMG signals

Wearable, Environmental, and Smartphone-Based Passive Sensing for Mental Health Monitoring

Collecting and analyzing data from sensors embedded in the context of daily life has been widely employed for the monitoring of mental health. Variations in parameters such as movement, sleep duration, heart rate, electrocardiogram, skin temperature, etc., are often associated with psychiatric disorders. Namely, accelerometer data, microphone, and call logs can be utilized to identify voice features and social activities indicative of depressive symptoms, and physiological factors such as heart rate and skin conductance can be used to detect stress and anxiety disorders. Therefore, a wide range of devices comprising a variety of sensors have been developed to capture these physiological and behavioral data and translate them into phenotypes and states related to mental health. Such systems aim to identify behaviors that are the consequence of an underlying physiological alteration, and hence, the raw sensor data are captured and converted into features that are used to define behavioral markers, often through machine learning. However, due to the complexity of passive data, these relationships are not simple and need to be well-established. Furthermore, parameters such as intrapersonal and interpersonal differences need to be considered when interpreting the data. Altogether, combining practical mobile and wearable systems with the right data analysis algorithms can provide a useful tool for the monitoring and management of mental disorders. The current review aims to comprehensively present and critically discuss all available smartphone-based, wearable, and environmental sensors for detecting such parameters in relation to the treatment and/or management of the most common mental health conditions

Bipolar Disorder Predictive Model: A Study to Analyze and Predict Emotional Change Using Physiological Signals

University of Minnesota M.S. thesis. July 2019. Major: Computer Science. Advisor: Arshia Khan. 1 computer file (PDF); viii, 95 pages.Bipolar disorder, a chronic mental illness, most common among people of ages 18 years and older, affects over 2.6 percent of the United States population alone. Although this illness cannot be cured, it can be managed by continuous tracking and monitoring. Hence, if a manic or depressive episode can be identified or predicted in its early stages, severe damage can be minimized, if not prevented. This thesis proposes the design of an objective sensor based system that is based on physiological predictors for the continuous and autonomous monitoring of bipolar patients. This system consists of a pulse rate sensor to record heart rate, and an electrodermal activity sensor to trace the emotional and cognitive state changes and does not rely completely on self-assessment or reporting. Furthermore, it investigates how psychological changes affect physiological responses, such as Heart rate variability (HRV) and Electrodermal activity (EDA). We conducted a study with 50 healthy participants, where each participant was subjected to a certain degree of image and audio induced emotion. Baseline and the emotional stimuli data was collected. Time- domain and non-linear analysis of HRV was then performed on the collected HRV data. In addition, EDA data analysis was performed by decomposing it into tonic and phasic components. We investigated the ability of HRV and EDA to identify the activity of the au- tonomic nervous system in response to emotional stimuli. Moreover, the extracted features from the data were then used to build machine learning models to predict the given psychological change in response to the emotional stimuli. Our results showed that these combination of HRV and EDA features from the study we conducted yielded an average accuracy of up to 73% with Support vector machine, and 68.3% with Discriminant analysis for predicting emotional change in healthy individuals

Autonomic Nervous System Dynamics for Mood and Emotional-State Recognition: Wearable Systems, Modeling, and Advanced Biosignal Processing

This thesis aims at investigating how electrophysiological signals related to the autonomic nervous system (ANS) dynamics could be source of reliable and effective markers for mood state recognition and assessment of emotional responses. In-depth methodological and applicative studies of biosignals such as electrocardiogram, electrodermal response, and respiration activity along with information coming from the eyes (gaze points and pupil size variation) were performed. Supported by the current literature, I found that nonlinear signal processing techniques play a crucial role in understanding the underlying ANS physiology and provide important quantifiers of cardiovascular control dynamics with prognostic value in both healthy subjects and patients. Two main applicative scenarios were identified: the former includes a group of healthy subjects who was presented with sets of images gathered from the International Affective Picture System hav- ing five levels of arousal and five levels of valence, including both a neutral reference level. The latter was constituted by bipolar patients who were followed for a period of 90 days during which psychophysical evaluations were performed. In both datasets, standard signal processing techniques as well as nonlinear measures have been taken into account to automatically and accurately recognize the elicited levels of arousal and valence and mood states, respectively. A novel probabilistic approach based on the point-process theory was also successfully applied in order to model and characterize the instantaneous ANS nonlinear dynamics in both healthy subjects and bipolar patients. According to the reported evidences on ANS complex behavior, experimental results demonstrate that an accurate characterization of the elicited affective levels and mood states is viable only when non- linear information are retained. Moreover, I demonstrate that the instantaneous ANS assessment is effective in both healthy subjects and patients. Besides mathematics and signal processing, this thesis also contributes to pragmatic issues such as emotional and mood state mod- eling, elicitation, and noninvasive ANS monitoring. Throughout the dissertation, a critical review on the current state-of-the-art is reported leading to the description of dedicated experimental protocols, reliable mood models, and novel wearable systems able to perform ANS monitoring in a naturalistic environment

Functional cerebral asymmetries of emotional processes in the healthy and bipolar brain

The perception and processing of emotions are of primary importance for social interaction, which confers faculties such as inferring what another person’s feels. Brain organisation of emotion perception has shown to primarily involve right hemisphere functioning. However, the brain may be functionally organised according to fundamental aspects of emotion such as valence, rather than involving processing of emotions in general. It should be noted, however, that emotion perception is not merely a perceptual process consisting in the input of emotional information, but also involves one’s emotional response. Therefore, the functional brain organisation of emotional processing may also be influenced by emotional experience. An experimental model for testing functional cerebral asymmetries (FCAs) of valenced emotional experience is uniquely found in bipolar disorder (BD) involving impaired ability to regulate emotions and eventually leading to depressive or manic episodes. Previous models have only explained hemispheric asymmetries for manic and depressive mood episodes, but not for BD euthymia. The present thesis sought to investigate FCAs in emotional processing in two major ways. First, FCAs underlying facial emotion perception under normal functioning was examined in healthy controls. Secondly, functional brain organisation in emotional processing was further investigated by assessing FCAs in the bipolarity continuum, used as an experimental model for studying the processing of emotions. In contrast with previous asymmetry models, results suggested a right hemisphere involvement in emotional experience regardless of valence. Atypical FCAs were found in euthymic BD patients reflecting inherent aspects of BD functional brain organisation that are free of symptomatic influence. Also, BD patients exhibited atypical connectivity in a default amygdala network particularly affecting the right hemisphere, suggesting intrinsic mechanisms associated with internal emotional states. Last, BD patients were associated with a reduced right hemisphere specialisation in visuospatial attention, therefore suggesting that right hemisphere dysfunction can also affect non-emotional processes. Taken together, the findings emphasize a BD continuum model relying on euthymia as a bridging state between usual mood and acute mood phases

The pill and the will : pharmacological and psychological modulation of cognitive and affective processes

Background: Impairments in cognition are components of practically all psychiatric disorders and in that sense transdiagnostic factors. In both clinical and non-clinical populations, ‘hot’ and ‘cold’ cognitive control, i.e., in emotional context and non-emotional context, is strongly associated with daily functioning and physical and mental well-being. The paradigm shift that the National Institute of Mental Health (NIMH) Research Domain Criteria initiative (RDoC) has introduced, signifies that targeting the underlying biological and behavioural endophenotypes that determine mental health and illness might be more fruitful than simply focusing on symptom based diagnostic categories. Yet, little is known on how pharmacological interventions such as selective serotonin reuptake inhibitors (SSRI) and psychostimulants (CS), that are routinely used in everyday clinical praxis, affect cognitive and emotional processes beyond the symptoms they are supposed to treat. Aim: The aim of this thesis was to compare induction and regulation of fear and disgust in healthy subjects, and to investigate how SSRI affect these processes. This basic design was expanded to also include the effect of stimulant medication on the induction and regulation of negative emotions in healthy controls and patients with ADHD. A parallel aim was to compare pharmacological emotion regulation (SSRI and CS) with psychological emotion regulation (reappraisal) and emotion regulation with skills training/ exposure (task repetition). Methods: A multimodal approach was used to explore (i) subjective rating of emotion intensity and objective measures of performance at the behavioural level, (ii) neural underpinnings in the CNS with functional near-infrared spectroscopy (fNIRS), functional magnetic resonance imaging (fMRI) and voxel-based morphometry (VBM) and (iii) physiological components of the sympathetic nervous system (electrodermal activity), which were all evaluated in the absence and presence of pharmacological and psychological interventions, during emotion induction, emotion regulation, cognitive Stroop and emotional Stroop paradigms. Results: Study I and IV demonstrated that emotion regulation with reappraisal is an effective strategy with robust effects on subjective emotional experience and electrodermal activity. Study II and III showed that task repetition improved performance during both cognitive and emotional Stroop tasks, and reduced electrodermal activity during cognitive Stroop, without significantly modifying emotion induction or emotion regulation. Study II and III showed significant effects of single dose escitalopram in reducing subjective emotional experience, improving task performance during affective interference of an ongoing cognitive process, altering prefrontal activity in a task-specific manner, and blurring the differences in the electrodermal activity between fear and disgust seen at baseline. Study IV showed that single dose CS reduced emotion induction, and that emotion regulation with reappraisal was significantly more effective in reducing subjective emotional experience compared to pharmacological emotion regulation with CS. Lastly, Study IV revealed aberrant emotion processing in patients with ADHD both at the behavioural and CNS levels, with patients reporting lower emotion induction and regulation scores, accompanied by less activation of dorsolateral prefrontal cortex, less deactivation of the default mode network and instead greater deactivation of the dorsal attention network, during emotion regulation compared to healthy controls. Structurally (VBM), less gray matter volume was found in limbic and paralimbic areas in patients with ADHD compared to healthy controls. Conclusions and implications: Dimensional approach using behavioural endophenotypes is a fruitful framework for studying normal physiology and diagnostic and treatment aspects of psychiatric disorders. In this thesis, it is demonstrated that emotional and non-emotional cognitive processes, although part of a continuum, likely respond differentially to psychological and pharmacological interventions and skills training with task repetition. Ultimately, improved knowledge in this field will help formulate hypothesisdriven and science-informed frameworks that will guide diagnosis and treatment plans, and usher a shift in psychiatric praxis

Automatic emotion recognition in clinical scenario: a systematic review of methods

none4Automatic emotion recognition has powerful opportunities in the clinical field, but several critical aspects are still open, such as heterogeneity of methodologies or technologies tested mainly on healthy people. This systematic review aims to survey automatic emotion recognition systems applied in real clinical contexts, to deeply analyse clinical and technical aspects, how they were addressed, and relationships among them. The literature review was conducted on: IEEEXplore, ScienceDirect, Scopus, PubMed, ACM. Inclusion criteria were the presence of an automatic emotion recognition algorithm and the enrollment of at least 2 patients in the experimental protocol. The review process followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Moreover, the works were analysed according to a reference model to deeply examine both clinical and technical topics. 52 scientific papers passed inclusion criteria. Most clinical scenarios involved neurodevelopmental, neurological and psychiatric disorders with the aims of diagnosing, monitoring, or treating emotional symptoms. The most adopted signals are video and audio, while supervised shallow learning is mostly used for emotion recognition. A poor study design, tiny samples, and the absence of a control group emerged as methodological weaknesses. Heterogeneity of performance metrics, datasets and algorithms challenges results comparability, robustness, reliability and reproducibility.openPepa, Lucia; Spalazzi, Luca; Capecci, Marianna; Ceravolo, Maria GabriellaPepa, Lucia; Spalazzi, Luca; Capecci, Marianna; Ceravolo, Maria Gabriell