Emotion classification in Parkinson's disease by higher-order spectra and power spectrum features using EEG signals: A comparative study

Deficits in the ability to process emotions characterize several neuropsychiatric disorders and are traits of Parkinson's disease (PD), and there is need for a method of quantifying emotion, which is currently performed by clinical diagnosis. Electroencephalogram (EEG) signals, being an activity of central nervous system (CNS), can reflect the underlying true emotional state of a person. This study applied machine-learning algorithms to categorize EEG emotional states in PD patients that would classify six basic emotions (happiness and sadness, fear, anger, surprise and disgust) in comparison with healthy controls (HC). Emotional EEG data were recorded from 20 PD patients and 20 healthy age-, education level- and sex-matched controls using multimodal (audio-visual) stimuli. The use of nonlinear features motivated by the higher-order spectra (HOS) has been reported to be a promising approach to classify the emotional states. In this work, we made the comparative study of the performance of k-nearest neighbor (kNN) and support vector machine (SVM) classifiers using the features derived from HOS and from the power spectrum. Analysis of variance (ANOVA) showed that power spectrum and HOS based features were statistically significant among the six emotional states (p < 0.0001). Classification results shows that using the selected HOS based features instead of power spectrum based features provided comparatively better accuracy for all the six classes with an overall accuracy of 70.10% ± 2.83% and 77.29% ± 1.73% for PD patients and HC in beta (13-30 Hz) band using SVM classifier. Besides, PD patients achieved less accuracy in the processing of negative emotions (sadness, fear, anger and disgust) than in processing of positive emotions (happiness, surprise) compared with HC. These results demonstrate the effectiveness of applying machine learning techniques to the classification of emotional states in PD patients in a user independent manner using EEG signals. The accuracy of the system can be improved by investigating the other HOS based features. This study might lead to a practical system for noninvasive assessment of the emotional impairments associated with neurological disorders

Detection of emotions in Parkinson's disease using higher order spectral features from brain's electrical activity

Non-motor symptoms in Parkinson's disease (PD) involving cognition and emotion have been progressively receiving more attention in recent times. Electroencephalogram (EEG) signals, being an activity of central nervous system, can reflect the underlying true emotional state of a person. This paper presents a computational framework for classifying PD patients compared to healthy controls (HC) using emotional information from the brain's electrical activity

Optimal set of EEG features for emotional state classification and trajectory visualization in Parkinson's disease

In addition to classic motor signs and symptoms, individuals with Parkinson's disease (PD) are characterized by emotional deficits. Ongoing brain activity can be recorded by electroencephalograph (EEG) to discover the links between emotional states and brain activity. This study utilized machine-learning algorithms to categorize emotional states in PD patients compared with healthy controls (HC) using EEG. Twenty non-demented PD patients and 20 healthy age-, gender-, and education level-matched controls viewed happiness, sadness, fear, anger, surprise, and disgust emotional stimuli while fourteen-channel EEG was being recorded. Multimodal stimulus (combination of audio and visual) was used to evoke the emotions. To classify the EEG-based emotional states and visualize the changes of emotional states over time, this paper compares four kinds of EEG features for emotional state classification and proposes an approach to track the trajectory of emotion changes with manifold learning. From the experimental results using our EEG data set, we found that (a) bispectrum feature is superior to other three kinds of features, namely power spectrum, wavelet packet and nonlinear dynamical analysis; (b) higher frequency bands (alpha, beta and gamma) play a more important role in emotion activities than lower frequency bands (delta and theta) in both groups and; (c) the trajectory of emotion changes can be visualized by reducing subject-independent features with manifold learning. This provides a promising way of implementing visualization of patient's emotional state in real time and leads to a practical system for noninvasive assessment of the emotional impairments associated with neurological disorders

Objectifying Facial Expressivity Assessment of Parkinson’s Patients: Preliminary Study

Patients with Parkinson’s disease (PD) can exhibit a reduction of spontaneous facial expression, designated as “facial masking,” a symptom in which facial muscles become rigid. To improve clinical assessment of facial expressivity of PD, this work attempts to quantify the dynamic facial expressivity (facial activity) of PD by automatically recognizing facial action units (AUs) and estimating their intensity. Spontaneous facial expressivity was assessed by comparing 7 PD patients with 8 control participants. To voluntarily produce spontaneous facial expressions that resemble those typically triggered by emotions, six emotions (amusement, sadness, anger, disgust, surprise, and fear) were elicited using movie clips. During the movie clips, physiological signals (facial electromyography (EMG) and electrocardiogram (ECG)) and frontal face video of the participants were recorded. The participants were asked to report on their emotional states throughout the experiment. We first examined the effectiveness of the emotion manipulation by evaluating the participant’s self-reports. Disgust-induced emotions were significantly higher than the other emotions. Thus we focused on the analysis of the recorded data during watching disgust movie clips. The proposed facial expressivity assessment approach captured differences in facial expressivity between PD patients and controls. Also differences between PD patients with different progression of Parkinson’s disease have been observed

Facial Mimicry and the Processing of Facial Emotional Expressions

In social interactions, facial expressions make a major contribution to our daily communication as they can transmit internal states like motivations and feelings of our conspecifics. In the last decades, research has revealed that facial mimicry plays a pivotal role in the accurate perception and interpretation of facial expressions. Embodied simulation theories claim that facial expressions are automatically mimicked, thereby producing a facial feedback signal, which in turn activates a corresponding state in the motor, somatosensory, affective and reward system of the observer. This activation - in turn - facilitates the processing of the observed emotional expression and hence supports the understanding of its meaning. Research on the influence of facial mimicry on the perception of emotional expressions is, to a large extent, driven by facial mimicry manipulation studies. Especially the classical facial mimicry manipulation method introduced by Strack, Martin, and Stepper (1988) has become a popular and established method. Here participants have to hold a pen in different positions with the mouth inducing a smiling or a frowning expression. The present thesis assessed the influence of facial mimicry on cognitive processes by means of this classical facial mimicry manipulation method. In three projects, I investigated the impact of (1) facial mimicry on the automatic processing of facial emotional expressions, (2) facial mimicry on the working memory for emotional expressions, and (3) facial mimicry manipulation on an impaired processing of emotional expressions in patients with Parkinson’s disease (PD). In a first project, the impact of facial mimicry manipulation was measured by electrophysiological recordings of the expression related mismatch negativity to unattended happy and sad faces. The findings reveal that the automatic processing of facial emotional expressions is systematically influenced by facial mimicry. In the second project, I assessed the behavioral performance during a facial emotional working memory task while the mimicry of participants was manipulated. Findings of this project highlight that working memory for emotional expressions is influenced by facial mimicry. Finally, in the third project, I investigated the link between the reduced facial mimicry in PD patients and their impaired ability to recognize emotional expressions. For this purpose, I compared the data of PD and healthy individuals during the performance of an emotional change detection task while undergoing facial mimicry manipulation. Although healthy participants show a typical pattern of facial mimicry manipulation influence, PD patients do not profit of the applied manipulation. The results of the present thesis demonstrate that facial mimicry is an indispensable part in our daily social interaction as it affects the processing of emotions on a perceptual as well as a cognitive level. I showed that facial mimicry influences the automatic processing of - as well as the working memory for - observed facial emotional expressions. Furthermore, the empirical evidence of the third project suggests that not only facial mimicry is reduced in patients with PD but rather that the whole process of facial feedback processing is impaired in those individuals. These results demonstrate the applicability of the classical facial mimicry manipulation method and further highlight the importance of research on the influence of facial mimicry on cognitive processing as our ability to understand the emotional expressions of our conspecifics and thus our social interaction depends on an intact facial mimicry processing

CNN AND LSTM FOR THE CLASSIFICATION OF PARKINSON'S DISEASE BASED ON THE GTCC AND MFCC

Parkinson's disease is a recognizable clinical syndrome with a variety of causes and clinical presentations; it represents a rapidly growing neurodegenerative disorder. Since about 90 percent of Parkinson's disease sufferers have some form of early speech impairment, recent studies on tele diagnosis of Parkinson's disease have focused on the recognition of voice impairments from vowel phonations or the subjects' discourse. In this paper, we present a new approach for Parkinson's disease detection from speech sounds that are based on CNN and LSTM and uses two categories of characteristics Mel Frequency Cepstral Coefficients (MFCC) and Gammatone Cepstral Coefficients (GTCC) obtained from noise-removed speech signals with comparative EMD-DWT and DWT-EMD analysis. The proposed model is divided into three stages. In the first step, noise is removed from the signals using the EMD-DWT and DWT-EMD methods. In the second step, the GTCC and MFCC are extracted from the enhanced audio signals. The classification process is carried out in the third step by feeding these features into the LSTM and CNN models, which are designed to define sequential information from the extracted features. The experiments are performed using PC-GITA and Sakar datasets and 10-fold cross validation method, the highest classification accuracy for the Sakar dataset reached 100% for both EMD-DWT-GTCC-CNN and DWT-EMD-GTCC-CNN, and for the PC-GITA dataset, the accuracy is reached 100% for EMD-DWT-GTCC-CNN and 96.55% for DWT-EMD-GTCC-CNN. The results of this study indicate that the characteristics of GTCC are more appropriate and accurate for the assessment of PD than MFCC

Affective Medicine: a review of Affective Computing efforts in Medical Informatics

Background: Affective computing (AC) is concerned with emotional interactions performed with and through computers. It is defined as “computing that relates to, arises from, or deliberately influences emotions”. AC enables investigation and understanding of the relation between human emotions and health as well as application of assistive and useful technologies in the medical domain. Objectives: 1) To review the general state of the art in AC and its applications in medicine, and 2) to establish synergies between the research communities of AC and medical informatics. Methods: Aspects related to the human affective state as a determinant of the human health are discussed, coupled with an illustration of significant AC research and related literature output. Moreover, affective communication channels are described and their range of application fields is explored through illustrative examples. Results: The presented conferences, European research projects and research publications illustrate the recent increase of interest in the AC area by the medical community. Tele-home healthcare, AmI, ubiquitous monitoring, e-learning and virtual communities with emotionally expressive characters for elderly or impaired people are few areas where the potential of AC has been realized and applications have emerged. Conclusions: A number of gaps can potentially be overcome through the synergy of AC and medical informatics. The application of AC technologies parallels the advancement of the existing state of the art and the introduction of new methods. The amount of work and projects reviewed in this paper witness an ambitious and optimistic synergetic future of the affective medicine field

Pupillary Response in an Auditory Rhythm Omissions Task in Parkinson´s Disease: A Pilot Study

When presented with short, rhythmical, musical excerpts, containing omitted beats which vary in saliency in terms of rhythmical patterns (contextual omission), and position (salience omissions), fMRI studies have shown a small effect depending on position of omission. Furthermore, when presented with auditory stimuli, a pupillary dilation response (PDR) is evoked, resulting in a pupillary peak dilation (PPD) sometime after stimulus onset. By utilizing and adapting an auditory beat-omission fMRI paradigm, to allow measurement of PDR and PPD, we used pupillometry data to investigate the effect of contextual omission (Simple vs Complex rhythm) and salience omission (O1 vs O2). We report data from a total of 25 participants, based on 45 datasets. The data were analyzed using four separate direct t-tests. We found that the omission has an effect on PPD, in that the most metrical salient omission (O1) results in a higher activation level compared to a less salient omission (O2), i.e., PPD was significantly higher in O1 simple rhythm omissions, and in O1 complex rhythm omissions, at an uncorrected threshold level.Masteroppgave i psykologiMAPSYK360INTL-HFINTL-PSYKINTL-SVINTL-MNINTL-MEDINTL-KMDMAPS-PSYKINTL-JU

Models and Analysis of Vocal Emissions for Biomedical Applications

The MAVEBA Workshop proceedings, held on a biannual basis, collect the scientific papers presented both as oral and poster contributions, during the conference. The main subjects are: development of theoretical and mechanical models as an aid to the study of main phonatory dysfunctions, as well as the biomedical engineering methods for the analysis of voice signals and images, as a support to clinical diagnosis and classification of vocal pathologies