Assessing the Effectiveness of Automated Emotion Recognition in Adults and Children for Clinical Investigation

Recent success stories in automated object or face recognition, partly fuelled by deep learning artificial neural network (ANN) architectures, has led to the advancement of biometric research platforms and, to some extent, the resurrection of Artificial Intelligence (AI). In line with this general trend, inter-disciplinary approaches have taken place to automate the recognition of emotions in adults or children for the benefit of various applications such as identification of children emotions prior to a clinical investigation. Within this context, it turns out that automating emotion recognition is far from being straight forward with several challenges arising for both science(e.g., methodology underpinned by psychology) and technology (e.g., iMotions biometric research platform). In this paper, we present a methodology, experiment and interesting findings, which raise the following research questions for the recognition of emotions and attention in humans: a) adequacy of well-established techniques such as the International Affective Picture System (IAPS), b) adequacy of state-of-the-art biometric research platforms, c) the extent to which emotional responses may be different among children or adults. Our findings and first attempts to answer some of these research questions, are all based on a mixed sample of adults and children, who took part in the experiment resulting into a statistical analysis of numerous variables. These are related with, both automatically and interactively, captured responses of participants to a sample of IAPS pictures

On the agreement between manual and automated methods for single-trial detection and estimation of features from event-related potentials

The agreement between humans and algorithms on whether an event-related potential (ERP) is present or not and the level of variation in the estimated values of its relevant features are largely unknown. Thus, the aim of this study was to determine the categorical and quantitative agreement between manual and automated methods for single-trial detection and estimation of ERP features. To this end, ERPs were elicited in sixteen healthy volunteers using electrical stimulation at graded intensities below and above the nociceptive withdrawal reflex threshold. Presence/absence of an ERP peak (categorical outcome) and its amplitude and latency (quantitative outcome) in each single-trial were evaluated independently by two human observers and two automated algorithms taken from existing literature. Categorical agreement was assessed using percentage positive and negative agreement and Cohen's κ, whereas quantitative agreement was evaluated using Bland-Altman analysis and the coefficient of variation. Typical values for the categorical agreement between manual and automated methods were derived, as well as reference values for the average and maximum differences that can be expected if one method is used instead of the others. Results showed that the human observers presented the highest categorical and quantitative agreement, and there were significantly large differences between detection and estimation of quantitative features among methods. In conclusion, substantial care should be taken in the selection of the detection/estimation approach, since factors like stimulation intensity and expected number of trials with/without response can play a significant role in the outcome of a study

Self-Regulation of SMR Power Led to an Enhancement of Functional Connectivity of Somatomotor Cortices in Fibromyalgia Patients

Neuroimaging studies have demonstrated that altered activity in somatosensory and motor cortices play a key role in pain chronification. Neurofeedback training of sensorimotor rhythm (SMR) is a tool which allow individuals to self-modulate their brain activity and to produce significant changes over somatomotor brain areas. Several studies have further shown that neurofeedback training may reduce pain and other pain-related symptoms in chronic pain patients. The goal of the present study was to analyze changes in SMR power and brain functional connectivity of the somatosensory and motor cortices elicited by neurofeedback task designed to both synchronize and desynchronize the SMR power over motor and somatosensory areas in fibromyalgia patients. Seventeen patients were randomly assigned to the SMR training (n = 9) or to a sham protocol (n = 8). All participants were trained during 6 sessions, and fMRI and EEG power elicited by synchronization and desynchronization trials were analyzed. In the SMR training group, four patients achieved the objective of SMR modulation in more than 70% of the trials from the second training session (good responders), while five patients performed the task at the chance level (bad responders). Good responders to the neurofeedback training significantly reduced pain and increased both SMR power modulationandfunctionalconnectivityofmotorandsomatosensoryrelatedareasduring the last neurofeedback training session, whereas no changes in brain activity or pain were observed in bad responders or participants in the sham group. In addition, we observed that good responders were characterized by reduced impact of fibromyalgia and pain symptoms, as well as by increased levels of health-related quality of life during the pre-training sessions. In summary, the present study revealed that neurofeedback training of SMR elicited significant brain changes in somatomotor areas leading to a significant reduction of pain in fibromyalgia patients. In this sense, our research provide evidence that neurofeedback training is a promising tool for a better understanding of brain mechanisms involved in pain chronification

Transcranial Focused Ultrasound to the Right Prefrontal Cortex Improves Mood and Alters Functional Connectivity in Humans

Transcranial focused ultrasound (tFUS) is an emerging method for non-invasive neuromodulation akin to transcranial magnetic stimulation (TMS) and transcranial direct current stimulation (tDCS). tFUS offers several advantages over electromagnetic methods including high spatial resolution and the ability to reach deep brain targets. Here we describe two experiments assessing whether tFUS could modulate mood in healthy human volunteers by targeting the right inferior frontal gyrus (rIFG), an area implicated in mood and emotional regulation. In a randomized, placebo-controlled, double-blind study, participants received 30 s of 500 kHz tFUS or a placebo control. Visual Analog Mood Scales (VAMS) assessed mood four times within an hour (baseline and three times after tFUS). Participants who received tFUS reported an overall increase in Global Affect (GA), an aggregate score from the VAMS scale, indicating a positive shift in mood. Experiment 2 examined resting-state functional (FC) connectivity using functional magnetic resonance imaging (fMRI) following 2 min of 500 kHz tFUS at the rIFG. As in Experiment 1, tFUS enhanced self-reported mood states and also decreased FC in resting state networks related to emotion and mood regulation. These results suggest that tFUS can be used to modulate mood and emotional regulation networks in the prefrontal cortex

Multimodal assessment of neonatal pain

Pain assessment is critical to prevent suffering and harm in infants admitted to the neonatal care unit. As pain is a subjective experience, its assessment in nonverbal infants relies on surrogate measures. Current infant pain assessment tools that are based on behaviour and autonomic nervous system measurements lack face validity — they are unlikely to reflect pain in all its dimensions. In recent years, EEG-derived measures of pain have been developed in late preterm and term infants. Multimodal tools which include these cerebral measurements are conceptually more appropriate to measure pain. Yet, their use is still limited to specific research applications. This thesis focuses on outstanding questions that need to be addressed in order to advance the development of multimodal pain assessment tools that incorporate cerebral measurements. In the first part of this thesis, I focus on the characterisation of preterm infants’ noxious-evoked responses and their development. Across several modalities, premature infants have dampened or altered responsiveness compared to term infants, and it is uncertain if these responses can be reliably discriminated from tactile-evoked responses. In particular, a discriminative pattern of noxious-evoked EEG activity that is present in term infants, is unlikely to be present in preterm infants. In addition, it is unclear how noxious-evoked responses, especially brainderived responses, change with age. In this thesis, I use a classification model to show that infants aged 28–40 weeks postmenstrual age display discriminable multimodal responses to a noxious clinical procedure and a tactile control procedure, and I provide examples of how a such a model could be used in clinical trials of analgesics. I show that noxious-evoked responses change magnitude and morphology across this age range, and that discriminative brain activity emerges in early prematurity. In the second part of this thesis, I focus on improving the neuroscientific validity of a noxious-evoked EEG response measured at the cot-side, as the spatial neural correlates of these responses are still poorly understood. I present an EEG-fMRI pilot study to investigate the spatial neural correlates of inter-individual differences in noxious-evoked EEG responses and provide recommendations for a larger follow-up study. Overall, this thesis provides a characterisation of infants’ noxious-evoked responses and their development across multiple modalities, a crucial next step in improving multimodal neonatal pain assessment

Single-trial laser-evoked potentials feature extraction for prediction of pain perception

Pain is a highly subjective experience, and the availability of an objective assessment of pain perception would be of great importance for both basic and clinical applications. The objective of the present study is to develop a novel approach to extract pain-related features from single-trial laser-evoked potentials (LEPs) for classification of pain perception. The single-trial LEP feature extraction approach combines a spatial filtering using common spatial pattern (CSP) and a multiple linear regression (MLR). The CSP method is effective in separating laser-evoked EEG response from ongoing EEG activity, while MLR is capable of automatically estimating the amplitudes and latencies of N2 and P2 from single-trial LEP waveforms. The extracted single-trial LEP features are used in a Naïve Bayes classifier to classify different levels of pain perceived by the subjects. The experimental results show that the proposed single-trial LEP feature extraction approach can effectively extract pain-related LEP features for achieving high classification accuracy.published_or_final_versio