Reconhecimento de expressões faciais em neonatos

Orientador: Profa Dra Olga R. P. BellonDissertação (mestrado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba, 30/10/2019Inclui referências: p. 46-51Área de concentração: Ciência da ComputaçãoResumo: A avaliação de dor é uma tarefa difícil e complexa, que é particularmente importante para recém-nascidos, que não conseguem verbaliza-la de maneira adequada e são vulneráveis a danos cerebrais decorrentes do não tratamento da dor. As ferramentas utilizadas no ambiente clínico para auxiliar na avaliação de dor requerem treinamento dos profissionais de saúde que irão utilizá-las, e seu uso é afetado pelo viés no reconhecimento da dor de cada indivíduo. Por essa razão, esforços tem sido colocados em automatizar essa tarefa, e uma das maneiras de fazê-lo é analisando a expressão facial do neonato, uma vez que esta é comprovadamente correlacionada à dor. Nessa dissertação, as diferenças entre os principais trabalhos em reconhecimento automático de expressão facial de neonatos são apresentadas, examinando os métodos utilizados, bases de dados e performances dos sistemas. Com isso em mente, testamos os principais métodos utilizados com objetivo de comparar suas performances mais a fundo. Esse estudo também avança o entendimento da base de dados COPE, a única base de dados de expressão facial de neonatos publicamente disponível. Conduzimos testes com métodos off the shelf para detecção de face, e em 54% das imagens nenhuma face foi detectada, reforçando a necessidade do desenvolvimento de sistemas específicos para recém-nascidos ou mais robustos à mudanças de público. Desde a publicação da base COPE em 2005, avanços significativos foram alcançados na área de processamento de imagens, e por essa razão comparamos métodos clássicos de extração de características em processamento de imagens com características provenientes de redes neurais convolucionais (CNNs), que são consideradas estado da arte para a maioria das aplicações de visão computacional. Um delta de 19% foi observado entre os filtros de gabor (melhor dos métodos clássicos) e características da ResNet50 (melhor das CNNs). Também testamos a robustez dos métodos a ruído, um fator importante em problemas de visão computacional onde devem ser considerados cenários da vida real. Para os métodos clássicos, foi observado um delta menor na performance entre cenários limpos e ruidosos, mas de maneira geral a performance foi pior que das CNNs. Em adição, estressando a performance das CNNs, testamos quais camadas produziriam melhor performance, na tentativa de verificar se camadas mais rasas poderiam ter desempenho igual ou melhor que camadas mais profundas, o que significaria menor custo computacional. Os resultados mostraram melhores resultados utilizando as camadas mais profundas. De maneira geral, estudando a literatura da área notamos uma tendência na utilização de métricas enviesadas, como acurácia, em um campo onde uma visão mais completa de performance de modelos deveria ser utilizada, por se tratar de um público tão vulnerável. Por fim, também observamos uma dificuldade no acesso as bases da literatura. Nossos esforços reforçam o potencial da utilização de métodos de visão computacional, porém fora limitados à base de dados utilizada. Palavras-chave: Expressões faciais, avaliação de dor, visão computacionalAbstract: Pain evaluation is a difficult and complex task, that is particularly important for newborns, who cannot verbalize it properly and are vulnerable to cerebral damage due to untreated pain. The current pain assessment tools used in clinical settings require extensive training for the caregivers and can be affected by each individual's bias towards pain recognition. For this reason, efforts have been made to automate this task, and one of the ways to do so is analyzing the newborn's facial expression, that has been proved to correlate with pain. In this dissertation, the differences among the most prominent works in automatic neonatal facial expression recognition were outlined, examining methods used, databases and final performance. With this in mind, we tested main methods used to compare their performances more in depth. This study also advances the understanding of the COPE database, the only publicly available newborn facial expression database. We conducted a test with off the shelf methods for face detection, and found that in 54% of the images, no face was found, reinforcing the need to develop either tailored applications or more robust ones. Since the COPE database was published, in 2005, significant advances in image processing have been made, and for this reason, we compared classical image processing feature extraction methods with Convolutional Neural Networks (CNNs), that are considered to be state of the art for most computer vision problems. We saw a difference of 19% in recall when using gabor filters (best of classical methods) and then the ResNet50 features (best of CNNs). We also tested the methods in regards to robustness to image noise, an important factor for computer vision problems when real world scenarios are considered. We found that image processing methods had a smaller delta in performance from clean to noisy scenarios, but had overall poor performance. In addition, stressing the CNNs performance, we also studied which layers yielded best performance in order to verify if shallow layers could produce the same results as deeper ones for this application, meaning less computational cost, but our test showed superior performance in deeper layers. Overall, studying the literature we noticed a tendency to use biased metrics, such as accuracy, in a field where a more complete view of model performance should be used. Moreover, we also found it very difficult to access data for this field. Our findings reinforce the potential of more complex computer vision methods, but are limited to the dataset that was used. Keywords: Facial expression, pain assessment, computer visio

Activity in area V3A predicts positions of moving objects

No description supplie

Unlocking the Pragmatics of Emoji: Evaluation of the Integration of Pragmatic Markers for Sarcasm Detection

Emojis have become an integral element of online communications, serving as a powerful, under-utilised resource for enhancing pragmatic understanding in NLP. Previous works have highlighted their potential for improvement of more complex tasks such as the identification of figurative literary devices including sarcasm due to their role in conveying tone within text. However present state-of-the-art does not include the consideration of emoji or adequately address sarcastic markers such as sentiment incongruence. This work aims to integrate these concepts to generate more robust solutions for sarcasm detection leveraging enhanced pragmatic features from both emoji and text tokens. This was achieved by establishing methodologies for sentiment feature extraction from emojis and a depth statistical evaluation of the features which characterise sarcastic text on Twitter. Current convention for generation of training data which implements weak-labelling using hashtags or keywords was evaluated against a human-annotated baseline; postulated validity concerns were verified where statistical evaluation found the content features deviated significantly from the baseline, highlighting potential validity concerns for many prominent works on the topic to date. Organic labelled sarcastic tweets containing emojis were crowd sourced by means of a survey to ensure valid outcomes for the sarcasm detection model. Given an established importance of both semantic and sentiment information, a novel sentiment-aware attention mechanism was constructed to enhance pattern recognition, balancing core features of sarcastic text: sentiment incongruence and context. This work establishes a framework for emoji feature extraction; a key roadblock cited in literature for their use in NLP tasks. The proposed sarcasm detection pipeline successfully facilitates the task using a GRU neural network with sentiment-aware attention, at an accuracy of 73% and promising indications regarding model robustness as part of a framework which is easily scalable for the inclusion of any future emojis released. Both enhanced sentiment information to supplement context in addition to consideration of the emoji were found to improve outcomes for the task

Social and Affective Neuroscience of Everyday Human Interaction

This Open Access book presents the current state of the art knowledge on social and affective neuroscience based on empirical findings. This volume is divided into several sections first guiding the reader through important theoretical topics within affective neuroscience, social neuroscience and moral emotions, and clinical neuroscience. Each chapter addresses everyday social interactions and various aspects of social interactions from a different angle taking the reader on a diverse journey. The last section of the book is of methodological nature. Basic information is presented for the reader to learn about common methodologies used in neuroscience alongside advanced input to deepen the understanding and usability of these methods in social and affective neuroscience for more experienced readers

Social and Affective Neuroscience of Everyday Human Interaction

This Open Access book presents the current state of the art knowledge on social and affective neuroscience based on empirical findings. This volume is divided into several sections first guiding the reader through important theoretical topics within affective neuroscience, social neuroscience and moral emotions, and clinical neuroscience. Each chapter addresses everyday social interactions and various aspects of social interactions from a different angle taking the reader on a diverse journey. The last section of the book is of methodological nature. Basic information is presented for the reader to learn about common methodologies used in neuroscience alongside advanced input to deepen the understanding and usability of these methods in social and affective neuroscience for more experienced readers

Leveraging Artificial Intelligence to Improve EEG-fNIRS Data Analysis

La spectroscopie proche infrarouge fonctionnelle (fNIRS) est apparue comme une technique de neuroimagerie qui permet une surveillance non invasive et à long terme de l'hémodynamique corticale. Les technologies de neuroimagerie multimodale en milieu clinique permettent d'étudier les maladies neurologiques aiguës et chroniques. Dans ce travail, nous nous concentrons sur l'épilepsie - un trouble chronique du système nerveux central affectant près de 50 millions de personnes dans le monde entier prédisposant les individus affectés à des crises récurrentes. Les crises sont des aberrations transitoires de l'activité électrique du cerveau qui conduisent à des symptômes physiques perturbateurs tels que des changements aigus ou chroniques des compétences cognitives, des hallucinations sensorielles ou des convulsions de tout le corps. Environ un tiers des patients épileptiques sont récalcitrants au traitement pharmacologique et ces crises intraitables présentent un risque grave de blessure et diminuent la qualité de vie globale. Dans ce travail, nous étudions 1. l'utilité des informations hémodynamiques dérivées des signaux fNIRS dans une tâche de détection des crises et les avantages qu'elles procurent dans un environnement multimodal par rapport aux signaux électroencéphalographiques (EEG) seuls, et 2. la capacité des signaux neuronaux, dérivé de l'EEG, pour prédire l'hémodynamique dans le cerveau afin de mieux comprendre le cerveau épileptique. Sur la base de données rétrospectives EEG-fNIRS recueillies auprès de 40 patients épileptiques et utilisant de nouveaux modèles d'apprentissage en profondeur, la première étude de cette thèse suggère que les signaux fNIRS offrent une sensibilité et une spécificité accrues pour la détection des crises par rapport à l'EEG seul. La validation du modèle a été effectuée à l'aide de l'ensemble de données CHBMIT open source documenté et bien référencé avant d'utiliser notre ensemble de données EEG-fNIRS multimodal interne. Les résultats de cette étude ont démontré que fNIRS améliore la détection des crises par rapport à l'EEG seul et ont motivé les expériences ultérieures qui ont déterminé la capacité prédictive d'un modèle d'apprentissage approfondi développé en interne pour décoder les signaux d'état de repos hémodynamique à partir du spectre complet et d'une bande de fréquences neuronale codée spécifique signaux d'état de repos (signaux sans crise). Ces résultats suggèrent qu'un autoencodeur multimodal peut apprendre des relations multimodales pour prédire les signaux d'état de repos. Les résultats suggèrent en outre que des gammes de fréquences EEG plus élevées prédisent l'hémodynamique avec une erreur de reconstruction plus faible par rapport aux gammes de fréquences EEG plus basses. De plus, les connexions fonctionnelles montrent des modèles spatiaux similaires entre l'état de repos expérimental et les prédictions fNIRS du modèle. Cela démontre pour la première fois que l'auto-encodage intermodal à partir de signaux neuronaux peut prédire l'hémodynamique cérébrale dans une certaine mesure. Les résultats de cette thèse avancent le potentiel de l'utilisation d'EEG-fNIRS pour des tâches cliniques pratiques (détection des crises, prédiction hémodynamique) ainsi que l'examen des relations fondamentales présentes dans le cerveau à l'aide de modèles d'apprentissage profond. S'il y a une augmentation du nombre d'ensembles de données disponibles à l'avenir, ces modèles pourraient être en mesure de généraliser les prédictions qui pourraient éventuellement conduire à la technologie EEG-fNIRS à être utilisée régulièrement comme un outil clinique viable dans une grande variété de troubles neuropathologiques.----------ABSTRACT Functional near-infrared spectroscopy (fNIRS) has emerged as a neuroimaging technique that allows for non-invasive and long-term monitoring of cortical hemodynamics. Multimodal neuroimaging technologies in clinical settings allow for the investigation of acute and chronic neurological diseases. In this work, we focus on epilepsy—a chronic disorder of the central nervous system affecting almost 50 million people world-wide predisposing affected individuals to recurrent seizures. Seizures are transient aberrations in the brain's electrical activity that lead to disruptive physical symptoms such as acute or chronic changes in cognitive skills, sensory hallucinations, or whole-body convulsions. Approximately a third of epileptic patients are recalcitrant to pharmacological treatment and these intractable seizures pose a serious risk for injury and decrease overall quality of life. In this work, we study 1) the utility of hemodynamic information derived from fNIRS signals in a seizure detection task and the benefit they provide in a multimodal setting as compared to electroencephalographic (EEG) signals alone, and 2) the ability of neural signals, derived from EEG, to predict hemodynamics in the brain in an effort to better understand the epileptic brain. Based on retrospective EEG-fNIRS data collected from 40 epileptic patients and utilizing novel deep learning models, the first study in this thesis suggests that fNIRS signals offer increased sensitivity and specificity metrics for seizure detection when compared to EEG alone. Model validation was performed using the documented open source and well referenced CHBMIT dataset before using our in-house multimodal EEG-fNIRS dataset. The results from this study demonstrated that fNIRS improves seizure detection as compared to EEG alone and motivated the subsequent experiments which determined the predictive capacity of an in-house developed deep learning model to decode hemodynamic resting state signals from full spectrum and specific frequency band encoded neural resting state signals (seizure free signals). These results suggest that a multimodal autoencoder can learn multimodal relations to predict resting state signals. Findings further suggested that higher EEG frequency ranges predict hemodynamics with lower reconstruction error in comparison to lower EEG frequency ranges. Furthermore, functional connections show similar spatial patterns between experimental resting state and model fNIRS predictions. This demonstrates for the first time that intermodal autoencoding from neural signals can predict cerebral hemodynamics to a certain extent. The results of this thesis advance the potential of using EEG-fNIRS for practical clinical tasks (seizure detection, hemodynamic prediction) as well as examining fundamental relationships present in the brain using deep learning models. If there is an increase in the number of datasets available in the future, these models may be able to generalize predictions which would possibly lead to EEG-fNIRS technology to be routinely used as a viable clinical tool in a wide variety of neuropathological disorders