Automatic Detection of Emotions and Distress in Textual Data

Online data can be analyzed for many purposes, including the prediction of stock market, business, and political planning. Online data can also be used to develop systems for the automatic emotion detection and mental health assessment of users. These systems can be used as complementary measures in monitoring online forums by detecting users who are in need of attention. In this thesis, we first present a new approach for contextual emotion detection, i.e. emotion detection in short conversations. The approach is based on a neural feature extractor, composed of a recurrent neural network with an attention mechanism, followed by a final classifier, that can be neural or SVM-based. The results from our experiments showed that, by providing a higher and more robust performance, SVM can act as a better final classifier in comparison to a feed-forward neural network. We then extended our model for emotion detection, and created an ensemble approach for the task of distress detection from online data. This extended approach utilizes several attention-based neural sub-models to extract features and predict class probabilities, which are later used as input features to a Support Vector Machine (SVM) making the final classification. Our experiments show that using an ensemble approach which makes use different sub-models accessing diverse sources of information can improve classification in the absence of a large annotated dataset. The extended model was evaluated on two shared tasks, CLPsych and eRisk 2019, which aim at suicide risk assessment, and early risk detection of anorexia, respectively. The model ranked first in tasks A and C of CLPsych 2019 (with macro-average F1 scores of 0.481 and 0.268, respectively), and ranked first in the first task of eRisk 2019 in terms of F1 and latency-weighted F1 scores (0.71 and 0.69, respectively)

Effective and Secure Healthcare Machine Learning System with Explanations Based on High Quality Crowdsourcing Data

Affordable cloud computing technologies allow users to efficiently outsource, store, and manage their Personal Health Records (PHRs) and share with their caregivers or physicians. With this exponential growth of the stored large scale clinical data and the growing need for personalized care, researchers are keen on developing data mining methodologies to learn efficient hidden patterns in such data. While studies have shown that those progresses can significantly improve the performance of various healthcare applications for clinical decision making and personalized medicine, the collected medical datasets are highly ambiguous and noisy. Thus, it is essential to develop a better tool for disease progression and survival rate predictions, where dataset needs to be cleaned before it is used for predictions and useful feature selection techniques need to be employed before prediction models can be constructed. In addition, having predictions without explanations prevent medical personnel and patients from adopting such healthcare deep learning models. Thus, any prediction models must come with some explanations. Finally, despite the efficiency of machine learning systems and their outstanding prediction performance, it is still a risk to reuse pre-trained models since most machine learning modules that are contributed and maintained by third parties lack proper checking to ensure that they are robust to various adversarial attacks. We need to design mechanisms for detection such attacks. In this thesis, we focus on addressing all the above issues: (i) Privacy Preserving Disease Treatment & Complication Prediction System (PDTCPS): A privacy-preserving disease treatment, complication prediction scheme (PDTCPS) is proposed, which allows authorized users to conduct searches for disease diagnosis, personalized treatments, and prediction of potential complications. (ii) Incentivizing High Quality Crowdsourcing Data For Disease Prediction: A new incentive model with individual rationality and platform profitability features is developed to encourage different hospitals to share high quality data so that better prediction models can be constructed. We also explore how data cleaning and feature selection techniques affect the performance of the prediction models. (iii) Explainable Deep Learning Based Medical Diagnostic System: A deep learning based medical diagnosis system (DL-MDS) is present which integrates heterogeneous medical data sources to produce better disease diagnosis with explanations for authorized users who submit their personalized health related queries. (iv) Attacks on RNN based Healthcare Learning Systems and Their Detection & Defense Mechanisms: Potential attacks on Recurrent Neural Network (RNN) based ML systems are identified and low-cost detection & defense schemes are designed to prevent such adversarial attacks. Finally, we conduct extensive experiments using both synthetic and real-world datasets to validate the feasibility and practicality of our proposed systems

Explainable depression detection with multi-aspect features using a hybrid deep learning model on social media

The ability to explain why the model produced results in such a way is an important problem, especially in the medical domain. Model explainability is important for building trust by providing insight into the model prediction. However, most existing machine learning methods provide no explainability, which is worrying. For instance, in the task of automatic depression prediction, most machine learning models lead to predictions that are obscure to humans. In this work, we propose explainable Multi-Aspect Depression Detection with Hierarchical Attention Network MDHAN, for automatic detection of depressed users on social media and explain the model prediction. We have considered user posts augmented with additional features from Twitter. Specifically, we encode user posts using two levels of attention mechanisms applied at the tweet-level and word-level, calculate each tweet and words’ importance, and capture semantic sequence features from the user timelines (posts). Our hierarchical attention model is developed in such a way that it can capture patterns that leads to explainable results. Our experiments show that MDHAN outperforms several popular and robust baseline methods, demonstrating the effectiveness of combining deep learning with multi-aspect features. We also show that our model helps improve predictive performance when detecting depression in users who are posting messages publicly on social media. MDHAN achieves excellent performance and ensures adequate evidence to explain the prediction

Deep learning approach to sentiment analysis in health and well-being

Sentiment analysis, also known as opinion mining, is an area of natural language processing which focuses on the classification of the sentiment that is expressed in a written document. Sentiment analysis has found applications in various domains including finance, politics, and health. This thesis is focused on sentiment analysis in the domain of health and well-being. An extensive systematic literature review was carried out to establish the state of the art in sentiment analysis in this domain. This systematic review provides evidence that the state-of-the-art results in sentiment analysis in the domain of health and well-being lags behind that in other domains. Additionally, it revealed that deep learning has not been used to classify the sentiment within the aforementioned domain. Furthermore, we performed a study and showed that the language that is used within the health and well-being domain is biased towards the negative sentiment. Aspect-based sentiment analysis refines the focus of sentiment analysis by classifying the sentiment associated with a specific aspect. Subsequently, we focus specifically on aspect-based sentiment analysis. To support it within the domain of health and well-being we created a dataset consisting of drug reviews, where the aspects were automatically annotated by matching concepts from the Unified Medical Language System. We have successfully shown that graph convolution can effectively utilise the context, represented with syntactic dependencies, to determine the intended sentiment of inherently negative aspects and consequently close the performance gap regardless of the domain. The advent of transformer-based architectures initiated a breakthrough in various tasks in natural language processing, including sentiment analysis. There-fore, we presented an approach to fine-tuning a transformer-based language model for the specific task of aspect-based sentiment analysis. The findings show the evidence that transformer-based models account for syntactic dependencies when classifying the sentiment of the given aspect

A text classification framework for simple and effective early depression detection over social media streams

With the rise of the Internet, there is a growing need to build intelligent systems that are capable of efficiently dealing with early risk detection (ERD) problems on social media, such as early depression detection, early rumor detection or identification of sexual predators. These systems, nowadays mostly based on machine learning techniques, must be able to deal with data streams since users provide their data over time. In addition, these systems must be able to decide when the processed data is sufficient to actually classify users. Moreover, since ERD tasks involve risky decisions by which people's lives could be affected, such systems must also be able to justify their decisions. However, most standard and state-of-the-art supervised machine learning models (such as SVM, MNB, Neural Networks, etc.) are not well suited to deal with this scenario. This is due to the fact that they either act as black boxes or do not support incremental classification/learning. In this paper we introduce SS3, a novel supervised learning model for text classification that naturally supports these aspects. SS3 was designed to be used as a general framework to deal with ERD problems. We evaluated our model on the CLEF's eRisk2017 pilot task on early depression detection. Most of the 30 contributions submitted to this competition used state-of-the-art methods. Experimental results show that our classifier was able to outperform these models and standard classifiers, despite being less computationally expensive and having the ability to explain its rationale.Fil: Burdisso, Sergio Gastón. Universidad Nacional de San Luis; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - San Luis; ArgentinaFil: Errecalde, Marcelo Luis. Universidad Nacional de San Luis; ArgentinaFil: Montes y Gómez, Manuel. Instituto Nacional de Astrofísica, Óptica y Electrónica; Méxic

RESULTS AND CHALLENGES OF ARTIFICIAL NEURAL NETWORKS USED FOR DECISION-MAKING AND CONTROL IN MEDICAL APPLICATIONS

The aim of this paper is to present several approaches by which technology can assist medical decision-making. This is an essential, but also a difficult activity, which implies a large number of medical and technical aspects. But, more important, it involves humans: on the one hand, the patient, who has a medical problem and who requires the best solution; on the other hand, the physician, who should be able to provide, in any circumstances, a decision or a prediction regarding the current and the future medical status of the patient. The technology, in general, and particularly the Artificial Intelligence (AI) tools could help both of them, and it is assisted by appropriate theory regarding modeling tools. One of the most powerful mechanisms that can be used in this field is the Artificial Neural Networks (ANNs). This paper presents some of the results obtained by the Process Control group of the Politehnica University Timisoara, Romania, in the field of ANNs applied to modeling, prediction and decision-making related to medical systems. An Iterative Learning Control-based approach to batch training a feedforward ANN architecture is given. The paper includes authors’ concerns in this domain and emphasizes that these intelligent models, even if they are artificial, are able to make decisions, being useful tools for prevention, early detection and personalized healthcare

DepressionNet: A Novel Summarization Boosted Deep Framework for Depression Detection on Social Media

Twitter is currently a popular online social media platform which allows users to share their user-generated content. This publicly-generated user data is also crucial to healthcare technologies because the discovered patterns would hugely benefit them in several ways. One of the applications is in automatically discovering mental health problems, e.g., depression. Previous studies to automatically detect a depressed user on online social media have largely relied upon the user behaviour and their linguistic patterns including user's social interactions. The downside is that these models are trained on several irrelevant content which might not be crucial towards detecting a depressed user. Besides, these content have a negative impact on the overall efficiency and effectiveness of the model. To overcome the shortcomings in the existing automatic depression detection methods, we propose a novel computational framework for automatic depression detection that initially selects relevant content through a hybrid extractive and abstractive summarization strategy on the sequence of all user tweets leading to a more fine-grained and relevant content. The content then goes to our novel deep learning framework comprising of a unified learning machinery comprising of Convolutional Neural Network (CNN) coupled with attention-enhanced Gated Recurrent Units (GRU) models leading to better empirical performance than existing strong baselines

EmoEEG - recognising people's emotions using electroencephalography

Tese de mestrado integrado em Engenharia Biomédica e Biofísica (Sinais e Imagens Médicas), Universidade de Lisboa, Faculdade de Ciências, 2020As emoções desempenham um papel fulcral na vida humana, estando envolvidas numa extensa variedade de processos cognitivos, tais como tomada de decisão, perceção, interações sociais e inteligência. As interfaces cérebro-máquina (ICM) são sistemas que convertem os padrões de atividade cerebral de um utilizador em mensagens ou comandos para uma determinada aplicação. Os usos mais comuns desta tecnologia permitem que pessoas com deficiência motora controlem braços mecânicos, cadeiras de rodas ou escrevam. Contudo, também é possível utilizar tecnologias ICM para gerar output sem qualquer controle voluntário. A identificação de estados emocionais é um exemplo desse tipo de feedback. Por sua vez, esta tecnologia pode ter aplicações clínicas tais como a identificação e monitorização de patologias psicológicas, ou aplicações multimédia que facilitem o acesso a músicas ou filmes de acordo com o seu conteúdo afetivo. O interesse crescente em estabelecer interações emocionais entre máquinas e pessoas, levou à necessidade de encontrar métodos fidedignos de reconhecimento emocional automático. Os autorrelatos podem não ser confiáveis devido à natureza subjetiva das próprias emoções, mas também porque os participantes podem responder de acordo com o que acreditam que os outros responderiam. A fala emocional é uma maneira eficaz de deduzir o estado emocional de uma pessoa, pois muitas características da fala são independentes da semântica ou da cultura. No entanto, a precisão ainda é insuficiente quando comparada com outros métodos, como a análise de expressões faciais ou sinais fisiológicos. Embora o primeiro já tenha sido usado para identificar emoções com sucesso, ele apresenta desvantagens, tais como o fato de muitas expressões faciais serem "forçadas" e o fato de que as leituras só são possíveis quando o rosto do sujeito está dentro de um ângulo muito específico em relação à câmara. Por estes motivos, a recolha de sinais fisiológicos tem sido o método preferencial para o reconhecimento de emoções. O uso do EEG (eletroencefalograma) permite-nos monitorizar as emoções sentidas sob a forma de impulsos elétricos provenientes do cérebro, permitindo assim obter uma ICM para o reconhecimento afetivo. O principal objetivo deste trabalho foi estudar a combinação de diferentes elementos para identificar estados afetivos, estimando valores de valência e ativação usando sinais de EEG. A análise realizada consistiu na criação de vários modelos de regressão para avaliar como diferentes elementos afetam a precisão na estimativa de valência e ativação. Os referidos elementos foram os métodos de aprendizagem automática, o género do indivíduo, o conceito de assimetria cerebral, os canais de elétrodos utilizados, os algoritmos de extração de características e as bandas de frequências analisadas. Com esta análise foi possível criarmos o melhor modelo possível, com a combinação de elementos que maximiza a sua precisão. Para alcançar os nossos objetivos, recorremos a duas bases de dados (AMIGOS e DEAP) contendo sinais de EEG obtidos durante experiências de desencadeamento emocional, juntamente com a autoavaliação realizada pelos respetivos participantes. Nestas experiências, os participantes visionaram excertos de vídeos de conteúdo afetivo, de modo a despoletar emoções sobre eles, e depois classificaram-nas atribuindo o nível de valência e ativação experienciado. Os sinais EEG obtidos foram divididos em epochs de 4s e de seguida procedeu-se à extração de características através de diferentes algoritmos: o primeiro, segundo e terceiro parâmetros de Hjorth; entropia espectral; energia e entropia de wavelets; energia e entropia de FMI (funções de modos empíricos) obtidas através da transformada de Hilbert-Huang. Estes métodos de processamento de sinal foram escolhidos por já terem gerado resultados bons noutros trabalhos relacionados. Todos estes métodos foram aplicados aos sinais EEG dentro das bandas de frequência alfa, beta e gama, que também produziram bons resultados de acordo com trabalhos já efetuados. Após a extração de características dos sinais EEG, procedeu-se à criação de diversos modelos de estimação da valência e ativação usando as autoavaliações dos participantes como “verdade fundamental”. O primeiro conjunto de modelos criados serviu para aferir quais os melhores métodos de aprendizagem automática a utilizar para os testes vindouros. Após escolher os dois melhores, tentámos verificar as diferenças no processamento emocional entre os sexos, realizando a estimativa em homens e mulheres separadamente. O conjunto de modelos criados a seguir visou testar o conceito da assimetria cerebral, que afirma que a valência emocional está relacionada com diferenças na atividade fisiológica entre os dois hemisférios cerebrais. Para este teste específico, foram consideradas a assimetria diferencial e racional segundo pares de elétrodos homólogos. Depois disso, foram criados modelos de estimação de valência e ativação considerando cada um dos elétrodos individualmente. Ou seja, os modelos seriam gerados com todos os métodos de extração de características, mas com os dados obtidos de um elétrodo apenas. Depois foram criados modelos que visassem comparar cada um dos algoritmos de extração de características utilizados. Os modelos gerados nesta fase incluíram os dados obtidos de todos os elétrodos, já que anteriormente se verificou que não haviam elétrodos significativamente melhores que outros. Por fim, procedeu-se à criação dos modelos com a melhor combinação de elementos possível, otimizaram-se os parâmetros dos mesmos, e procurámos também aferir a sua validação. Realizámos também um processo de classificação emocional associando cada par estimado de valores de valência e ativação ao quadrante correspondente no modelo circumplexo de afeto. Este último passo foi necessário para conseguirmos comparar o nosso trabalho com as soluções existentes, pois a grande maioria delas apenas identificam o quadrante emocional, não estimando valores para a valência e ativação. Em suma, os melhores métodos de aprendizagem automática foram RF (random forest) e KNN (k-nearest neighbours), embora a combinação dos melhores métodos de extração de características fosse diferente para os dois. KNN apresentava melhor precisão considerando todos os métodos de extração menos a entropia espectral, enquanto que RF foi mais preciso considerando apenas o primeiro parâmetro de Hjorth e a energia de wavelets. Os valores dos coeficientes de Pearson obtidos para os melhores modelos otimizados ficaram compreendidos entre 0,8 e 0,9 (sendo 1 o valor máximo). Não foram registados melhoramentos nos resultados considerando cada género individualmente, pelo que os modelos finais foram criados usando os dados de todos os participantes. É possível que a diminuição da precisão dos modelos criados para cada género seja resultado da menor quantidade de dados envolvidos no processo de treino. O conceito de assimetria cerebral só foi útil nos modelos criados usando a base de dados DEAP, especialmente para a estimação de valência usando as características extraídas segundo a banda alfa. Em geral, as nossas abordagens mostraram-se a par ou mesmo superiores a outros trabalhos, obtendo-se valores de acurácia de 86.5% para o melhor modelo de classificação gerado com a base de dados AMIGOS e 86.6% usando a base de dados DEAP.Emotion recognition is a field within affective computing that is gaining increasing relevance and strives to predict an emotional state using physiological signals. Understanding how these biological factors are expressed according to one’s emotions can enhance the humancomputer interaction (HCI). This knowledge, can then be used for clinical applications such as the identification and monitoring of psychiatric disorders. It can also be used to provide better access to multimedia content, by assigning affective tags to videos or music. The goal of this work was to create several models for estimating values of valence and arousal, using features extracted from EEG signals. The different models created were meant to compare how various elements affected the accuracy of the model created. These elements were the machine learning techniques, the gender of the individual, the brain asymmetry concept, the electrode channels, the feature extraction methods and the frequency of the brain waves analysed. The final models contained the best combination of these elements and achieved PCC values over 0.80. As a way to compare our work with previous approaches, we also implemented a classification procedure to find the correspondent quadrant in the valence and arousal space according to the circumplex model of affect. The best accuracies achieved were over 86%, which was on par or even superior to some of the works already done