Exploiting Emotions via Composite Pretrained Embedding and Ensemble Language Model

Decisions in the modern era are based on more than just the available data; they also incorporate feedback from online sources. Processing reviews known as Sentiment analysis (SA) or Emotion analysis. Understanding the user's perspective and routines is crucial now-a-days for multiple reasons. It is used by both businesses and governments to make strategic decisions. Various architectural and vector embedding strategies have been developed for SA processing. Accurate representation of text is crucial for automatic SA. Due to the large number of languages spoken and written,  polysemy and syntactic or semantic issues were common. To get around these problems, we developed effective composite embedding (ECE), a method that combines the advantages of vector embedding techniques that are either context-independent (like glove & fasttext) or context-aware (like  XLNet) to effectively represent the features needed for processing.  To improve the performace towards emotion or  sentiment we proposed stacked ensemble model of deep lanugae models.ECE with Ensembled model is evaluated on balanced  dataset to prove that it is a reliable embedding technique and a generalised model for SA.In order to evaluate ECE, cutting-edge ML and Deep net language models are deployed and comapared. The model is evaluated using benchmark datset such as  MR, Kindle along with realtime tweet dataset of user complaints . LIME is used to verify the model's predictions and to provide statistical results for sentence.The model with ECE embedding provides state-of-art results with real time dataset as well

EVALITA Evaluation of NLP and Speech Tools for Italian Proceedings of the Final Workshop

Editor of the proceedings of EVALITA 2016

A DEEP LEARNING APPROACH FOR SENTIMENT ANALYSIS

La Sentiment Analysis si riferisce alla analisi qualitativa volta ad identificare e classificare opinioni contenute in frasi e testi, allo scopo di stabilire lo \u201cstato d\u2019animo\u201d dell\u2019autore rispetto ad un particolare argomento o prodotto, e di determinare se tale stato \ue8 di fatto positivo, negativo oppure neutrale. Le opinioni espresse in un testo, come ad esempio giudizi, sentimenti ed emozioni, sono di recente diventate oggetto di studio e di ricerca sia in ambito accademico che industriale. Sfortunatamente la comprensione del linguaggio, applicata a commenti di utenti, \ue8 un attivit\ue0 estremamente complessa per una macchina, specialmente se ci si riferisce ai contesti dei moderni social network. Le modalit\ue0 in cui le persone si esprimono in linguaggio naturale, sono molteplici, e l\u2019utilizzo \u201cinformale\u201d della lingua adottato tipicamente nei social netowrks, genera frasi spesso dense di errori, modi di dire (slang), costrutti sintattici \u201dpersonalizzati\u201d, o anche frasi arricchite da caratteri speciali (come l\u2019hashtag in Twitter), il che complica notevolmente l\u2019analisi. Recentemente, le tecniche di Deep Learning, stanno emergendo nel panorama del machine learning, come un modello computazionale che pu\uf2 essere adoperato con efficacia per scoprire relazioni semantiche complesse, all\u2019interno di un testo, anche senza la necessit\ue0 di dover individuare a priori caratteristiche (features) di tali relazioni. Questi approcci hanno migliorato l\u2019attuale stato dell\u2019arte in diversi settori della Sentiment Analysis, come ad esempio la classificazione di frasi o di documenti, l\u2019apprendimento basato su lexicon, fino ad arrivare alla analisi di fenomeni complessi come il cyber bullismo. I contributi di questa tesi sono di due tipi. Il primo contributo fornito, relativo ad aspetti generali di Sentiment Analysis, riguarda la proposta di un modello di rete neurale semi supervisionata, basato sulle reti di tipo Deep Belief, in grado di affrontare l\u2019incertezza dei dati insita nelle frasi testuali, con particolare riferimento alla lingua italiana. Il modello proposto \ue8 stato testato rispetto a diversi datasets presi dalla letteratura di riferimento, composti da testi relativi a critiche cinematografiche, adottando una rappresentazione dell\u2019informazione basata su vettori (Word2Vec) ed introducendo anche metodi derivati dal campo del Natural Language Processing (NLP). Il secondo contributo fornito in questa tesi, partendo dall\u2019assunto che il cyber bullismo pu\uf2 essere considerato come un caso particolare di Sentiment Analysis, propone un approccio non supervisionato alla rilevazione automatica di tracce di cyber bullismo all\u2019interno di social networks, basato sia su di una rete neurale di tipo GHSOM (Growing Hierarchical Self Organizing Map), sia su di un modello di caratteristiche (features) predefinito. Il modello non supervisionato proposto dimostra di raggiungere comunque risultati interessanti rispetto ai tipici modelli supervisionati, applicati solitamente in questo ambito.Sentiment Analysis refers to the process of computationally identifying and categorizing opinions expressed in a piece of text, in order to determine whether the writer\u2019s attitude towards a particular topic or product is positive, negative, or even neutral. The views expressed and its related concepts, such as feelings, judgments, and emotions have become recently a subject of study and research in both academic and industrial areas. Unfortunately language comprehension of user comments, especially in social networks, is inherently complex to computers. The ways in which humans express themselves with natural language are nearly unlimited and informal texts is riddled with typos, misspellings, badly set up syntactic constructions and also specific symbols (e.g. hashtags in Twitter) which exponentially complicate this task. Recently, deep learning approaches are emerging as powerful computational models that discover intricate semantic representations of texts automatically from data without hand-made feature engineering. These approaches have improved the state-of-the-art in many Sentiment Analysis tasks including sentiment classification of sentences or documents, sentiment lexicon learning and also in more complex problems as cyber bullying detection. The contributions of this work are twofold. First, related to the general Sentiment Analysis problem, we propose a semi-supervised neural network model, based on Deep Belief Networks, able to deal with data uncertainty for text sentences in Italian language. We test this model against some datasets from literature related to movie reviews, adopting a vectorized representation of text (Word2Vec) and exploiting methods from Natural Language Processing (NLP) pre-processing. Second, assuming that the cyber bullying phenomenon can be treated as a particular Sentiment Analysis problem, we propose an unsupervised approach to automatic cyber bullying detection in social networks, based both on Growing Hierarchical Self Organizing Map (GHSOM) and on a new specific features model, showing that our solution can achieve interesting results, respect to classical supervised approaches

A Collaborative Platform to Support the Enterprise 2.0 in Active Interactions with Customers

In recent years a new model of Enterprise 2.0, which interacts actively with customers using web 2.0 tools (chat, forum, blog, wiki), is developing. The enterprises, listening opinions and suggestions of customers, can improve the product/service. For a company, customer's opinions are very important both for the improvement of products and also for the reinforcement of the customer loyalty. The customer will be motivated to be loyal if the enterprise shows a strong attention to his/her needs. This paper presents a model of a collaborative and interactive platform that supports the Enterprise 2.0 in the management of communications and relationships with all stakeholder of the supply chain and in particular with customers. A good e-reputation of the company improves business performances

The impact of news narrative on the economy and financial markets

This thesis investigates the impact of news narrative on socio-economic systems across four experiments. Recent years have witnessed a rise in the use of so-called alternative data sources to model and predict dynamics in socio-economic systems. Notably, sources such as newspaper text allow researchers to quantify the elusive concept of narrative, to incorporate text-based features into forecasting frameworks and thus to evaluate the impact of narrative on economic events. The first experiment proposes a new method of incorporating a wide array of sentiment scores from global newspaper articles into macroeconomic forecasts, attempting to forecast industrial production and consumer prices leveraging narrative and sentiment from global newspapers. I model industrial production and consumer prices across a diverse range of economies using an autoregressive framework. The second experiment uses narrative from global newspapers to construct themebased knowledge graphs about world events, demonstrating that features extracted from such graphs improve forecasts of industrial production in three large economies. The third experiment proposes a novel method of including news themes and their associated sentiment into predictions of changes in breakeven inflation rates (BEIR) for eight diverse economies with mature fixed income markets. I utilise five types of machine learning algorithms incorporating narrative-based features for each economy. In the above experiments, models incorporating narrative-based features generally outperform their benchmarks that do not contain such variables, demonstrating the predictive power of features derived from news narrative. The fourth experiment utilises GDELT data and the filtering methodology introduced in the first experiment to create a profitable systematic trading strategy based on the average tone scores for 15 diverse economies

The text classification pipeline: Starting shallow, going deeper

An increasingly relevant and crucial subfield of Natural Language Processing (NLP), tackled in this PhD thesis from a computer science and engineering perspective, is the Text Classification (TC). Also in this field, the exceptional success of deep learning has sparked a boom over the past ten years. Text retrieval and categorization, information extraction and summarization all rely heavily on TC. The literature has presented numerous datasets, models, and evaluation criteria. Even if languages as Arabic, Chinese, Hindi and others are employed in several works, from a computer science perspective the most used and referred language in the literature concerning TC is English. This is also the language mainly referenced in the rest of this PhD thesis. Even if numerous machine learning techniques have shown outstanding results, the classifier effectiveness depends on the capability to comprehend intricate relations and non-linear correlations in texts. In order to achieve this level of understanding, it is necessary to pay attention not only to the architecture of a model but also to other stages of the TC pipeline. In an NLP framework, a range of text representation techniques and model designs have emerged, including the large language models. These models are capable of turning massive amounts of text into useful vector representations that effectively capture semantically significant information. The fact that this field has been investigated by numerous communities, including data mining, linguistics, and information retrieval, is an aspect of crucial interest. These communities frequently have some overlap, but are mostly separate and do their research on their own. Bringing researchers from other groups together to improve the multidisciplinary comprehension of this field is one of the objectives of this dissertation. Additionally, this dissertation makes an effort to examine text mining from both a traditional and modern perspective. This thesis covers the whole TC pipeline in detail. However, the main contribution is to investigate the impact of every element in the TC pipeline to evaluate the impact on the final performance of a TC model. It is discussed the TC pipeline, including the traditional and the most recent deep learning-based models. This pipeline consists of State-Of-The-Art (SOTA) datasets used in the literature as benchmark, text preprocessing, text representation, machine learning models for TC, evaluation metrics and current SOTA results. In each chapter of this dissertation, I go over each of these steps, covering both the technical advancements and my most significant and recent findings while performing experiments and introducing novel models. The advantages and disadvantages of various options are also listed, along with a thorough comparison of the various approaches. At the end of each chapter, there are my contributions with experimental evaluations and discussions on the results that I have obtained during my three years PhD course. The experiments and the analysis related to each chapter (i.e., each element of the TC pipeline) are the main contributions that I provide, extending the basic knowledge of a regular survey on the matter of TC.An increasingly relevant and crucial subfield of Natural Language Processing (NLP), tackled in this PhD thesis from a computer science and engineering perspective, is the Text Classification (TC). Also in this field, the exceptional success of deep learning has sparked a boom over the past ten years. Text retrieval and categorization, information extraction and summarization all rely heavily on TC. The literature has presented numerous datasets, models, and evaluation criteria. Even if languages as Arabic, Chinese, Hindi and others are employed in several works, from a computer science perspective the most used and referred language in the literature concerning TC is English. This is also the language mainly referenced in the rest of this PhD thesis. Even if numerous machine learning techniques have shown outstanding results, the classifier effectiveness depends on the capability to comprehend intricate relations and non-linear correlations in texts. In order to achieve this level of understanding, it is necessary to pay attention not only to the architecture of a model but also to other stages of the TC pipeline. In an NLP framework, a range of text representation techniques and model designs have emerged, including the large language models. These models are capable of turning massive amounts of text into useful vector representations that effectively capture semantically significant information. The fact that this field has been investigated by numerous communities, including data mining, linguistics, and information retrieval, is an aspect of crucial interest. These communities frequently have some overlap, but are mostly separate and do their research on their own. Bringing researchers from other groups together to improve the multidisciplinary comprehension of this field is one of the objectives of this dissertation. Additionally, this dissertation makes an effort to examine text mining from both a traditional and modern perspective. This thesis covers the whole TC pipeline in detail. However, the main contribution is to investigate the impact of every element in the TC pipeline to evaluate the impact on the final performance of a TC model. It is discussed the TC pipeline, including the traditional and the most recent deep learning-based models. This pipeline consists of State-Of-The-Art (SOTA) datasets used in the literature as benchmark, text preprocessing, text representation, machine learning models for TC, evaluation metrics and current SOTA results. In each chapter of this dissertation, I go over each of these steps, covering both the technical advancements and my most significant and recent findings while performing experiments and introducing novel models. The advantages and disadvantages of various options are also listed, along with a thorough comparison of the various approaches. At the end of each chapter, there are my contributions with experimental evaluations and discussions on the results that I have obtained during my three years PhD course. The experiments and the analysis related to each chapter (i.e., each element of the TC pipeline) are the main contributions that I provide, extending the basic knowledge of a regular survey on the matter of TC

Natural Language Processing: Emerging Neural Approaches and Applications

This Special Issue highlights the most recent research being carried out in the NLP field to discuss relative open issues, with a particular focus on both emerging approaches for language learning, understanding, production, and grounding interactively or autonomously from data in cognitive and neural systems, as well as on their potential or real applications in different domains