Application of Common Sense Computing for the Development of a Novel Knowledge-Based Opinion Mining Engine

The ways people express their opinions and sentiments have radically changed in the past few years thanks to the advent of social networks, web communities, blogs, wikis and other online collaborative media. The distillation of knowledge from this huge amount of unstructured information can be a key factor for marketers who want to create an image or identity in the minds of their customers for their product, brand, or organisation. These online social data, however, remain hardly accessible to computers, as they are specifically meant for human consumption. The automatic analysis of online opinions, in fact, involves a deep understanding of natural language text by machines, from which we are still very far. Hitherto, online information retrieval has been mainly based on algorithms relying on the textual representation of web-pages. Such algorithms are very good at retrieving texts, splitting them into parts, checking the spelling and counting their words. But when it comes to interpreting sentences and extracting meaningful information, their capabilities are known to be very limited. Existing approaches to opinion mining and sentiment analysis, in particular, can be grouped into three main categories: keyword spotting, in which text is classified into categories based on the presence of fairly unambiguous affect words; lexical affinity, which assigns arbitrary words a probabilistic affinity for a particular emotion; statistical methods, which calculate the valence of affective keywords and word co-occurrence frequencies on the base of a large training corpus. Early works aimed to classify entire documents as containing overall positive or negative polarity, or rating scores of reviews. Such systems were mainly based on supervised approaches relying on manually labelled samples, such as movie or product reviews where the opinionist’s overall positive or negative attitude was explicitly indicated. However, opinions and sentiments do not occur only at document level, nor they are limited to a single valence or target. Contrary or complementary attitudes toward the same topic or multiple topics can be present across the span of a document. In more recent works, text analysis granularity has been taken down to segment and sentence level, e.g., by using presence of opinion-bearing lexical items (single words or n-grams) to detect subjective sentences, or by exploiting association rule mining for a feature-based analysis of product reviews. These approaches, however, are still far from being able to infer the cognitive and affective information associated with natural language as they mainly rely on knowledge bases that are still too limited to efficiently process text at sentence level. In this thesis, common sense computing techniques are further developed and applied to bridge the semantic gap between word-level natural language data and the concept-level opinions conveyed by these. In particular, the ensemble application of graph mining and multi-dimensionality reduction techniques on two common sense knowledge bases was exploited to develop a novel intelligent engine for open-domain opinion mining and sentiment analysis. The proposed approach, termed sentic computing, performs a clause-level semantic analysis of text, which allows the inference of both the conceptual and emotional information associated with natural language opinions and, hence, a more efficient passage from (unstructured) textual information to (structured) machine-processable data. The engine was tested on three different resources, namely a Twitter hashtag repository, a LiveJournal database and a PatientOpinion dataset, and its performance compared both with results obtained using standard sentiment analysis techniques and using different state-of-the-art knowledge bases such as Princeton’s WordNet, MIT’s ConceptNet and Microsoft’s Probase. Differently from most currently available opinion mining services, the developed engine does not base its analysis on a limited set of affect words and their co-occurrence frequencies, but rather on common sense concepts and the cognitive and affective valence conveyed by these. This allows the engine to be domain-independent and, hence, to be embedded in any opinion mining system for the development of intelligent applications in multiple fields such as Social Web, HCI and e-health. Looking ahead, the combined novel use of different knowledge bases and of common sense reasoning techniques for opinion mining proposed in this work, will, eventually, pave the way for development of more bio-inspired approaches to the design of natural language processing systems capable of handling knowledge, retrieving it when necessary, making analogies and learning from experience

Application of Common Sense Computing for the Development of a Novel Knowledge-Based Opinion Mining Engine

The ways people express their opinions and sentiments have radically changed in the past few years thanks to the advent of social networks, web communities, blogs, wikis and other online collaborative media. The distillation of knowledge from this huge amount of unstructured information can be a key factor for marketers who want to create an image or identity in the minds of their customers for their product, brand, or organisation. These online social data, however, remain hardly accessible to computers, as they are specifically meant for human consumption. The automatic analysis of online opinions, in fact, involves a deep understanding of natural language text by machines, from which we are still very far. Hitherto, online information retrieval has been mainly based on algorithms relying on the textual representation of web-pages. Such algorithms are very good at retrieving texts, splitting them into parts, checking the spelling and counting their words. But when it comes to interpreting sentences and extracting meaningful information, their capabilities are known to be very limited. Existing approaches to opinion mining and sentiment analysis, in particular, can be grouped into three main categories: keyword spotting, in which text is classified into categories based on the presence of fairly unambiguous affect words; lexical affinity, which assigns arbitrary words a probabilistic affinity for a particular emotion; statistical methods, which calculate the valence of affective keywords and word co-occurrence frequencies on the base of a large training corpus. Early works aimed to classify entire documents as containing overall positive or negative polarity, or rating scores of reviews. Such systems were mainly based on supervised approaches relying on manually labelled samples, such as movie or product reviews where the opinionist’s overall positive or negative attitude was explicitly indicated. However, opinions and sentiments do not occur only at document level, nor they are limited to a single valence or target. Contrary or complementary attitudes toward the same topic or multiple topics can be present across the span of a document. In more recent works, text analysis granularity has been taken down to segment and sentence level, e.g., by using presence of opinion-bearing lexical items (single words or n-grams) to detect subjective sentences, or by exploiting association rule mining for a feature-based analysis of product reviews. These approaches, however, are still far from being able to infer the cognitive and affective information associated with natural language as they mainly rely on knowledge bases that are still too limited to efficiently process text at sentence level. In this thesis, common sense computing techniques are further developed and applied to bridge the semantic gap between word-level natural language data and the concept-level opinions conveyed by these. In particular, the ensemble application of graph mining and multi-dimensionality reduction techniques on two common sense knowledge bases was exploited to develop a novel intelligent engine for open-domain opinion mining and sentiment analysis. The proposed approach, termed sentic computing, performs a clause-level semantic analysis of text, which allows the inference of both the conceptual and emotional information associated with natural language opinions and, hence, a more efficient passage from (unstructured) textual information to (structured) machine-processable data. The engine was tested on three different resources, namely a Twitter hashtag repository, a LiveJournal database and a PatientOpinion dataset, and its performance compared both with results obtained using standard sentiment analysis techniques and using different state-of-the-art knowledge bases such as Princeton’s WordNet, MIT’s ConceptNet and Microsoft’s Probase. Differently from most currently available opinion mining services, the developed engine does not base its analysis on a limited set of affect words and their co-occurrence frequencies, but rather on common sense concepts and the cognitive and affective valence conveyed by these. This allows the engine to be domain-independent and, hence, to be embedded in any opinion mining system for the development of intelligent applications in multiple fields such as Social Web, HCI and e-health. Looking ahead, the combined novel use of different knowledge bases and of common sense reasoning techniques for opinion mining proposed in this work, will, eventually, pave the way for development of more bio-inspired approaches to the design of natural language processing systems capable of handling knowledge, retrieving it when necessary, making analogies and learning from experience

Sentic Computing for Aspect-Based Opinion Summarization Using Multi-Head Attention with Feature Pooled Pointer Generator Network

Neural sequence to sequence models have achieved superlative performance in summarizing text. But they tend to generate generic summaries that under-represent the opinion-sensitive aspects of the document. Additionally, the sequence to sequence models are prone to test-train discrepancy (exposure-bias) arising from the differential summary decoding processes in the training and testing phases. The models use ground truth summary words in the decoder training phase and predicted outputs in the testing phase. This inconsistency leads to error accumulation and substandard performance. To address these gaps, a cognitive aspect-based opinion summarizer, Feature Pooled Pointer Generator Network (FP2GN), is proposed which selectively attends to thematic and contextual cues to generate sentiment-aware review summaries. This study augments the pointer generator framework with opinion feature extraction, feature pooling, and mutual attention mechanism for opinion summarization. The proposed model FP2GN identifies the aspect terms in review text using sentic computing (SenticNet 5 and concept frequency-inverse opinion frequency) and statistical feature engineering. These aspect terms are encoded into context embeddings using weighted average feature pooling, which is processed in a pointer-generator framework inspired stacked Bi-LSTM encoder–decoder model with multi-head self-attention. The decoder system uses temporal and mutual attention mechanisms to ensure the appropriate representation of input-sequence. The study also proffers the use of teacher forcing ratio to curtail the exposure-bias-related error-accumulation. The model achieves ROUGE-1 score of 86.04% and ROUGE-L score of 88.51% on the Amazon Fine Foods dataset. An average gain of 2% over other methods is observed. The proposed model reinforces pointer generator network architecture with opinion feature extraction, feature pooling, and mutual attention mechanism to generate human-readable opinion summaries. Empirical analysis substantiates that the proposed model is better than the baseline opinion summarizers

BERT-Deep CNN: State-of-the-Art for Sentiment Analysis of COVID-19 Tweets

The free flow of information has been accelerated by the rapid development of social media technology. There has been a significant social and psychological impact on the population due to the outbreak of Coronavirus disease (COVID-19). The COVID-19 pandemic is one of the current events being discussed on social media platforms. In order to safeguard societies from this pandemic, studying people's emotions on social media is crucial. As a result of their particular characteristics, sentiment analysis of texts like tweets remains challenging. Sentiment analysis is a powerful text analysis tool. It automatically detects and analyzes opinions and emotions from unstructured data. Texts from a wide range of sources are examined by a sentiment analysis tool, which extracts meaning from them, including emails, surveys, reviews, social media posts, and web articles. To evaluate sentiments, natural language processing (NLP) and machine learning techniques are used, which assign weights to entities, topics, themes, and categories in sentences or phrases. Machine learning tools learn how to detect sentiment without human intervention by examining examples of emotions in text. In a pandemic situation, analyzing social media texts to uncover sentimental trends can be very helpful in gaining a better understanding of society's needs and predicting future trends. We intend to study society's perception of the COVID-19 pandemic through social media using state-of-the-art BERT and Deep CNN models. The superiority of BERT models over other deep models in sentiment analysis is evident and can be concluded from the comparison of the various research studies mentioned in this article.Comment: 20 pages, 5 figure

Propuesta de un método para detección de emociones en e-learning

Las interfaces afectivas se están desarrollando para diferentes dominios, incluyendo los entornos de e-learning. La detección de emoción en el texto pretende inferir las emociones en el escritor. En un entorno virtual de aprendizaje, donde no existe el contacto con los alumnos para detectar sus emociones, el único medio para tratar de percibirlas es desde el texto que se ingresa. El objetivo de este trabajo es presentar una posible manera de interpretar las emociones negativas, expresadas en forma escrita por estudiantes que participan interactuando en un foro educativo virtual, con el fin de que el profesor tenga conocimiento de esto y realice acciones en su materia tendientes a mejorar el estado anímico del alumno, y así mejorar su predisposición para el aprendizaje.Sociedad Argentina de Informática e Investigación Operativa (SADIO

Propuesta de un método para detección de emociones en e-learning

Las interfaces afectivas se están desarrollando para diferentes dominios, incluyendo los entornos de e-learning. La detección de emoción en el texto pretende inferir las emociones en el escritor. En un entorno virtual de aprendizaje, donde no existe el contacto con los alumnos para detectar sus emociones, el único medio para tratar de percibirlas es desde el texto que se ingresa. El objetivo de este trabajo es presentar una posible manera de interpretar las emociones negativas, expresadas en forma escrita por estudiantes que participan interactuando en un foro educativo virtual, con el fin de que el profesor tenga conocimiento de esto y realice acciones en su materia tendientes a mejorar el estado anímico del alumno, y así mejorar su predisposición para el aprendizaje.Sociedad Argentina de Informática e Investigación Operativa (SADIO

Successes and challenges in developing a hybrid approach to sentiment analysis

This article covers some success and learning experiences attained during the developing of a hybrid approach to Sentiment Analysis (SA) based on a Sentiment Lexicon, Semantic Rules, Negation Handling, Ambiguity Management and Linguistic Variables. The proposed hybrid method is presented and applied to two selected datasets: Movie Review and Sentiment Twitter datasets. The achieved results are compared against those obtained when Naïve Bayes (NB) and Maximum Entropy (ME) supervised machine learning classification methods are used for the same datasets. The proposed hybrid system attained higher accuracy and precision scores than NB and ME, which shows its superiority when applied to the SA problem at the sentence level. Finally, an alternative strategy to calculating the orientation polarity and polarity intensity in one step instead of the two steps method used in the hybrid approach is explored. The analysis of the yielded mixed results achieved with this alternative approach shows its potential as an aid in the computation of semantic orientations and produced some lessons learnt in developing a more effective mechanism to calculating the orientation polarity and polarity intensity

MENGUKUR TINGKAT KEPUASAN MAHASISWA DALAM PEMBELAJARAN DENGAN NAÏVE BAYES

Mengukur kepuasan pembelajaran adalah hak mutlak bagi mahasiswa sehinggakampus wajib memperbaiki layanan baik kwalitas dan sarana dan prasarana. Karenakeberhasilan suatu institusi adalah di ukur dari tingkat kepuasan. Sering jadi kendalaadalah bagaimana mengukur kepuasan pelanggan dengan data teks berbentuk tidaktersetruktur. Sehingga konsumen kesulitan mendapatkan hasil yang baik. Mahasiswamemiliki sentiment yang berbeda beda terhadap baik pelayanan, proses pembelajarandan sarana prasaran. Untuk mempermudah mengukur kepuasan pelanggan. Penelitianini menerapkan teknik analisis sentimen untuk menilai kecenderungan saran terkaitdengan berbagai faktor yang berkontribusi pada keberhasilan proses pembelajaranseperti metode pengajaran, suasana akademik, fasilitas dan ruang kelas, laboratorium,perpustakaan dan fasilitas kampus lainnya. Metode yang digunakan penelitian iniadalah algoritma Naive Bayes. Dalam penelitian ini mendapatkan akurasi 86,9

Machine Learning in Resource-constrained Devices: Algorithms, Strategies, and Applications

The ever-increasing growth of technologies is changing people's everyday life. As a major consequence: 1) the amount of available data is growing and 2) several applications rely on battery supplied devices that are required to process data in real time. In this scenario the need for ad-hoc strategies for the development of low-power and low-latency intelligent systems capable of learning inductive rules from data using a modest mount of computational resources is becoming vital. At the same time, one needs to develop specic methodologies to manage complex patterns such as text and images. This Thesis presents different approaches and techniques for the development of fast learning models explicitly designed to be hosted on embedded systems. The proposed methods proved able to achieve state-of-the-art performances in term of the trade-off between generalization capabilities and area requirements when implemented in low-cost digital devices. In addition, advanced strategies for ecient sentiment analysis in text and images are proposed