eSOLHotel: Building an Spanish opinion lexicon adapted to the tourism domain

Desde que la web 2.0 es el mayor contenedor de opiniones en todos los idiomas sobre distintos temas o asuntos, el estudio del Análisis de Sentimientos ha crecido exponencialmente. En este trabajo nos centramos en la clasificación de polaridad de opiniones en español y se presenta un nuevo recurso léxico adaptado al dominio turístico (eSOLHotel). Este nuevo lexicón usa el enfoque basado en corpus. Se han realizado varios experimentos usando una aproximación no supervisada para la clasificación de polaridad de las opiniones en la categoría de hoteles del corpus SFU. Los resultados obtenidos con el nuevo lexicón eSOLHotel superan los resultados obtenidos con otro lexicón de propósito general y nos animan a seguir trabajando en esta línea.Since Web 2.0 is the largest container for subjective expressions about different topics or issues expressed in all languages, the study of Sentiment Analysis has grown exponentially. In this work, we focus on Spanish polarity classification of hotel reviews and a new domain-dependent lexical resource (eSOLHotel) is presented. This new lexicon has been compiled following a corpus-based approach. We have carried out several experiments using an unsupervised approach for the polarity classification over the category of hotels from corpus SFU. The results obtained with the new lexicon eSOLHotel outperform the results with other general purpose lexicon.Esta investigación ha sido parcialmente financiada por el Fondo Europeo de Desarrollo Regional (FEDER), el proyecto ATTOS (TIN2012-38536-C03-0) del Gobierno de España y el proyecto AORESCU (P11-TIC-7684 MO) del gobierno autonómico de la Junta de Andalucía. Por último, el proyecto CEATIC (CEATIC-2013-01) de la Universidad de Jaén también ha financiado parcialmente este artículo

Improving Spanish Polarity Classification Combining Different Linguistic Resources

Sentiment analysis is a challenging task which is attracting the attention of researchers. However, most of work is only focused on English documents, perhaps due to the lack of linguistic resources for other languages. In this paper, we present several Spanish opinion mining resources in order to develop a polarity classification system. In addition, we propose the combination of different features extracted from each resource in order to train a classifier over two different opinion corpora. We prove that the integration of knowledge from several resources can improve the final Spanish polarity classification system. The good results encourage us to continue developing sentiment resources for Spanish, and studying the combination of features extracted from different resourcesMinisterio de Economía y Competitividad TIN2012-38536-C03-0Junta de Andalucía P11-TIC-7684Universidad de Jaén CEATIC-2013-0

Web Service SWePT: A Hybrid Opinion Mining Approach

[EN] The increasing use of social networks and online sites where people can express their opinions has created a growing interest in Opinion Mining. One of the main tasks of Opinion Mining is to determine whether an opinion is positive or negative. Therefore, the role of the feelings expressed on the web has become crucial, mainly due to the concern of businesses and government to automatically identify the semantic orientation of the views of customers or citizens. This is also a concern, in the area of health to identify psychological disorders. This research focuses on the development of a web application called SWePT (Web Service for Polarity detection in Spanish Texts), which implements the Sequential Minimal Optimization (SMO) algorithm, extracting its features from an affective lexicon in Mexican Spanish. For this purpose, a corpus and an affective lexicon in Mexican Spanish were created. The experiments using three (positive, neutral, negative) and five categories (very positive, positive, neutral, negative, and very negative) allow us to demonstrate the effectiveness of the presented method. SWePT has also been implemented in the Emotion-bracelet interface, which shows the opinion of a user graphically.This work has been partially supported by the Sectorial Fund CONACyT-INEGI: Project with ref. 208471, INFOTEC, Mexico. And, also by the project CNDT-PYR2015-0016, CENIDET, Mexico. The work of the third author was in the framework of the SomEMBED MINECO TIN2015-71147-C2-1-P research project. The National Council for Science and Technology (CONACyT Mexico) has funded the research work of Delia Irazu Hernandez Farias (Grant No. 218109/313683 CVU-369616).Baca-Gomez, YR.; Martínez, A.; Rosso, P.; Estrada Esquivel, H.; Hernandez-Farias, DI. (2016). Web Service SWePT: A Hybrid Opinion Mining Approach. Journal of Universal Computer Science. 22(5):671-690. https://doi.org/10.3217/jucs-022-05-067167169022

Can social media predict soccer clubs' stock prices?: The case of Turkish teams

Finance literature in sports focuses on three main methods of stock price prediction in soccer: based on match results, pre-match expectations or match importance. For pre-match expectations, betting odds is commonly used as the indicator of investors' sentiments. We propose to include Twitter data as another indicator of this variable, and analyze the links between soccer match results, sentiments, and stock returns of the four major Turkish soccer teams. Our results show that social media can be a strong indicator of pre-match expectations and investors’ sentiments in stock price prediction

Sentiment analysis in arabic: opinion polarity detection

Con Mención de Doctorado Internacional[ES]El análisis de sentimientos está obteniendo una gran importancia debido al aumento de popularidad de la web 2.0. Esta memoria se centra en el estudio de diferentes aspectos del análisis de sentimientos. El primer objetivo es analizar las opiniones que provienen del árabe y predecir su polaridad. Para alcanzar este objetivo se han generado dos corpora: OCA y EVOCA. OCA es un corpus de opinión de películas en árabe, y EVOCA es un corpus paralelo a OCA que incluye la traducción al inglés de las opiniones. Otro objetivo consiste en el análisis de sentimientos adaptado a diferentes dominios. Para ello, se ha generado el corpus SINAI-SA y se han aplicado distintas técnicas de aprendizaje automático. Finalmente, en esta memoria se realiza un estudio sobre revisiones neutrales. Para llevar a cabo este objetivo, se han investigado dos enfoque principales, uno basado en orientación semántica y el otro basado en algoritmos de aprendizaje automático como SVM o NB.[EN]Sentiment analysis is becoming increasingly important due the growing popularity of Web 2.0. This study focuses mainly on how to analyze opinions in Arabic language and predict their polarity. To achieve that, two corpora have been generated (OCA and EVOCA), OCA is an opinion corpus for Arabic movie reviews, while EVOCA is the translated version of OCA to English. Another corpus was created (SINAI-SA corpus) used with other corpora in order to predict sentiments in different domains. SINAI corpus was also used to study how to sort comments behave as textual information for the prediction of customer rates. Another question that was solved in this study is “How to treat with the neutral reviews”. Two main approaches have been investigated in this research, one based on semantic orientation and the other one based on machine learning algorithms like SVM or NBTesis Univ. Jaén. Departamento de Informática, leída el 7 de octubre de 201

Generación de recursos para Análisis de Opiniones en español

[ES] El Análisis de Sentimientos (AS) se refiere al tratamiento de la información subjetiva en los textos, sobretodo comentarios u opiniones personales. Una de las tareas básicas de AS es la clasificación de la polaridad de un texto determinado en un documento o frase, es decir, si la opinión expresada es positiva, negativa o neutra. Mucho se ha investigado en la clasificación de polaridad en documentos escritos en inglés. Sin embargo, actualmente cada vez más personas expresan comentarios u opiniones en su propio idioma. Para llevar a cabo esta labor es necesario el uso de los recursos lingüísticos (lexicones y corpora) que son escasos, cuando no inexistentes, en idiomas distintos al inglés. Por tales circunstancias, esta tesis tiene como objetivo la generación de nuevos recursos para el AS en español, tercer idioma con más relevancia en la web 2.0.[EN] Sentiment Analysis (SA) refers to the treatment of the subjective information in texts, product reviews, comments on blogs or personal opinions. One of the basic tasks in SA is classifying the polarity of a given text in a document, i.e., whether the opinion expressed is positive, negative, or neutral. Many studies have investigated the polarity classification in documents written in English. However, nowadays more and more people express their comments, opinions or points of view in their own language. For this reason, it is necessary to develop systems than can extract and analyze all this information in different languages. In this work we focus on polarity detection for Spanish reviews. We are mainly concerned with linguistic resources for Spanish sentiment analysis because, in addition to the lack of resources for this language in this area, it is currently the third most used language in the web 2.0.Tesis Univ. Jaén. Departamento de Informática- Leída el 28 de noviembre de 201

ANALYZING CUSTOMER REVIEWS IN TURKISH USING MACHINE LEARNING AND DATA SCIENCE METHODOLOGIES

Digital life, especially after the introduction of Web 2.0, has significantly altered human relations, providing all people the “right of public speech”. Ideas, emotions, and opinions on many topics are generously shared in virtual environments. A new age global and digital Mouth of World is shaping the society where knowledge is the most influential power. Being fed by social media data highly dynamic in either amount or shape, automatic handling is indispensable. Natural Language Processing, in cooperation with Machine Language techniques, has an important say in analyzing written textual data. Traditional techniques exploited in the literature are empowered when hybrid ones are applied, in accordance also with the characteristic properties of the language used and the domain-specific data. Although all the subsequent steps of the text classification chain are important, adequate feature selecting has a notable huge impact on accurate classification prediction. In this study, a simple classification of the sentiment polarity of comments in document level of subjective texts in Turkish is done. Different domains include reviews of customers towards company products, movies, and healthcare services, deciding on the positivity or negativity of the comments. Another domain includes doctors’ notes on patients’ symptoms aiming to predict and thus recommend some of the most often used medical tests according to general doctors’ procedures. The features used included a part of or all distinct words roots together with their binary or frequency information. Linear or vector analysis of the feature sets was done employing Machine Learning algorithms provided by the Weka tool. Hybrid features set was proposed and found more efficient combining binary vectors and frequency meta-features from nodes and leaves of J48 tree classifier for all or a set of correlation based selected features, improving both prediction accuracy and classification performance

Exploiting domain knowledge to enhance opinion mining using a hybrid semantic knowledgebase-machine learning approach

With the fast growth of World Wide Web 2.0, a great number of opinions about a variety of products have been published on blogs, forums, and social networks. Online opinions play an important role in supporting consumers make decisions about purchasing products or services. In addition, customer reviews allow companies to understand the strengths and limitations of their products and services, which aids in improving their marketing campaigns. The challenge is that online opinions are predominantly expressed in natural language text, and hence opinion mining tools are required to facilitate the effective analysis of opinions from the unstructured text and to allow for qualitative information extraction. This research presents a Hybrid Semantic Knowledgebase-Machine Learning approach for mining opinions at the domain feature level and classifying the overall opinion on a multi-point scale. The proposed approach benefits from the advantages of deploying a novel Semantic Knowledgebase approach to analyse a collection of reviews at the domain feature level and produce a set of structured information that associates the expressed opinions with specific domain features. The information in the knowledgebase is further supplemented with domain-relevant facts sourced from public Semantic datasets, and the enriched semantically-tagged information is then used to infer valuable semantic information about the domain as well as the expressed opinions on the domain features by summarising the overall opinions about the domain across multiple reviews, and by averaging the overall opinions about other cinematic features. The retrieved semantic information represents a valuable resource for training a Machine Learning classifier to predict the numerical rating of each review. Experimental evaluation revealed that the proposed Hybrid Semantic Knowledgebase-Machine Learning approach improved the precision and recall of the extracted domain features, and hence proved suitable for producing an enriched dataset of semantic features that resulted in higher classification accuracy

Opinion classification techniques applied to a Spanish corpus

El análisis de sentimientos es una nueva tarea que combina técnicas de minería de texto y Procesamiento del Lenguaje Natural (PLN). Aunque existen ya varios trabajos relacionados con la temática, la mayoría de ellos únicamente usan textos en inglés. Sin embargo, el número de páginas webs, blogs u opiniones vertidas en Internet que usan cualquier idioma, no sólo el inglés, crece exponencialmente. De hecho, otros idiomas como el español incrementan su presencia en la red día a día. Es por ello que en este trabajo se presenta un estudio experimental sobre un corpus de opiniones sobre películas escrito en español. Nuestro objetivo principal consiste en comprobar cómo se comportan varios clasificadores entrenados para determinar la polaridad de las opiniones en dicho corpus. Para ello hemos usado dos clasificadores (SVM y Naïve Bayes) variando además distintos parámetros como el esquema de pesado o la utilización o no de stopper y stemmer. Los experimentos realizados muestran que SVM se comporta mejor que Naïve Bayes y que el uso de stopper y stemmer también mejora los resultados.Sentiment analysis is a new challenging task related to Text Mining and Natural Language Processing (NLP). Although there are some current works, most of them only focus on English texts. However, web pages, blogs and opinions on the Internet are increasing every day in any language and not only in English. Other language like Spanish is increasingly present so we have carried out an experimental study with a Spanish films reviews corpus. Our main goal is to check the results obtained using several classifiers trained in order to determinate the opinion polarity. We have tested two classification algorithms (SVM, Naïve Bayes) and several weighting schemes and different linguistic preprocessing (stopper and stemmer). The accomplished experiments show that SVM works better than Naïve Bayes. In addition, the stopper and stemmer also obtain a slight improvement.Esta investigación ha sido parcialmente financiada por el Fondo Europeo de Desarrollo Regional (FEDER), proyecto TEXT-COOL 2.0 (TIN2009-13391-C04-02) del Gobierno Español, por la Junta de Andalucía, proyecto GeOasis (P08-TIC-41999), y por el Instituto de Estudios Giennenses, proyecto RFC/IEG2010

Peer assessment model with the sentiment análisis approach

La evaluación de los trabajos de respuesta abierta es una tarea que debe ser realizada por un experto; sin embargo, suponen una importante carga de trabajo de corrección para el docente. En este contexto, la evaluación entre pares se ha considerado como un enfoque alternativo para abordar el problema. Este tipo de evaluación no solo proporciona la reducción en la carga de trabajo de corrección, sino que también aporta beneficios adicionales, como la posibilidad de que el estudiante verifique diferentes soluciones para un mismo problema y la provisión de retroalimentaciones útiles. Los cambios en los paradigmas educativos han promovido la integración de métodos de evaluación que pretenden ir más allá de la evaluación de conocimientos (sumativa), que estén más integrados en el proceso de formación y aprendizaje (formativa). La evaluación formativa contribuye significativamente en la calidad de aprendizaje que los estudiantes obtienen al dar y recibir retroalimentación, y en el acceso inmediato que los docentes pueden tener sobre el progreso de la clase. Las instituciones educativas actualmente buscan obtener el conocimiento inmerso de estos textos no estructurado. Por lo tanto, el objetivo general de esta tesis ha sido diseñar un modelo de evaluación entre pares, que coadyuve a los docentes a mejorar sus procesos de enseñanza-aprendizaje mediante métodos de análisis de sentimiento. Se aplicó la metodología de diseño investigación-acción, en primer lugar, se realizó el estado del arte sobre evaluación entre pares, minería de texto y técnicas de computación blanda. Subsecuentemente, se diseñó un modelo que combina la evaluación entre pares con el aprendizaje colaborativo y el método calibrado en varias fases: a) se formó grupos de estudiantes que participan en realizar el trabajo de manera colaborativa, con la finalidad de tener grupos similares, pero tener diferencias individuales en el proceso de evaluación entre pares para beneficiarse de la colaboración entre estudiantes; b) se diseñó una rúbrica para la recolección de datos, donde los evaluadores evaluaron aspectos específicos del trabajo, proporcionando por cada criterio una puntuación numérica y retroalimentación textual; c) los evaluados evaluaron la calidad de evaluación de la tarea (evaluación inversa) para obtener el rating de confianza del evaluador; d) los grupos corrigieron el trabajo basándose en las retroalimentaciones dadas por los evaluadores en la primera ronda (evaluación en dos rondas); e) la puntuación de evaluación de tarea se calibró en función del rendimiento e índice (rating) de confianza del evaluador. Durante el desarrollo del modelo se obtuvo dos conjuntos de datos en español, uno de evaluación de tarea y otro de evaluación de calidad de la evaluación. Se aplicó el enfoque de aprendizaje automático supervisado para obtener una puntuación de sentimiento correspondiente a una retroalimentación textual específica. Se analizó distintas técnicas de minería de texto y procesamiento de lenguaje natural sobre la tarea de clasificación de sentimiento como Bag of Words, combinaciones de (N-Grams + Term Frequency-Inverse Document Frequency + Stop- Words), y Word2Vec/Glove pre-entrenados para formar los distintos vocabularios. Se evaluó algoritmos de aprendizaje automático clásico (Naïve Bayes, Multinomial Naive Bayes, Support Vector Machine, Logistic Regression, Random Forest, Decision Trees), de aprendizaje automático moderno (Vote Ensemble), y de aprendizaje profundo (Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (Bi-LSTM)). Se obtuvo dos modelos predictivos con mejor rendimiento. Un modelo con Bi-LSTM utilizando representación de Glove, para predecir la puntuación de sentimiento de la retroalimentación textual de evaluación de tarea; y un modelo con LSTM utilizando representación de Glove, para predecir la puntuación de sentimiento de la retroalimentación textual de evaluación de calidad de la evaluación. Seguidamente, se obtuvo un modelo de cálculo que contribuyó a mejorar la confiabilidad del proceso de evaluación entre pares. La puntuación de cada criterio de evaluación de tarea y evaluación de calidad de evaluación se generó con la técnica computacional de lógica difusa correlacionando puntuación numérica y sentimiento, determinando que los métodos de defuzzificación (máximo más chico, media de máximo y máximo más grande) fueron los más apropiados para este estudio. La puntuación individual de cada evaluador se obtuvo con cálculos de media de todos los criterios. La puntuación del colectivo de evaluación de tarea y rating de confianza del evaluador se obtuvo con cálculos de media/mediana del conjunto de puntuaciones individuales, determinando que la mediana tiene el mejor ajuste para generar una puntuación del colectivo confiable. Se probó la validez del modelo propuesto en 3 escenarios de educación superior: virtual asincrónico, virtual sincrónico y presencial. Se correlacionó mediante Pearson la puntación que recibe el estudiante del colectivo con la puntuación que proporciona el docente, obteniendo similaridad fuerte en el 8% de las actividades en virtual asincrónico (r=0.718-0.790), en el 25% de las actividades en virtual sincrónico (r=0.741 a 0.971) y en el 40% de las actividades en presencial (r=0.780 a 0.951), determinando que el modelo se puede aplicar en todos los escenarios de educación evaluados, y con mayor efectividad en el presencial. Finalmente, se obtuvo un modelo de calibración que contribuyó a mejorar la fiabilidad en el proceso de evaluación entre pares, ya que, mediante el ajuste de la puntuación individual de cada tarea en función del rendimiento y índice (rating) de confianza del evaluador, se logró que la relación entre la puntación del colectivo y puntuación que proporciona el docente tendiera a subir el 46% de las actividades en escenario virtual asincrónico, 69% en virtual sincrónico y 60% en presencial. Además, se evaluó si existe mejora del rendimiento estudiantil en la segunda ronda aplicando el modelo en el proceso de evaluación entre pares, mediante la prueba t de Student, se determinó que el 100% de las actividades evaluadas obtuvieron la puntuación media en la segunda ronda mayor que la primera ronda con un valor de significancia menor a 0.05, el incremento en la segunda ronda del rendimiento del estudiante en virtual asincrónico fue de 3%- 12%, en virtual sincrónico de 7%-22%, y en presencial de 15%-34%. En tal sentido, el modelo de evaluación entre pares basado en análisis de sentimiento podría implementarse como una herramienta pedagógica para apoyar al docente en enriquecer el proceso de enseñanza aprendizaje, ya que los estudiantes dieron y recibieron retroalimentaciones detalladas sobre lo correcto o incorrecto de un trabajo específico, y pudieron refutar sobre las retroalimentaciones dadas; lo que a su vez indujo que mejoraran el trabajo y el rendimiento en la segunda rondaThe assessment of open response work is a task that must be carried out by an expert; however, they represent a significant correction workload for the teacher. In this context, peer assessment has been considered as an alternative approach to address the problem. This type of assessment not only reduces the correction workload but also brings additional benefits, such as the possibility for the student to verify different solutions for the same problem and the provision of useful feedback. Changes in educational paradigms have promoted the integration of assessment methods that aim to go beyond (summative) knowledge assessment, which is more integrated into the training and learning process (formative). Formative assessment contributes significantly to the quality of learning students gain from giving and receiving feedback, and the immediate Access teachers can have to class progress. Educational institutions currently seek to gain immersed knowledge from these unstructured texts. Therefore, the general objective of this thesis has been to design a peer assessment model that helped teachers improve their teaching-learning processes through sentiment analysis methods. The research-action design methodology was applied, firstly, the state of the art on peer assessment, text mining, and computational techniques was carried out. Subsequently, a model was designed that combines peer assessment with collaborative learning and the calibrated method in several phases: a) groups of students were formed to participate in carrying out the work collaboratively, to have similar groups, but have individual differences in the peer assessment process to benefit from collaboration among students; b) a rubric was designed for data collection, where the evaluators evaluated specific aspects of the work, providing a numerical score and textual feedback for each criterion; c) the evaluators evaluated the quality of the task assessment (inverse assessment) to obtain the evaluator's confidence rating; d) the groups corrected the work based on the feedback given by the evaluators in the first round (evaluation in two rounds); e) the task assessment score was calibrated based on the performance and confidence rating of the evaluator. During the development of the model, two sets of data were obtained in Spanish, one for task assessment and the other for assessment of the quality of the assessment. The supervised machine learning approach was applied to obtain a sentiment score corresponding to specific textual feedback. Different text mining and natural language processing techniques were analyzed on the sentiment classification task, such as Bag of Words, combinations of (N-Grams+Term Frequency-Inverse Document Frequency+Stop-Words), and Word2Vec/Glove pre-trained to form the different vocabularies. Algorithms were evaluated of classic machine learning (Naïve Bayes, Multinomial Naïve Bayes, Support Vector Machine, Logistic Regression, Random Forest, Decision Trees), modern machine learning (Vote Ensemble), and deep learning (Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (Bi-LSTM)). Two predictive models with better performance were obtained. A model with Bi-LSTM using Glove's representation, to predict the sentiment score of task assessment textual feedback; and a model with LSTM using Glove's representation, to predict the sentiment score of the assessment quality assessment textual feedback. Thereafter, a calculation model was obtained that contributed to improving the reliability of the peer assessment process. The score for each task assessment criterion and quality assessment was generated with the fuzzy logic computational technique correlating numerical score and sentiment, determining that the (Smallest of Maximum, Middle of Maximum, and Largest of Maximum) defuzzification methods were the most appropriate for this study. The individual score of each evaluator was obtained with mean calculations of all the criteria. The task assessment collective score and the evaluator confidence rating were obtained with mean/median calculations of the set of individual scores, determining that the median has the best fit to generate a reliable collective score. The validity of the proposed model was tested in 3 higher education scenarios: virtual asynchronous, virtual synchronous, and face-to-face. Using Pearson, the score received by the student from the group was correlated with the score provided by the teacher, obtaining strong similarity in 8% of the activities in virtual asynchronous (r=0.718-0.790), 25% of the activities in virtual synchronous (r=0.741 to 0.971) and 40% of the activities in face-to-face (r=0.780 to 0.951), determining that the model can be applied in all the education scenarios evaluated, and with greater effectiveness in face-to-face. Finally, a calibration model was obtained that contributed to improving the reliability of the peer assessment process, since, by adjusting the individual score of each task based on the performance and confidence rating of the evaluator, it was achieved that the relationship between the score of the group and the score provided by the teacher would tend to increase in 46% of the activities in asynchronous virtual settings, 69% in synchronous virtual settings and 60% in faceto- face settings. In addition, it was evaluated if there is an improvement in student performance in the second round by applying the model in the peer assessment process, using the student's t-test, it was determined that 100% of the activities evaluated obtained the average score in the second round greater than the first round with a significance value of less than 0.05, the increase in the second round of student performance in asynchronous virtual was 3%-12%, in synchronous virtual 7%-22%, and in face-to-face 15 %-3. 4%. In this sense, the peer assessment model based on sentiment analysis could be implemented as a pedagogical tool to support the teacher in enriching the teaching-learning process, since the students gave and received detailed feedback on the correct or incorrect of a specific work, and they were able to refute the feedback given; which in turn induced improved work and performance in the second round