Spanish sentiment analysis in Twitter at the TASS workshop

[EN] This paper describes a support vector machine-based approach to different tasks related to sentiment analysis in Twitter for Spanish. We focus on parameter optimization of the models and the combination of several models by means of voting techniques. We evaluate the proposed approach in all the tasks that were defined in the five editions of the TASS workshop, between 2012 and 2016. TASS has become a framework for sentiment analysis tasks that are focused on the Spanish language. We describe our participation in this competition and the results achieved, and then we provide an analysis of and comparison with the best approaches of the teams who participated in all the tasks defined in the TASS workshops. To our knowledge, our results exceed those published to date in the sentiment analysis tasks of the TASS workshops.This work has been partially funded by the Spanish MINECO and FEDER founds under project ASLP-MULAN: Audio, Speech and Language Processing for Multimedia Analytics, TIN2014-54288-C4-3-R.Pla Santamaría, F.; Hurtado Oliver, LF. (2018). Spanish sentiment analysis in Twitter at the TASS workshop. Language Resources and Evaluation. 52(2):645-672. https://doi.org/10.1007/s10579-017-9394-7S645672522Álvarez-López, T., Juncal-Martínez, J., Fernández-Gavilanes, M., Costa-Montenegro, E., González-Castaño, F.J., Cerezo-Costas, H. , & Celix-Salgado, D. (2015). GTI-gradiant at TASS 2015: A hybrid approach for sentiment analysis in Twitter. In Proceedings of TASS 2015: Workshop on sentiment analysis at SEPLN co-located with 31st SEPLN conference (SEPLN 2015) (pp. 35–40), Alicante, Spain, September 15, 2015.Álvarez-López, T., Fernández-Gavilanes, M., García-Méndez, S., Juncal-Martínez, J., & González-Castaño, F.J. (2016). GTI at TASS 2016: Supervised approach for aspect based sentiment analysis in Twitter. In Proceedings of TASS 2016: Workshop on sentiment analysis at SEPLN co-located with 32nd SEPLN conference (SEPLN 2016) (pp. 53–57), Salamanca, Spain, September 13th, 2016.Araque, O., Corcuera, I., Román, C., Iglesias, C. A., & Sánchez-Rada, J. F. (2015). Aspect based sentiment analysis of Spanish tweets. In Proceedings of TASS 2015: Workshop on sentiment analysis at SEPLN co-located with 31st SEPLN conference (SEPLN 2015) (pp. 29–34), Alicante, Spain, September 15, 2015.Balahur, A., & Perea-Ortega, J. M. (2013). Experiments using varying sizes and machine translated data for sentiment analysis in Twitter. In Proceedings of the TASS workshop at SEPLN 2013, IV Congreso Español de Informática.Barbosa, L., & Feng, J. (2010). Robust sentiment detection on Twitter from biased and noisy data. In Proceedings of the 23rd international conference on computational linguistics: posters, association for computational linguistics (pp. 36–44).Batista, F., & Ribeiro, R. (2012). The L2F Strategy for Sentiment Analysis and Topic Classification. Technical report, http://www.sepln.org/workshops/tass/2012/participation.php .Casasola Murillo, E., & Marín Raventós, G. (2016). Evaluación de Modelos de Representación del Texto con Vectores de Dimensiónn Reducida para Análisis de Sentimiento. In Proceedings of TASS 2016: Workshop on sentiment analysis at SEPLN co-located with 32nd SEPLN conference (SEPLN 2016) (pp. 23–28), Salamanca, Spain, September 13th, 2016.Castellano, A., Cigarrán, J. & García-Serrano, A. (2012). UNED @ TASS: Using IR techniques for topic-based sentiment analysis through divergence models. Technical report, http://www.sepln.org/workshops/tass/2012/participation.php .Castellanos-González, A., Cigarrán-Recuero, J. & García-Serrano, A. (2013). UNED LSI @ TASS 2013: Considerations about textual representation for IR based tweet classification. In: Proceedings of the TASS workshop at SEPLN 2013, IV Congreso Español de Informática.Cerón-Guzmán, J. A. (2016). JACERONG at TASS 2016: An ensemble classifier for sentiment analysis of Spanish tweets at global level. In: Proceedings of TASS 2016: Workshop on sentiment analysis at SEPLN co-located with 32nd SEPLN conference (SEPLN 2016) (pp. 35–39), Salamanca, Spain, September 13th, 2016.del-Hoyo-Alonso, R., Hupont, I., & Lacueva, F. (2013). Affective polarity word discovering by means of artificial general intelligence techniques. In Proceedings of the TASS workshop at SEPLN 2013, IV Congreso Español de Informática.del-Hoyo-Alonso, R., de la Vega Rodrigalvarez-Chamorro, M., Vea-Murguía, J., & Montañes-Salas, R. M. (2015). Ensemble algorithm with syntactical tree features to improve the opinion analysis. In Proceedings of TASS 2015: workshop on sentiment analysis at SEPLN co-located with 31st SEPLN conference (SEPLN 2015) (pp. 53–58), Alicante, Spain, September 15, 2015.Deriu, J., Gonzenbach, M., Uzdilli, F., Lucchi, A., De Luca, V., & Jaggi, M. (2016). Swisscheese at semeval-2016 task 4: Sentiment classification using an ensemble of convolutional neural networks with distant supervision. In Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016) (pp. 1124–1128), Association for Computational Linguistics, San Diego, California, http://www.aclweb.org/anthology/S16-1173 .Díaz-Galiano, M. C., & Montejo-Ráez, A. (2015). Participación de SINAI DW2Vec en TASS 2015. In Proceedings of TASS 2015: Workshop on sentiment analysis at SEPLN co-located with 31st SEPLN conference (SEPLN 2015) (pp. 59–64), Alicante, Spain, September 15, 2015.Fernández, J., Gutiérrez, Y., Tomás, D., Gómez, J. M. & Martínez-Barco, P. (2015). Evaluating a sentiment analysis approach from a business point of view. In Proceedings of TASS 2015: Workshop on sentiment analysis at SEPLN co-located with 31st SEPLN conference (SEPLN 2015) (pp. 93–98), Alicante, Spain, September 15, 2015.Fernández, J., Gutiérrez, Y., Gómez, J.M., Martínez-Barco, P., Montoyo A., & Muñoz, R. (2013). Sentiment analysis of Spanish Tweets using a ranking algorithm and skipgrams. In Proceedings of the TASS workshop at SEPLN 2013, IV Congreso Español de Informática.Frank, E., Hall, M. A., & Witten, I. H. (2016). The WEKA workbench. Online appendix for “Data mining: Practical machine learning tools and techniques” (4th ed.). Burlington: Morgan Kaufmann.Gamallo, P., García, M. & Fernández-Lanza, S. (2013). TASS: A Naive-Bayes strategy for sentiment analysis on Spanish tweets. In Proceedings of the TASS workshop at SEPLN 2013, IV Congreso Español de Informática.García Cumbreras, M. Á., Martínez Cámara, E., Villena-Román, J., & García Morera, J. (2016a). TASS 2015—The evolution of the Spanish opinion mining systems. Procesamiento del Lenguaje Natural.García Cumbreras, M. Á., Villena Román, J., Martínez Cámara, E., Díaz Galiano, M. C., Martín Valdivia, M. T., & Ureña López, L. A. (2016b). Overview of TASS 2016. In Proceedings of TASS 2016: Workshop on sentiment analysis at SEPLN co-located with 32nd SEPLN conference (SEPLN 2016) (pp. 13–21), Salamanca, Spain, September 13th, 2016.García, D., & Thelwall, M. (2013). Political alignment and emotional expression in Spanish Tweets. In Proceedings of the TASS workshop at SEPLN 2013, IV Congreso Español de Informática.Hagen, M., Potthast, M., Büchner, M., & Stein, B. (2015). Webis: An ensemble for twitter sentiment detection. In Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015) (pp. 582–589), Association for Computational Linguistics, Denver, Colorado, http://www.aclweb.org/anthology/S15-2097 .Hamdan, H., Bellot, P., & Bechet, F. (2015). Lsislif: Crf and logistic regression for opinion target extraction and sentiment polarity analysis. In Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015) (pp. 753–758), Association for Computational Linguistics, Denver, Colorado, http://www.aclweb.org/anthology/S15-2128 .Hernández Petlachi, R., & Li, X. (2014). Análisis de sentimiento sobre textos en Español basado en aproximaciones semánticas con reglas lingüísticas. In Proceedings of the TASS workshop at SEPLN 2014.Hurtado, L.F. , & Pla, F. (2014). ELiRF-UPV en TASS 2014: Análisis de Sentimientos, Detección de Tópicos y Análisis de Sentimientos de Aspectos en Twitter. In Proceedings of the TASS workshop at SEPLN 2014.Hurtado, L. F., & Pla, F. (2016). ELiRF-UPV en TASS 2016: Análisis de Sentimientos en Twitter. In Proceedings of TASS 2016: Workshop on sentiment analysis at SEPLN co-located with 32nd SEPLN conference (SEPLN 2016) (pp. 47–51), Salamanca, Spain, September 13th, 2016.Hurtado, L. F., Pla, F., & Buscaldi, D. (2015). ELiRF-UPV en TASS 2015: Análisis de Sentimientos en Twitter. In Proceedings of TASS 2015: Workshop on sentiment analysis at SEPLN co-located with 31st SEPLN conference (SEPLN 2015) (pp. 75–79), Alicante, Spain, September 15, 2015.Jansen, B. J., Zhang, M., Sobel, K., & Chowdury, A. (2009). Twitter power: Tweets as electronic word of mouth. Journal of the American Society for Information Science and Technology, 60(11), 2169–2188.Jiménez Zafra, S. M., Martínez Cámara, E., Martín Valdivia, M. T., & Ureña López, L. A. (2014) SINAI-ESMA: An unsupervised approach for sentiment analysis in Twitter. In Proceedings of the TASS workshop at SEPLN 2014.Liu, B. (2012). Sentiment analysis and opinion mining. A comprehensive introduction and survey. San Rafael: Morgan & Claypool Publishers.Liu, B., Hu, M., & Cheng, J. (2005). Opinion observer: Analyzing and comparing opinions on the web. In Proceedings of the 14th international conference on world wide web (pp. 342–351), ACM, New York, NY, USA, WWW ’05, doi: 10.1145/1060745.1060797 , http://doi.acm.org/10.1145/1060745.1060797Martínez-Cámara, E., Martín-Valdivia, M. T., Ureña-López, L. A., & Montejo-Raéz, A. (2014). Sentiment analysis in Twitter. Natural Language Engineering, 1(1), 1–28.Martínez-Cámara, E., García-Cumbreras, M.Á., Martín-Valdivia, M. T., & López, L. A. U. (2015). SINAI-EMMA: Vectores de Palabras para el Análisis de Opiniones en Twitter. In Proceedings of TASS 2015: Workshop on sentiment analysis at SEPLN co-located with 31st SEPLN conference (SEPLN 2015) (pp. 41–46), Alicante, Spain, September 15, 2015.Martín-Wanton, T., & de Albornoz, J. C. (2012). UNED at TASS 2012: Polarity classification and trending topic system. Technical report, http://www.sepln.org/workshops/tass/2012/participation.php .Martínez-Cámara, E., Ángel García-Cumbreras, M., Martín-Valdivia, M. T., & Ureña-López, L. A. (2013). SINAI-EMML: Combinación de Recursos Lingüíticos para el Análisis de la Opinión en Twitter. In Proceedings of the TASS workshop at SEPLN 2013, IV Congreso Español de Informática.Martínez-Cámara, E., Martín-Valdivia, M. T., Molina-González, M. D., & Ureña-López, L. A. (2013). Bilingual experiments on an opinion comparable corpus. In Proceedings of the 4th workshop on computational approaches to subjectivity, sentiment and social media analysis (pp. 87–93).Mendizabal, I., & Carandell, J. (2015). BittenPotato: Tweet sentiment analysis by combining multiple classifiers. In Proceedings of TASS 2015: Workshop on sentiment analysis at SEPLN co-located with 31st SEPLN conference (SEPLN 2015) (pp. 71–74), Alicante, Spain, September 15, 2015.Mohammad, S., Kiritchenko, S., & Zhu, X. (2013). Nrc-canada: Building the state-of-the-art in sentiment analysis of tweets. In Second joint conference on lexical and computational semantics (*SEM), Volume 2: Proceedings of the seventh international workshop on semantic evaluation (SemEval 2013) (pp. 321–327), Association for Computational Linguistics, Atlanta, Georgia, USA, http://www.aclweb.org/anthology/S13-2053 .Montejo-Ráez, A., & Díaz-Galiano, M. C. (2016). Participación de SINAI en TASS 2016. In Proceedings of TASS 2016: Workshop on sentiment analysis at SEPLN co-located with 32nd SEPLN conference (SEPLN 2016) (pp. 41–45), Salamanca, Spain, September 13th, 2016.Montejo-Ráez, A., Díaz-Galiano, M. C., & García-Vega, M. (2013). LSA based approach to TASS 2013. In Proceedings of the TASS workshop at SEPLN 2013, IV Congreso Español de Informática.Montejo-Ráez, A., García-Cumbreras, M., & Díaz-Galiano, M. (2014). Participación de SINAI Word2Vec en TASS 2014. In Proceedings of the TASS workshop at SEPLN 2014.Moreno-Ortiz, A., & Pérez-Hernández, C. (2012). Lexicon-based sentiment analysis of Twitter messages in Spanish. Technical report, http://www.sepln.org/workshops/tass/2012/participation.php .Nakov, P., Kozareva, Z., Ritter, A., Rosenthal, S., Stoyanov, V., & Wilson, T. (2013). SemEval-2013 Task 2: Sentiment analysis in Twitter.Nakov, P., Ritter, A., Rosenthal, S., Stoyanov, V., & Sebastiani, F. (2016). SemEval-2016 Task 4: Sentiment analysis in Twitter. In Proceedings of the 10th international workshop on semantic evaluation (pp. 1–18), Association for Computational Linguistics, San Diego, California, SemEval ’16.O’Connor, B., Krieger, M., & Ahn, D. (2010). TweetMotif: Exploratory search and topic summarization for Twitter. In Cohen, W. W. & Gosling, S. (Eds)., Proceedings of the fourth international conference on weblogs and social media, ICWSM 2010, Washington, DC, USA, May 23-26, 2010, The AAAI Press, http://www.aaai.org/ocs/index.php/ICWSM/ICWSM10/paper/view/1540 .Padró, L., & Stanilovsky, E. (2012). FreeLing 3.0: Towards Wider Multilinguality. In Proceedings of the language resources and evaluation conference (LREC 2012), ELRA, Istanbul, Turkey.Pang, B., Lee, L., & Vaithyanathan, S. (2002). Thumbs up? Sentiment classification using machine learning techniques. In Proceedings of EMNLP (pp. 79–86).Park, S. (2015). Sentiment Classification Using Sociolinguistic Clusters. In Proceedings of TASS 2015: Workshop on sentiment analysis at SEPLN co-located with 31st SEPLN conference (SEPLN 2015) (pp. 99–104), Alicante, Spain, September 15, 2015.Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.Perea-Ortega, J. M. & Balahur, A. (2014). Experiments on feature replacements for polarity classification of Spanish tweets. In Proceedings of the TASS workshop at SEPLN 2014.Perez-Rosas, V., Banea, C., & Mihalcea, R. (2012). Learning Sentiment Lexicons in Spanish. In: N. C. C. Chair, K. Choukri, T. Declerck, M. U. Doğan, B. Maegaard, J. Mariani, J. Odijk, & S. Piperidis (Eds.), Proceedings of the eight international conference on language resources and evaluation (LREC’12), European Language Resources Association (ELRA), Istanbul, Turkey.Pla, F., & Hurtado, L. F. (2013a) ELiRF-UPV en TASS-2013: Análisis de sentimientos en Twitter. In Proceedings of the TASS workshop at SEPLN 2013, IV Congreso Español de Informática.Pla, F., & Hurtado, L. F. (2013b) ELiRF-UPV en TASS-2013: Análisis de sentimientos en Twitter. In XXIX Congreso de la Sociedad Espanola para el Procesamiento del Lenguaje Natural (SEPLN 2013) TASS (pp. 220–227).Pla, F., & Hurtado, L. F. (2014a) Political tendency identification in Twitter using sentiment analysis techniques. In Proceedings of COLING 2014, the 25th international conference on computational linguistics: Technical Papers (pp. 183–192), Dublin City University and Association for Computational Linguistics, Dublin, Ireland, http://www.aclweb.org/anthology/C14-1019 .Pla, F., & Hurtado, L. F. (2014b) Sentiment analysis in Twitter for Spanish. In International conference on applications of natural language to data bases/information systems (pp. 208–213), Springer International Publishing.Quirós, A., Segura-Bedmar, I., & Martínez, P. (2016). LABDA at the 2016 TASS challenge task: Using word embeddings for the sentiment analysis task. In Proceedings of TASS 2016: workshop on sentiment analysis at SEPLN co-located with 32nd SEPLN conference (SEPLN 2016) (pp. 29–33), Salamanca, Spain, September 13th, 2016.Ramón Quevedo, J., Luaces, O., & Bahamonde, A. (2012). Multilabel classifiers with a probabilistic thresholding strategy. Pattern Recogn, 45(2), 876–883.Rosenthal, S., Nakov, P., Ritter, A., & Stoyanov, V. (2014). SemEval-2014 Task 9: Sentiment analysis in Twitter. In: P. Nakov, T. Zesch (Eds.), Proceedings of the 8th international workshop on semantic evaluation, SemEval ’14, Dublin, Ireland.Rosenthal, S., Nakov, P., Kiritchenko, S., Mohammad, S., Ritter, A., & Stoyanov, V. (2015). SemEval-2015 Task 10: Sentiment analysis in Twitter. In: Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015) (pp. 451–463), Association for Computational Linguistics, Denver, Colorado, http://www.aclweb.org/anthology/S15-2078 .Rouvier, M., & Favre, B. (2016). SENSEI-LIF at SemEval-2016 task 4: Polarity embedding fusion for robust sentiment analysis. In Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016) (pp. 202–208), Association for Computational Linguistics, San Diego, California, http://www.aclweb.org/anthology/S16-1030 .San Vicente Roncal, I., & Saralegi Urizar, X. (2014). Looking for features for supervised tweet polarity classification. In Proceedings of the TASS workshop at SEPLN 2014.Santos-Deas, M., Biran, O., McKeown, K., & Rosenthal, S. (2015). Spanish Twitter messages polarized through the lens of an english system. In Proceedings of TASS 2015: Workshop on sentiment analysis at SEPLN co-located with 31st SEPLN conference (SEPLN 2015) (pp. 81–86), Alicante, Spain, September 15, 2015.Saralegi, X., & San Vicente, I. (2012). TASS: Detecting sentiments in Spanish tweets. Technical report, http://www.sepln.org/workshops/tass/2012/participation.php .Saralegi, X., & San Vicente, I. (2013). Elhuyar at TASS 2013. In Proceedings of the TASS workshop at SEPLN 2013, IV Congreso Español de Informática.Sebastiani, F. (2002). Machine learning in automated text categorization. ACM Computing Surveys, 34(1), 1–47. doi: 10.1145/505282.505283 .Segura-Bedmar, I., Quiròs, A., & Martìnez, P. (2017). Exploring convolutional neural networks for sentiment analysis of Spanish tweets. In Proceedings of EACL (15th conference of the European chapter of the Association for Computational Linguistics) (pp. 1014–1022), Association for Computational Linguistics.Severyn, A., & Moschitti, A. (2015). Unitn: Training deep convolutional neural network for twitter sentiment classification. In Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015) (pp. 464–469), Association for Computational Linguistics, Denver, Colorado, http://www.aclweb.org/anthology/S15-2079 .Siordia, O. S., Moctezuma, D., Graff, M., Miranda-Jiménez, S., Téllez, E. S., & Villaseñor, E. (2015). Sentiment analysis for Twitter: TASS 2015. In Proceedings of TASS 2015: Workshop on sentiment analysis at SEPLN co-located with 31st SEPLN Conference (SEPLN 2015) (pp 65–70), Alicante, Spain, September 15, 2015.Sixto-Cesteros, J., Almeida, A., & López-de-Ipiña, D. (2015). DeustoTech Internet at TASS 2015: Sentiment analysis and polarity classification in Spanish tweets. In: Proceedings of TASS 2015: Workshop on sentiment analysis at SEPLN co-located with 31st SEPLN conference (SEPLN 2015) (pp. 23–28), Alicante, Spain, September 15, 2015.Trilla, A., & Alías, F. (2012). Sentiment analysis of Twitter messages based on multinomial Naive Bayes. Technical report, http://www.sepln.org/workshops/tass/2012/participation.php .Tsoumakas, G., & Katakis, I. (2007). Multi-label classification: An overview. International Journal of Data Warehousing and Mining, 2007, 1–13.Turney, P. D. (2002). Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In ACL (pp. 417–424), http://www.aclweb.org/anthology/P02-1053.pdf .Valverde-Tohalino, J., & Tejada-Cárcamo, J. (2015). Comparing supervised learning methods for classifying Spanish tweets. In Proceedings of TASS 2015: Workshop on sentiment analysis at SEPLN co-located with 31st SEPLN conference (SEPLN 2015) (pp. 87–92), Alicante, Spain, September 15, 2015.Vilares, D., Alonso, M. A., & Gómez-Rodríguez, C. (2013). LyS at TASS 2013: Analysing Spanish tweets by means of dependency parsing, semantic-oriented lexicons and psychometric word-properties. In Proceedings of the TASS workshop at SEPLN 2013, IV Congreso Español de Informática.Vilares, D., Doval, Y., Alonso, M. A. & Gómez-Rodríguez, C. (2014). LyS at TASS 2014: A prototype for extracting and analysing aspects from Spanish tweets. In Proceedings of the TASS workshop at SEPLN 2014.Vilares, D., Doval, Y., Alonso, M. A., & Gómez-Rodríguez, C. (2015). LyS at TASS 2015: Deep learning experiments for sentiment analysis on Spanish tweets. In Proceedings of TASS 2015: Workshop on sentiment analysis at SEPLN co-located with 31st SEPLN conference (SEPLN 2015) (pp. 47–52), Alicante, Spain, September 15, 2015.Villar Rodríguez, E., Torre Bastida, A. I., García Serrano, A., & González Rodríguez, M. (2013). TECNALIA-UNED @ TASS: Uso de un enfoque lingüístico para el análisis de sentimientos. In Proceedings of the TASS workshop at SEPLN 2013, IV Congreso Español de Informática.Villena-Román, J., García Morera, J., García Cumbreras, MÁ., Martínez Cámara, E., Martín Valdivia, M. T., & Ureña López, L. A. (2013a). Workshop on sentiment analysis at SEPLN 2013: An overview. In Proceedings of the TASS workshop at SEPLN 2013, Villena-Román, Julio; García Morera, Janine; García Cumbreras, Miguel Ángel; Martínez Cámara, Eugenio; Martín Valdivia, M. Teresa; Ureña López, L. Alfonso.Villena-Román, J., Lana-Serrano, S., Martínez-Cámara, E., & González-Cristóbal, J. C. (2013b). TASS-workshop on sentiment analysis at SEPLN. Procesamiento del Lenguaje Natural, 50, 37–44.Villena-Román, J., García Morera, J., García Cumbreras, MÁ., Martínez Cámara, E., Martín Valdivia, M. T., & Ureña López, L.A. (2014). Workshop on sentiment analysis at SEPLN: Overview. In Proceedings of the TASS workshop at SEPLN 2014, Villena-Román, Julio; García Morera, Janine; García Cumbreras, Miguel Ángel; Martínez Cámara, Eugenio; Martín Val

Cross-Domain Polarity Models to Evaluate User eXperience in E-learning

[EN] Virtual learning environments are growing in importance as fast as e-learning is becoming highly demanded by universities and students all over the world. This paper investigates how to automatically evaluate User eXperience in this domain using sentiment analysis techniques. For this purpose, a corpus with the opinions given by a total of 583 users (107 English speakers and 476 Spanish speakers) about three learning management systems in different courses has been built. All the collected opinions were manually labeled with polarity information (positive, negative or neutral) by three human annotators, both at the whole opinion and sentence levels. We have applied our state-of-the-art sentiment analysis models, trained with a corpus of a different semantic domain (a Twitter corpus), to study the use of cross-domain models for this task. Cross-domain models based on deep neural networks (convolutional neural networks, transformer encoders and attentional BLSTM models) have been tested. In order to contrast our results, three commercial systems for the same task (MeaningCloud, Microsoft Text Analytics and Google Cloud) were also tested. The obtained results are very promising and they give an insight to keep going the research of applying sentiment analysis tools on User eXperience evaluation. This is a pioneering idea to provide a better and accurate understanding on human needs in the interaction with virtual learning environments and a step towards the development of automatic tools that capture the feed-back of user perception for designing virtual learning environments centered in user's emotions, beliefs, preferences, perceptions, responses, behaviors and accomplishments that occur before, during and after the interaction.Partially supported by the Spanish MINECO and FEDER founds under Project TIN2017-85854-C4-2-R. Work of J.A. Gonzalez is financed under Grant PAID-01-17Sanchis-Font, R.; Castro-Bleda, MJ.; González-Barba, JÁ.; Pla Santamaría, F.; Hurtado Oliver, LF. (2021). Cross-Domain Polarity Models to Evaluate User eXperience in E-learning. Neural Processing Letters. 53:3199-3215. https://doi.org/10.1007/s11063-020-10260-5S3199321553Ba J, Kiros JR, Hinton GE (2016) Layer normalization. arxiv:1607.06450Bahdanau D, Cho K, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: 3rd international conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, conference track proceedings. arxiv:1409.0473Baziotis C, Pelekis N, Doulkeridis C (2017) Datastories at SemEval-2017 task 4: deep LSTM with attention for message-level and topic-based sentiment analysis. In: Proceedings of the 11th international workshop on semantic evaluation (SemEval-2017). Association for Computational Linguistics, Vancouver, Canada, pp 747–754Cliche M (2017) BB\_twtr at SemEval-2017 task 4: Twitter sentiment analysis with CNNs and LSTMs. In: Proceedings of the 11th international workshop on semantic evaluation (SemEval-2017). Association for Computational Linguistics, Vancouver, Canada, pp 573–580. https://doi.org/10.18653/v1/S17-2094. https://www.aclweb.org/anthology/S17-2094Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies, volume 1 (long and short papers). Association for Computational Linguistics, Minneapolis, Minnesota, pp 4171–4186. https://doi.org/10.18653/v1/N19-1423. https://www.aclweb.org/anthology/N19-1423Diaz-Galiano MC, et al (2019) Overview of TASS 2019: one more further for the global Spanish sentiment analysis corpus. In: Proceedings of the Iberian languages evaluation forum (IberLEF 2019), CEUR-WS, Bilbao, Spain, CEUR workshop proceedings, pp 550–560Godin F, Vandersmissen B, De Neve W, Van de Walle R (2015) Multimedia lab @ ACL WNUT NER shared task: named entity recognition for Twitter microposts using distributed word representations. In: Proceedings of the workshop on noisy user-generated text. Association for Computational Linguistics, Beijing, China, pp 146–153. https://doi.org/10.18653/v1/W15-4322. https://www.aclweb.org/anthology/W15-4322González J, Pla F, Hurtado L (2018) Elirf-upv en TASS 2018: Análisis de sentimientos en twitter basado en aprendizaje profundo (elirf-upv at TASS 2018: sentiment analysis in Twitter based on deep learning). In: Proceedings of TASS 2018: workshop on semantic analysis at SEPLN, TASS@SEPLN 2018, co-located with 34nd SEPLN conference (SEPLN 2018), Sevilla, Spain, September 18th, 2018, pp 37–44. http://ceur-ws.org/Vol-2172/p2_elirf_tass2018.pdfGonzález J, Hurtado L, Pla F (2019) Elirf-upv at TASS 2019: transformer encoders for Twitter sentiment analysis in Spanish. In: Proceedings of the Iberian languages evaluation forum co-located with 35th conference of the Spanish Society for Natural Language Processing, IberLEF@SEPLN 2019, Bilbao, Spain, September 24th, 2019, pp 571–578. http://ceur-ws.org/Vol-2421/TASS_paper_2.pdfGonzález JÁ, Pla F, Hurtado LF (2017) ELiRF-UPV at SemEval-2017 task 4: sentiment analysis using deep learning. In: Proceedings of the 11th international workshop on semantic evaluation (SemEval-2017). Association for Computational Linguistics, Vancouver, Canada, pp 723–727. https://doi.org/10.18653/v1/S17-2121. https://www.aclweb.org/anthology/S17-2121González JÁ, Hurtado LF, Pla F (2019) ELiRF-UPV at TASS 2019: transformer encoders for Twitter sentiment analysis in Spanish. In: Proceedings of the Iberian languages evaluation forum (IberLEF 2019), CEUR-WS, Bilbao, Spain, CEUR workshop proceedingsGoogleCloud (2019) Cloud natural language API. https://cloud.google.com/natural-language/. Accessed 27 Dec 2019Hassenzahl M, Tractinsky N (2006) User experience—a research agenda. Behav Inf Technol 25(2):91–97. https://doi.org/10.1080/01449290500330331Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735Hurtado Oliver LF, Pla F, González Barba J (2017) ELiRF-UPV at TASS 2017: sentiment analysis in Twitter based on deep learning. In: TASS 2017: workshop on semantic analysis at SEPLN, pp 29–34IBM (2019) Natural language understanding. https://www.ibm.com/watson/services/natural-language-understanding/. Accessed 27 Dec 2019ISO 9241-210:2019 (2019) Ergonomics of human-system interaction—part 210: human-centred design for interactive systems. International Standardization Organization (ISO). https://www.iso.org/standard/77520.html. Accessed 27 Dec 2019Kim Y (2014) Convolutional neural networks for sentence classification. In: Proceedings of the 2014 conference on empirical methods in natural language processing, EMNLP 2014, October 25–29, 2014, Doha, Qatar, a meeting of SIGDAT, a special interest group of the ACL, pp 1746–1751. http://aclweb.org/anthology/D/D14/D14-1181.pdfKrippendorff K (2004) Reliability in content analysis. Hum Commun Res 30(3):411–433Kujala S, Roto V, Väänänen-Vainio-Mattila K, Karapanos E, Sinnelä A (2011) UX curve: a method for evaluating long-term user experience. Interact Comput 23(5):473–483Liu B (2012) Sentiment analysis and opinion mining. A comprehensive introduction and survey. Morgan & Claypool Publishers, San RafaelLiu B, Hu M, Cheng J (2005) Opinion observer: analyzing and comparing opinions on the web. In: Proceedings of the 14th international conference on world wide web. ACM, New York, NY, USA, WWW ’05, pp 342–351. https://doi.org/10.1145/1060745.1060797Luque FM (2019) Atalaya at TASS 2019: data augmentation and robust embeddings for sentiment analysis. In: Proceedings of the Iberian languages evaluation forum (IberLEF 2019), CEUR-WS, Bilbao, Spain, CEUR workshop proceedingsManning CD, Surdeanu M, Bauer J, Finkel J, Bethard SJ, McClosky D (2014) The Stanford CoreNLP natural language processing toolkit. In: Association for computational linguistics (ACL) system demonstrations, pp 55–60. http://www.aclweb.org/anthology/P/P14/P14-5010Martínez-Cámara E, Díaz-Galiano M, García-Cumbreras M, García-Vega M, Villena-Román J (2017) Overview of TASS 2017. In: Proceedings of TASS 2017: workshop on semantic analysis at SEPLN (TASS 2017), CEUR-WS, Murcia, Spain, CEUR workshop proceedings, vol 1896MeaningCloud (2019) Demo de Analítica de Textos. https://www.meaningcloud.com/es/demos/demo-analitica-textos. Accessed 27 Dec 2019MeaningCloud (2019) MeaningCloud: Servicios web de analítica y minería de textos. https://www.meaningcloud.com/. Accessed 27 Dec 2019MicrosoftAzure (2019) Text analytics API. https://azure.microsoft.com/es-es/services/cognitive-services/text-analytics/. Accessed 27 Dec 2019Pang B, Lee L, Vaithyanathan S (2002) Thumbs up? sentiment classification using machine learning techniques. In: Proceedings of the ACL-02 conference on empirical methods in natural language processing, vol 10. Association for Computational Linguistics, pp 79–86Pla F, Hurtado LF (2018) Spanish sentiment analysis in Twitter at the TASS workshop. Lang Resour Eval 52(2):645–672. https://doi.org/10.1007/s10579-017-9394-7Rauschenberger M, Schrepp M, Cota MP, Olschner S, Thomaschewski J (2013) Efficient measurement of the user experience of interactive products. How to use the user experience questionnaire (UEQ). Example: Spanish language version. Int J Interact Multimed Artif Intell 2(1):39–45. https://doi.org/10.9781/ijimai.2013.215Rosenthal S, Farra N, Nakov P (2017) SemEval-2017 task 4: sentiment analysis in Twitter. In: Proceedings of the 11th international workshop on semantic evaluation (SemEval-2017). Association for Computational Linguistics, Vancouver, Canada, pp 502–518. https://doi.org/10.18653/v1/S17-2088. https://www.aclweb.org/anthology/S17-2088Sadr H, Pedram MM, Teshnehlab M (2019) A robust sentiment analysis method based on sequential combination of convolutional and recursive neural networks. Neural Process Lett 50:2745–2761. https://doi.org/10.1007/s11063-019-10049-1Sanchis-Font R, Castro-Bleda M, González J (2019) Applying sentiment analysis with cross-domain models to evaluate user experience in virtual learning environments. In: Rojas I, Joya G, Catala A (eds) Advances in computational intelligence. IWANN (2019). Lecture notes in computer science, vol 11506. Springer, Cham, pp 609–620Schuster M, Paliwal K (1997) Bidirectional recurrent neural networks. Trans Signal Process 45(11):2673–2681. https://doi.org/10.1109/78.650093Scott WA (1955) Reliability of content analysis: the case of nominal scale coding. Public Opin Q 19(3):321–325. https://doi.org/10.1086/266577Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng A, Potts C (2013) Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 conference on empirical methods in natural language processing. Association for Computational Linguistics, Seattle, Washington, USA, pp 1631–1642. https://www.aclweb.org/anthology/D13-1170Turney PD (2002) Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In: ACL, pp 417–424. http://www.aclweb.org/anthology/P02-1053.pdfVaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Proceedings of the 31st international conference on neural information processing systems, NIPS’17. Curran Associates Inc., USA, pp 6000–6010. http://dl.acm.org/citation.cfm?id=3295222.3295349Wilson T, Hoffmann P, Somasundaran S, Kessler J, Wiebe J, Choi Y, Cardie C, Riloff E, Patwardhan S (2005) OpinionFinder: a system for subjectivity analysis. In: Proceedings of HLT/EMNLP on interactive demonstrations. Association for Computational Linguistics, pp 34–35Zaharias P, Mehlenbacher B (2012) Editorial: exploring user experience (UX) in virtual learning environments. Int J Hum Comput Stud 70(7):475–477. https://doi.org/10.1016/j.ijhcs.2012.05.001Zhang L, Wang S, Liu B (2018) Deep learning for sentiment analysis: a survey. Wiley Interdiscip Rev Data Min Knowl Discov 8(4):e125

ELiRF-UPV at TASS 2020: TWilBERT for Sentiment Analysis and Emotion Detection in Spanish Tweets

[EN] This paper describes the participation of the ELiRF research group of the Universitat Politècnica de València in the TASS 2020 Workshop, framed within the XXXVI edition of the International Conference of the Spanish Society for the Processing of Natural Language (SEPLN). We present the approach used for the Monolingual Sentiment Analysis and Emotion Detection tasks of the workshop, as well as the results obtained. Our participation has focused mainly on employing an adaptation of BERT for text classification on the Twitter domain and the Spanish language. This system, that we have called TWilBERT, shown systematic improvements of the state of the art in almost all the tasks framed in the SEPLN conference of previous years, and also obtains the most competitive performance in the tasks addressed in this work.This work has been partially supported by the Spanish MINECO and FEDER founds under project AMIC (TIN2017-85854-C4-2-R) and by the GiSPRO project (PROMETEU/2018/176). Work of José-Ángel González is financed by Universitat Politècnica de València under grant PAID-01-17.González-Barba, JÁ.; Arias-Moncho, J.; Hurtado Oliver, LF.; Pla Santamaría, F. (2020). ELiRF-UPV at TASS 2020: TWilBERT for Sentiment Analysis and Emotion Detection in Spanish Tweets. CEUR. 179-186. http://hdl.handle.net/10251/17855817918

Sentiment Analysis in Twitter for Spanish

The final publication is available at Springer via http://dx.doi.org/ 10.1007/978-3-319-07983-7_27This paper describes a SVM-approach for Sentiment Analysis (SA) in Twitter for Spanish. This task was part of the TASS2013 workshop, which is a framework for SA that is focused on the Spanish language. We describe the approach used, and we present an experimental comparison of the approaches presented by the di erent teams that took part in the competition. We also describe the improvements that were added to our system after our participation in the competition. With these improvements, we obtained an accuracy of 62.88% and 70.25% on the SA test set for 5-level and 3-level tasks respectively. To our knowledge, these results are the best results published until now for the SA tasks of the TASS2013 workshop.This work has been funded by the projects, DIANA (MEC TIN2012-38603-C02-01) and Tímpano (MEC TIN2011-28169-C05-01).Pla Santamaría, F.; Hurtado Oliver, LF. (2014). Sentiment Analysis in Twitter for Spanish. En Natural Language Processing and Information Systems. Springer Lecture Notes in Computer Science Volume 8455 2014. 208-213. https://doi.org/10.1007/978-3-319-07983-7_27S208213Barbosa, L., Feng, J.: Robust sentiment detection on Twitter from biased and noisy data. In: Proceedings of the 23rd International Conference on Computational Linguistics: Posters, Association for Computational Linguistics, pp. 36–44 (2010)Jansen, B.J., Zhang, M., Sobel, K., Chowdury, A.: Twitter power: Tweets as electronic word of mouth. Journal of the American Society for Information Science and Technology 60(11), 2169–2188 (2009)Liu, B., Hu, M., Cheng, J.: Opinion observer: Analyzing and comparing opinions on the web. In: Proceedings of the 14th International Conference on World Wide Web, WWW 2005, pp. 342–351. ACM, New York (2005)Martínez-Cámara, E., Martín-Valdivia, M.T., Ureña-López, L.A., Montejo-Raéz, A.: Sentiment analysis in twitter. Natural Language Engineering 1(1), 1–28 (2012)O’Connor, B., Krieger, M., Ahn, D.: Tweetmotif: Exploratory search and topic summarization for twitter. In: Cohen, W.W., Gosling, S. (eds.) Proceedings of the Fourth International Conference on Weblogs and Social Media, ICWSM 2010, Washington, DC, USA, May 23-26, The AAAI Press (2010)Padró, L., Stanilovsky, E.: Freeling 3.0: Towards wider multilinguality. In: Proceedings of the Language Resources and Evaluation Conference (LREC 2012). ELRA, Istanbul (2012)Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up? sentiment classification using machine learning techniques. In: Proceedings of EMNLP, pp. 79–86 (2002)Perez-Rosas, V., Banea, C., Mihalcea, R.: Learning sentiment lexicons in spanish. In: Chair, N.C.C., Choukri, K., Declerck, T., Doǧan, M.U., Maegaard, B., Mariani, J., Odijk, J., Piperidis, S. (eds.) Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC 2012). European Language Resources Association (ELRA), Istanbul (2012)Pla, F., Hurtado, L.F.: Análisis de sentimientos en twitter. In: Proceedings of the TASS workshop at SEPLN 2013, IV Congreso Español de Informática (2013)Saralegi, X., San Vicente, I.: Elhuyar at tass 2013. In: Proceedings of the TASS workshop at SEPLN 2013, IV Congreso Español de Informática (2013)Schölkopf, B., Smola, A.J., Williamson, R.C., Bartlett, P.L.: New support vector algorithms. Neural Comput. 12(5), 1207–1245 (2000)Turney, P.D.: Thumbs up or thumbs down? semantic orientation applied to unsupervised classification of reviews. In: ACL, pp. 417–424 (2002)Villena-Román, J., García-Morera, J.: Workshop on sentiment analysis at sepln 2013: An over view. In: Proceedings of the TASS Workshop at SEPLN 2013, IV Congreso Español de Informática (2013)Vinodhini, G., Chandrasekaran, R.: Sentiment analysis and opinion mining: A survey. International Journal 2(6) (2012)Wilson, T., Hoffmann, P., Somasundaran, S., Kessler, J., Wiebe, J., Choi, Y., Cardie, C., Riloff, E., Patwardhan, S.: Opinionfinder: A system for subjectivity analysis. In: Proceedings of HLT/EMNLP on Interactive Demonstrations, Association for Computational Linguistics, pp. 34–35 (2005)Wilson, T., Kozareva, Z., Nakov, P., Rosenthal, S., Stoyanov, V., Ritter, A.: Semeval-2013 task 2: Sentiment analysis in twitter. In: Proceedings of the International Workshop on Semantic Evaluation, SemEval, vol. 13 (2013

Towards Syntactic Iberian Polarity Classification

Lexicon-based methods using syntactic rules for polarity classification rely on parsers that are dependent on the language and on treebank guidelines. Thus, rules are also dependent and require adaptation, especially in multilingual scenarios. We tackle this challenge in the context of the Iberian Peninsula, releasing the first symbolic syntax-based Iberian system with rules shared across five official languages: Basque, Catalan, Galician, Portuguese and Spanish. The model is made available.Comment: 7 pages, 5 tables. Contribution to the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA-2017) at EMNLP 201

Self-attention for Twitter sentiment analysis in Spanish

[EN] This paper describes our proposal for Sentiment Analysis in Twitter for the Spanish language. The main characteristics of the system are the use of word embedding specifically trained from tweets in Spanish and the use of self-attention mechanisms that allow to consider sequences without using convolutional nor recurrent layers. These self-attention mechanisms are based on the encoders of the Transformer model. The results obtained on the Task 1 of the TASS 2019 workshop, for all the Spanish variants proposed, support the correctness and adequacy of our proposal.This work has been partially supported by the Spanish MINECO and FEDER founds under project AMIC (TIN2017-85854-C4-2-R) and the GiSPRO project (PROMETEU/2018/176). Work of Jose-Angel Gonzalez is financed by Universitat Politecnica de Valencia under grant PAID-01-17.González-Barba, JÁ.; Hurtado Oliver, LF.; Pla Santamaría, F. (2020). Self-attention for Twitter sentiment analysis in Spanish. Journal of Intelligent & Fuzzy Systems. 39(2):2165-2175. https://doi.org/10.3233/JIFS-179881S21652175392Hochreiter, S., & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735-1780. doi:10.1162/neco.1997.9.8.173

TASS 2015 – La evolución de los sistemas de análisis de opiniones para español

El análisis de opiniones en microblogging sigue siendo una tarea de actualidad, que permite conocer la orientación de las opiniones que minuto tras minuto se publican en medios sociales en Internet. TASS es un taller de participación que tiene como finalidad promover la investigación y desarrollo de nuevos algoritmos, recursos y técnicas aplicado al análisis de opiniones en español. En este artículo se describe la cuarta edición de TASS, resumiendo las principales aportaciones de los sistemas presentados, analizando los resultados y mostrando la evolución de los mismos. Además de analizar brevemente los sistemas que se presentaron, se presenta un nuevo corpus de tweets etiquetados en el dominio político, que se desarrolló para la tarea de Análisis de Opiniones a nivel de Aspecto.Sentiment Analysis in microblogging continues to be a trendy task, which allows to understand the polarity of the opinions published in social media. TASS is a workshop whose goal is to boost the research on Sentiment Analysis in Spanish. In this paper we describe the fourth edition of TASS, showing a summary of the systems, analyzing the results to check their evolution. In addition to a brief description of the participant systems, a new corpus of tweets is presented, compiled for the Sentiment Analysis at Aspect Level task.This work has been partially supported by a grant from the Fondo Europeo de Desarrollo Regional (FEDER), REDES project (TIN2015-65136-C2-1-R) and Ciudad2020 (INNPRONTA IPT-20111006) from the Spanish Government