Cross-Domain Polarity Models to Evaluate User eXperience in E-learning

[EN] Virtual learning environments are growing in importance as fast as e-learning is becoming highly demanded by universities and students all over the world. This paper investigates how to automatically evaluate User eXperience in this domain using sentiment analysis techniques. For this purpose, a corpus with the opinions given by a total of 583 users (107 English speakers and 476 Spanish speakers) about three learning management systems in different courses has been built. All the collected opinions were manually labeled with polarity information (positive, negative or neutral) by three human annotators, both at the whole opinion and sentence levels. We have applied our state-of-the-art sentiment analysis models, trained with a corpus of a different semantic domain (a Twitter corpus), to study the use of cross-domain models for this task. Cross-domain models based on deep neural networks (convolutional neural networks, transformer encoders and attentional BLSTM models) have been tested. In order to contrast our results, three commercial systems for the same task (MeaningCloud, Microsoft Text Analytics and Google Cloud) were also tested. The obtained results are very promising and they give an insight to keep going the research of applying sentiment analysis tools on User eXperience evaluation. This is a pioneering idea to provide a better and accurate understanding on human needs in the interaction with virtual learning environments and a step towards the development of automatic tools that capture the feed-back of user perception for designing virtual learning environments centered in user's emotions, beliefs, preferences, perceptions, responses, behaviors and accomplishments that occur before, during and after the interaction.Partially supported by the Spanish MINECO and FEDER founds under Project TIN2017-85854-C4-2-R. Work of J.A. Gonzalez is financed under Grant PAID-01-17Sanchis-Font, R.; Castro-Bleda, MJ.; González-Barba, JÁ.; Pla Santamaría, F.; Hurtado Oliver, LF. (2021). Cross-Domain Polarity Models to Evaluate User eXperience in E-learning. Neural Processing Letters. 53:3199-3215. https://doi.org/10.1007/s11063-020-10260-5S3199321553Ba J, Kiros JR, Hinton GE (2016) Layer normalization. arxiv:1607.06450Bahdanau D, Cho K, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: 3rd international conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, conference track proceedings. arxiv:1409.0473Baziotis C, Pelekis N, Doulkeridis C (2017) Datastories at SemEval-2017 task 4: deep LSTM with attention for message-level and topic-based sentiment analysis. In: Proceedings of the 11th international workshop on semantic evaluation (SemEval-2017). Association for Computational Linguistics, Vancouver, Canada, pp 747–754Cliche M (2017) BB\_twtr at SemEval-2017 task 4: Twitter sentiment analysis with CNNs and LSTMs. In: Proceedings of the 11th international workshop on semantic evaluation (SemEval-2017). Association for Computational Linguistics, Vancouver, Canada, pp 573–580. https://doi.org/10.18653/v1/S17-2094. https://www.aclweb.org/anthology/S17-2094Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies, volume 1 (long and short papers). Association for Computational Linguistics, Minneapolis, Minnesota, pp 4171–4186. https://doi.org/10.18653/v1/N19-1423. https://www.aclweb.org/anthology/N19-1423Diaz-Galiano MC, et al (2019) Overview of TASS 2019: one more further for the global Spanish sentiment analysis corpus. In: Proceedings of the Iberian languages evaluation forum (IberLEF 2019), CEUR-WS, Bilbao, Spain, CEUR workshop proceedings, pp 550–560Godin F, Vandersmissen B, De Neve W, Van de Walle R (2015) Multimedia lab @ ACL WNUT NER shared task: named entity recognition for Twitter microposts using distributed word representations. In: Proceedings of the workshop on noisy user-generated text. Association for Computational Linguistics, Beijing, China, pp 146–153. https://doi.org/10.18653/v1/W15-4322. https://www.aclweb.org/anthology/W15-4322González J, Pla F, Hurtado L (2018) Elirf-upv en TASS 2018: Análisis de sentimientos en twitter basado en aprendizaje profundo (elirf-upv at TASS 2018: sentiment analysis in Twitter based on deep learning). In: Proceedings of TASS 2018: workshop on semantic analysis at SEPLN, TASS@SEPLN 2018, co-located with 34nd SEPLN conference (SEPLN 2018), Sevilla, Spain, September 18th, 2018, pp 37–44. http://ceur-ws.org/Vol-2172/p2_elirf_tass2018.pdfGonzález J, Hurtado L, Pla F (2019) Elirf-upv at TASS 2019: transformer encoders for Twitter sentiment analysis in Spanish. In: Proceedings of the Iberian languages evaluation forum co-located with 35th conference of the Spanish Society for Natural Language Processing, IberLEF@SEPLN 2019, Bilbao, Spain, September 24th, 2019, pp 571–578. http://ceur-ws.org/Vol-2421/TASS_paper_2.pdfGonzález JÁ, Pla F, Hurtado LF (2017) ELiRF-UPV at SemEval-2017 task 4: sentiment analysis using deep learning. In: Proceedings of the 11th international workshop on semantic evaluation (SemEval-2017). Association for Computational Linguistics, Vancouver, Canada, pp 723–727. https://doi.org/10.18653/v1/S17-2121. https://www.aclweb.org/anthology/S17-2121González JÁ, Hurtado LF, Pla F (2019) ELiRF-UPV at TASS 2019: transformer encoders for Twitter sentiment analysis in Spanish. In: Proceedings of the Iberian languages evaluation forum (IberLEF 2019), CEUR-WS, Bilbao, Spain, CEUR workshop proceedingsGoogleCloud (2019) Cloud natural language API. https://cloud.google.com/natural-language/. Accessed 27 Dec 2019Hassenzahl M, Tractinsky N (2006) User experience—a research agenda. Behav Inf Technol 25(2):91–97. https://doi.org/10.1080/01449290500330331Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735Hurtado Oliver LF, Pla F, González Barba J (2017) ELiRF-UPV at TASS 2017: sentiment analysis in Twitter based on deep learning. In: TASS 2017: workshop on semantic analysis at SEPLN, pp 29–34IBM (2019) Natural language understanding. https://www.ibm.com/watson/services/natural-language-understanding/. Accessed 27 Dec 2019ISO 9241-210:2019 (2019) Ergonomics of human-system interaction—part 210: human-centred design for interactive systems. International Standardization Organization (ISO). https://www.iso.org/standard/77520.html. Accessed 27 Dec 2019Kim Y (2014) Convolutional neural networks for sentence classification. In: Proceedings of the 2014 conference on empirical methods in natural language processing, EMNLP 2014, October 25–29, 2014, Doha, Qatar, a meeting of SIGDAT, a special interest group of the ACL, pp 1746–1751. http://aclweb.org/anthology/D/D14/D14-1181.pdfKrippendorff K (2004) Reliability in content analysis. Hum Commun Res 30(3):411–433Kujala S, Roto V, Väänänen-Vainio-Mattila K, Karapanos E, Sinnelä A (2011) UX curve: a method for evaluating long-term user experience. Interact Comput 23(5):473–483Liu B (2012) Sentiment analysis and opinion mining. A comprehensive introduction and survey. Morgan & Claypool Publishers, San RafaelLiu B, Hu M, Cheng J (2005) Opinion observer: analyzing and comparing opinions on the web. In: Proceedings of the 14th international conference on world wide web. ACM, New York, NY, USA, WWW ’05, pp 342–351. https://doi.org/10.1145/1060745.1060797Luque FM (2019) Atalaya at TASS 2019: data augmentation and robust embeddings for sentiment analysis. In: Proceedings of the Iberian languages evaluation forum (IberLEF 2019), CEUR-WS, Bilbao, Spain, CEUR workshop proceedingsManning CD, Surdeanu M, Bauer J, Finkel J, Bethard SJ, McClosky D (2014) The Stanford CoreNLP natural language processing toolkit. In: Association for computational linguistics (ACL) system demonstrations, pp 55–60. http://www.aclweb.org/anthology/P/P14/P14-5010Martínez-Cámara E, Díaz-Galiano M, García-Cumbreras M, García-Vega M, Villena-Román J (2017) Overview of TASS 2017. In: Proceedings of TASS 2017: workshop on semantic analysis at SEPLN (TASS 2017), CEUR-WS, Murcia, Spain, CEUR workshop proceedings, vol 1896MeaningCloud (2019) Demo de Analítica de Textos. https://www.meaningcloud.com/es/demos/demo-analitica-textos. Accessed 27 Dec 2019MeaningCloud (2019) MeaningCloud: Servicios web de analítica y minería de textos. https://www.meaningcloud.com/. Accessed 27 Dec 2019MicrosoftAzure (2019) Text analytics API. https://azure.microsoft.com/es-es/services/cognitive-services/text-analytics/. Accessed 27 Dec 2019Pang B, Lee L, Vaithyanathan S (2002) Thumbs up? sentiment classification using machine learning techniques. In: Proceedings of the ACL-02 conference on empirical methods in natural language processing, vol 10. Association for Computational Linguistics, pp 79–86Pla F, Hurtado LF (2018) Spanish sentiment analysis in Twitter at the TASS workshop. Lang Resour Eval 52(2):645–672. https://doi.org/10.1007/s10579-017-9394-7Rauschenberger M, Schrepp M, Cota MP, Olschner S, Thomaschewski J (2013) Efficient measurement of the user experience of interactive products. How to use the user experience questionnaire (UEQ). Example: Spanish language version. Int J Interact Multimed Artif Intell 2(1):39–45. https://doi.org/10.9781/ijimai.2013.215Rosenthal S, Farra N, Nakov P (2017) SemEval-2017 task 4: sentiment analysis in Twitter. In: Proceedings of the 11th international workshop on semantic evaluation (SemEval-2017). Association for Computational Linguistics, Vancouver, Canada, pp 502–518. https://doi.org/10.18653/v1/S17-2088. https://www.aclweb.org/anthology/S17-2088Sadr H, Pedram MM, Teshnehlab M (2019) A robust sentiment analysis method based on sequential combination of convolutional and recursive neural networks. Neural Process Lett 50:2745–2761. https://doi.org/10.1007/s11063-019-10049-1Sanchis-Font R, Castro-Bleda M, González J (2019) Applying sentiment analysis with cross-domain models to evaluate user experience in virtual learning environments. In: Rojas I, Joya G, Catala A (eds) Advances in computational intelligence. IWANN (2019). Lecture notes in computer science, vol 11506. Springer, Cham, pp 609–620Schuster M, Paliwal K (1997) Bidirectional recurrent neural networks. Trans Signal Process 45(11):2673–2681. https://doi.org/10.1109/78.650093Scott WA (1955) Reliability of content analysis: the case of nominal scale coding. Public Opin Q 19(3):321–325. https://doi.org/10.1086/266577Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng A, Potts C (2013) Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 conference on empirical methods in natural language processing. Association for Computational Linguistics, Seattle, Washington, USA, pp 1631–1642. https://www.aclweb.org/anthology/D13-1170Turney PD (2002) Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In: ACL, pp 417–424. http://www.aclweb.org/anthology/P02-1053.pdfVaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Proceedings of the 31st international conference on neural information processing systems, NIPS’17. Curran Associates Inc., USA, pp 6000–6010. http://dl.acm.org/citation.cfm?id=3295222.3295349Wilson T, Hoffmann P, Somasundaran S, Kessler J, Wiebe J, Choi Y, Cardie C, Riloff E, Patwardhan S (2005) OpinionFinder: a system for subjectivity analysis. In: Proceedings of HLT/EMNLP on interactive demonstrations. Association for Computational Linguistics, pp 34–35Zaharias P, Mehlenbacher B (2012) Editorial: exploring user experience (UX) in virtual learning environments. Int J Hum Comput Stud 70(7):475–477. https://doi.org/10.1016/j.ijhcs.2012.05.001Zhang L, Wang S, Liu B (2018) Deep learning for sentiment analysis: a survey. Wiley Interdiscip Rev Data Min Knowl Discov 8(4):e125

Separate cortical stages in amodal completion revealed by functional magnetic resonance adaptation

<p>Abstract</p> <p>Background</p> <p>Objects in our environment are often partly occluded, yet we effortlessly perceive them as whole and complete. This phenomenon is called visual amodal completion. Psychophysical investigations suggest that the process of completion starts from a representation of the (visible) physical features of the stimulus and ends with a completed representation of the stimulus. The goal of our study was to investigate both stages of the completion process by localizing both brain regions involved in processing the physical features of the stimulus as well as brain regions representing the completed stimulus.</p> <p>Results</p> <p>Using fMRI adaptation we reveal clearly distinct regions in the visual cortex of humans involved in processing of amodal completion: early visual cortex – presumably V1 -processes the local contour information of the stimulus whereas regions in the inferior temporal cortex represent the completed shape. Furthermore, our data suggest that at the level of inferior temporal cortex information regarding the original local contour information is not preserved but replaced by the representation of the amodally completed percept.</p> <p>Conclusion</p> <p>These findings provide neuroimaging evidence for a multiple step theory of amodal completion and further insights into the neuronal correlates of visual perception.</p

Multi-Jet Event Rates in Deep Inelastic Scattering and Determination of the Strong Coupling Constant

Jet event rates in deep inelastic ep scattering at HERA are investigated applying the modified JADE jet algorithm. The analysis uses data taken with the H1 detector in 1994 and 1995. The data are corrected for detector and hadronization effects and then compared with perturbative QCD predictions using next-to-leading order calculations. The strong coupling constant alpha_S(M_Z^2) is determined evaluating the jet event rates. Values of alpha_S(Q^2) are extracted in four different bins of the negative squared momentum transfer~\qq in the range from 40 GeV2 to 4000 GeV2. A combined fit of the renormalization group equation to these several alpha_S(Q^2) values results in alpha_S(M_Z^2) = 0.117+-0.003(stat)+0.009-0.013(syst)+0.006(jet algorithm).Comment: 17 pages, 4 figures, 3 tables, this version to appear in Eur. Phys. J.; it replaces first posted hep-ex/9807019 which had incorrect figure 4

Measurements of Transverse Energy Flow in Deep-Inelastic Scattering at HERA

Measurements of transverse energy flow are presented for neutral current deep-inelastic scattering events produced in positron-proton collisions at HERA. The kinematic range covers squared momentum transfers Q^2 from 3.2 to 2,200 GeV^2, the Bjorken scaling variable x from 8.10^{-5} to 0.11 and the hadronic mass W from 66 to 233 GeV. The transverse energy flow is measured in the hadronic centre of mass frame and is studied as a function of Q^2, x, W and pseudorapidity. A comparison is made with QCD based models. The behaviour of the mean transverse energy in the central pseudorapidity region and an interval corresponding to the photon fragmentation region are analysed as a function of Q^2 and W.Comment: 26 pages, 8 figures, submitted to Eur. Phys.

Measurement of D* Meson Cross Sections at HERA and Determination of the Gluon Density in the Proton using NLO QCD

With the H1 detector at the ep collider HERA, D* meson production cross sections have been measured in deep inelastic scattering with four-momentum transfers Q^2>2 GeV2 and in photoproduction at energies around W(gamma p)~ 88 GeV and 194 GeV. Next-to-Leading Order QCD calculations are found to describe the differential cross sections within theoretical and experimental uncertainties. Using these calculations, the NLO gluon momentum distribution in the proton, x_g g(x_g), has been extracted in the momentum fraction range 7.5x10^{-4}< x_g <4x10^{-2} at average scales mu^2 =25 to 50 GeV2. The gluon momentum fraction x_g has been obtained from the measured kinematics of the scattered electron and the D* meson in the final state. The results compare well with the gluon distribution obtained from the analysis of scaling violations of the proton structure function F_2.Comment: 27 pages, 9 figures, 2 tables, submitted to Nucl. Phys.

Searches at HERA for Squarks in R-Parity Violating Supersymmetry

A search for squarks in R-parity violating supersymmetry is performed in e^+p collisions at HERA at a centre of mass energy of 300 GeV, using H1 data corresponding to an integrated luminosity of 37 pb^(-1). The direct production of single squarks of any generation in positron-quark fusion via a Yukawa coupling lambda' is considered, taking into account R-parity violating and conserving decays of the squarks. No significant deviation from the Standard Model expectation is found. The results are interpreted in terms of constraints within the Minimal Supersymmetric Standard Model (MSSM), the constrained MSSM and the minimal Supergravity model, and their sensitivity to the model parameters is studied in detail. For a Yukawa coupling of electromagnetic strength, squark masses below 260 GeV are excluded at 95% confidence level in a large part of the parameter space. For a 100 times smaller coupling strength masses up to 182 GeV are excluded.Comment: 32 pages, 14 figures, 3 table

Measurement of Leading Proton and Neutron Production in Deep Inelastic Scattering at HERA

Deep--inelastic scattering events with a leading baryon have been detected by the H1 experiment at HERA using a forward proton spectrometer and a forward neutron calorimeter. Semi--inclusive cross sections have been measured in the kinematic region 2 <= Q^2 <= 50 GeV^2, 6.10^-5 <= x <= 6.10^-3 and baryon p_T <= MeV, for events with a final state proton with energy 580 <= E' <= 740 GeV, or a neutron with energy E' >= 160 GeV. The measurements are used to test production models and factorization hypotheses. A Regge model of leading baryon production which consists of pion, pomeron and secondary reggeon exchanges gives an acceptable description of both semi-inclusive cross sections in the region 0.7 <= E'/E_p <= 0.9, where E_p is the proton beam energy. The leading neutron data are used to estimate for the first time the structure function of the pion at small Bjorken--x.Comment: 30 pages, 9 figures, 2 tables, submitted to Eur. Phys.

Deep-Inelastic Inclusive ep Scattering at Low x and a Determination of alpha_s

A precise measurement of the inclusive deep-inelastic e^+p scattering cross section is reported in the kinematic range 1.5<= Q^2 <=150 GeV^2 and 3*10^(-5)<= x <=0.2. The data were recorded with the H1 detector at HERA in 1996 and 1997, and correspond to an integrated luminosity of 20 pb^(-1). The double differential cross section, from which the proton structure function F_2(x,Q^2) and the longitudinal structure function F_L(x,Q^2) are extracted, is measured with typically 1% statistical and 3% systematic uncertainties. The measured partial derivative (dF_2(x,Q^2)/dln Q^2)_x is observed to rise continuously towards small x for fixed Q^2. The cross section data are combined with published H1 measurements at high Q^2 for a next-to-leading order DGLAP QCD analysis.The H1 data determine the gluon momentum distribution in the range 3*10^(-4)<= x <=0.1 to within an experimental accuracy of about 3% for Q^2 =20 GeV^2. A fit of the H1 measurements and the mu p data of the BCDMS collaboration allows the strong coupling constant alpha_s and the gluon distribution to be simultaneously determined. A value of alpha _s(M_Z^2)=0.1150+-0.0017 (exp) +0.0009-0.0005 (model) is obtained in NLO, with an additional theoretical uncertainty of about +-0.005, mainly due to the uncertainty of the renormalisation scale.Comment: 68 pages, 24 figures and 18 table

Forward pi^0 Production and Associated Transverse Energy Flow in Deep-Inelastic Scattering at HERA

Deep-inelastic positron-proton interactions at low values of Bjorken-x down to x \approx 4.10^-5 which give rise to high transverse momentum pi^0 mesons are studied with the H1 experiment at HERA. The inclusive cross section for pi^0 mesons produced at small angles with respect to the proton remnant (the forward region) is presented as a function of the transverse momentum and energy of the pi^0 and of the four-momentum transfer Q^2 and Bjorken-x. Measurements are also presented of the transverse energy flow in events containing a forward pi^0 meson. Hadronic final state calculations based on QCD models implementing different parton evolution schemes are confronted with the data.Comment: 27 pages, 8 figures and 3 table