Chiarcos, Christian

Damova, Mariana

Liebeskind, Chaya

Silvano, Purificação

Trajanov, Dimitar

Valūnaitė Oleškevičienė, Giedrė

English

OPUS Augsburg

Workshop onDeep Learning and Neural Approachesfor Linguistic DataSkopje, North Macedonia & online30 September 2021Book of abstractsScientific committee:Eliot Bytyçi, University of PrishtinaRadovan Garabík, Ľ. Štúr Institute of Linguistics, Slovak Academy of SciencesDagmar Gromann, University of ViennaChaya Liebeskind, Jerusalem College of Technology, Lev Academic CenterGiedrė Valūnaitė Oleškevičienė, Mykolas Romeris UniversityHugo Gonçalo Oliveira, University of CoimbraPurificação Silvano, University of Porto© by respective authors, 2021Editor: Radovan GarabíkĽ. Štúr Institute of Linguistics, Slovak Academy of SciencesThis work is licensed under the Creative Commons Attribution 4.0International License. To view a copy of this license, visithttp://creativecommons.org/licenses/by/4.0/https://nexuslinguarum.eu/This publication is based upon work from COST Action CA18209 – Europeannetwork for Web-centred linguistic data science, supported by COST (EuropeanCooperation in Science and Technology).COST (European Cooperation in Science and Technology) is a funding agencyfor research and innovation networks. Our Actions help connect research initia-tives across Europe and enable scientists to grow their ideas by sharing themwith their peers. This boosts their research, career and innovation.www.cost.euCOST is supportedby the Horizon 2020Framework Programmeof the European Union ForewordDeep learning and neural approaches are indispensable in modern Natural Lan-guage Processing and generally in all kinds of linguistic data analysis tasks.This workshop is aimed at deep learning in connection with linguistic data andthe effective use of deep learning in understanding the specificities of linguisticdata. The submissions collected in this book of abstracts deal with deep learningused to improve named entity recognition; BERT in conjunction with a com-pilation of lexical patterns to automatically acquire lexico-semantic relations;using transformer models to predict discourse relations and speaker’s attitudes;using transformer models to automatically extract terminological concept sys-tems; and an automatic detection of rhetorical patterns in academic texts us-ing machine learning algorithms designed for image object detection purposestrained on the page layout and graphical elements.The workshop shows just a small fraction of the variety of problems thatmodern deep learning methods can successfully tackle, and demonstrates theusefulness of linguistic linked open data, as results of and interconnected withneural approaches.Radovan Garabík, Dagmar GromannTable of ContentsDagmar Gromann, Lennart Wachowiak, Christian Lang, BarbaraHeinischMultilingual Extraction of Terminological Concept Systems. . . . . . . . . . . . 5Giedrė Valūnaitė Oleškevičienė, Chaya Liebeskind, Dimitar Trajanov,Purificação Silvano, Christian Chiarcos, Mariana DamovaSpeaker Attitudes Detection through Discourse Markers Analysis . . . . . . 8Hugo Gonçalo OliveiraAcquiring Lexico-Semantic Knowledge from a Portuguese Masked Lan-guage Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Vasile Păiș, Maria MitrofanTowards a named entity recognition system in the Romanian legal do-main using a linked open data corpus. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16Margaux Susman, Djuddah Leijen, Nicholas Groom, Christer Johans-sonInvestigating Academic Document Structure using Object DetectionMethods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Speaker Attitudes Detection through DiscourseMarkers AnalysisGiedrė Valūnaitė Oleškevičienė1, Chaya Liebeskind2, Dimitar Trajanov3,Purificação Silvano4, Christian Chiarcos5, Mariana Damova61Institute of Humanities, Mykolas Romeris University, Vilnius, Lithuaniagentrygiedre@gmail.com2Jerusalem College of Technology, Jerusalem, Israelliebchaya@gmail.com3Ss. Cyril and Methodius University – Skopje, North Macedoniadimitar.trajanov@gmail.com4University of Porto, Portugalpuri.msilvano@gmail.com5University of Frankfurt, Frunkfurt, Germanychiarcos@informatik.uni-frankfurt.de6Mozaika Ltd, Solunska 52, Sofia 1000, Bulgariamariana.damova@mozajka.coKeywords: discourse markers, speaker attitudes detection, annotation, linguistic linked open data, transformer models, machine learning.Extended AbstractSpeaker attitude detection is important for processing opinionated text. Sur-vey data as such provide a valuable source of information and research for dif-ferent scientific disciplines. They are also of interest to practitioners such aspolicymakers,  politicians,  government bodies, educators,  journalists,  and allother stakeholders with occupations related to people and society. Survey dataprovide evidence about particular language phenomena and public attitudes toprovide a broader picture about the clusters of social attitudes. In this regard,attitudinal discourse markers play a central role in the sense that they arepointers to the speaker's attitudes. These single word or multiword expressions(MWE) are mainly drawn from syntactic classes of conjunctions, adverbials,and prepositional phrases (Fraser, 2009), as well as expressions such as  youknow,  you see,  and  I  mean (Schiffrin,  2001;  Hasselgren,  2002;  Maschler  &Schiffrin, 2015). Discourse markers are regarded as significant discourse rela-tions’ triggers, and, consequently, are largely studied (e.g. Sanders et al. 1992;Knott & Dale 1994; Wellner et al 2006; Taboada & Das 2013; Das 2014; Das& Taboada 2019; Silvano 2011). Recently, discourse relations and discoursemarker research has gained certain impetus with corpora annotation for ex-ploring  discourse  structure  in  texts,  for  example,  RST-DT English  corpus(Carlson,  Marcu  &  Okurowski  2003);  Penn  Discourse  Tree  Bank  (PDTB)(Prasad et al. 2008); SDRT Annodis French corpus (Afantenos et al., 2012).Speaker Attitudes Detection through Discourse Markers Analysis 9The large bulk of these corpora is manually annotated, mostly by trainedlinguists,  less by non-experts, and only a reduced number undergoes auto-matic/semiautomatic annotation (with human supervision).This study describes ongoing work whose ultimate goals are: (i) to collectmethods for appropriate processing of free text answers to open questions insurveys with respect to speaker attitudes identified by discourse markers; and(ii) to establish guidelines for the creation of LLOD vocabularies for discoursemarkers. In particular, this paper presents the process of constituting a multi-lingual corpus, creating an annotation schema of discourse relations for mark-ing the discourse markers, and applying machine learning transformer modelsto predict their appearance in unknown texts. We apply a two-step approachto detecting speaker attitudes by identifying discourse markers and the seman-tics of  the discourse relations they introduce in text using neural machinelearning  transformer  models  to  ensure  the  interlinking  of  multilingual  dis-course markers.To achieve the aforementioned goals, so far, we have created a parallel cor-pus containing data from 6 languages, using the publicly available TED Talktranscripts. It is an ongoing expansion of TED-EHL parallel corpus publishedin  LINDAT/CLARIN-LT  repository  http://hdl.handle.net/20.500.11821/34.The multilingual corpus contains alignments of Lithuanian, Bulgarian, Por-tuguese, Macedonian, and German languages with English as pivot languagewith a size of 1.3 million sentences. Secondly, we constitute a vocabulary ofmultiword expression that can play the role of discourse markers in text basedon theoretical insights by Schiffrin (1987) and classification provided by Fraser(2009). The next step was the manual annotation of the 2428 English-Bulgar-ian-Lithuanian aligned sentences containing the multiword expressions (MWE)as discourse markers or content expressions (1 or 0). Example (1) below classi-fies the multiword expression you know as a discourse marker (annotated 1)used to introduce a new discourse message, whereas example (2) representscontent words (annotated 0) fully integrated into the sentence. (1) That's  ridiculous.  You  know,  this  is  New York,  this  chair  will  beempty, nobody has time to sit in front of you.(2) You know some people who say “Well”The annotated corpora have been used to train machine learning models topredict the existence of discourse markers in a text. Because we had a multi-lingual dataset, we chose FastText (Joulin et al. 2016) XLM-Roberta (Con-neau et al. l 2019) as the base models. The model was fine-tuned using the k-train library (Maiya 2020),  a low-code Python library built  on top of  thestate-of-the-art Transformers library (Wolf et al. 2020). The dataset was di-vided 80-20 for train and test datasets, and the model was trained using alearning rate of 0.00001 for three epochs. The dataset was slightly unbalanced(53% records without a discourse marker and 47% with a discourse marker),so we used class balancing weights to compensate. The model fine-tuning wasrun ten times, and the average performance is reported in Table 2.Table 1 shows an example of annotated corpus used for training the trans-former models.10 G. Valūnaitė Oleškevičienė et al.Table 1: Example of annotated corpus entriesMWE SentencechunkContext DiscourseMarkerPresenceI remember And I remembered thatthe old and drunken guydestroying my statistical significance ofthe test. So I looked carefully at this guy. He was 20-some years older than anybody else in the sample. And I re-membered that the old and drunken guy came one day to the lab wanting to make some easy cash0You know But you know, these stories,because he would have pulled the mean of the group lower, giving us even stronger statistical results than we could. So we decided not to throw the guy out and to rerun the experi-ment. But you know, these stories, andlots of other experiments that we've done on conflicts of interest, basically kind of bring two points1The results of the two trained models for English is given in table 2 and figure1 below. As this is the first attempt to identify the presence of discourse mark-ers in unseen text with transformer models we think the results are promising.Table 2: Results FastText XLM-RoBERTa-LargeFastText XML-Roberta-LargeAccuracy 0.46 0.90Precision 0.65 0.87Recall 0.19 0.97Specificity 0.85 0.78F1-Score 0.30 0.90MCC 0.05 0.79Figure 1: Confusion matrices – FastText and XLM-RoBERTa-LargeRegarding the semantics of discourse markers, we are adopting ISO 24618-8annotation scheme to semantically annotate discourse relations as carriers ofspeaker attitudes in English, and Chiarcos (2014) methodology to representthem as LLOD and extend the semantic vocabularies of discourse relations(reference). Consequently, we will apply transformer models to predict the se-mantics of present discourse markers in unseen text in the 6 languages of theresearch.Speaker Attitudes Detection through Discourse Markers Analysis 11References[1] Afantenos, S., Asher, N., Benamara, F., Bras, M., Fabre, C., Ho-Dac, L.-M.,  Le  Draoulec,  A.,  Muller,  P.,  Péry-Woodley,  M.-P.,  Prévot,  L.,Rebeyrolle,  J.,  Tanguy,  L.,  Vergez-Couret,  M.  & Vieu,  L.  (2012)  Anempirical  resource  for  discovering  cognitive  principles  of  discourseorganization:  The ANNODIS corpus.  In  N.  Calzolari,  K.  Choukri,  T.Declerck, M. Uğur Doğan, B. Maegaard, J. Mariani, A. Moreno, J. Odijk& S. Piperidis (eds.), Proceedings of the 8th International Conference onLanguage  Resources  and  Evaluation  –  LREC  2012  (pp.  2727-2734).Luxembourg: European Language Resources Association.[2] Carlson, L.; Marcu, D. & Okurowsi, M. E. (2003) Building a discourse-tagged  corpus  in  the  framework  of  rhetorical  structure  theory.  InProceedings of the second Sigdial Workshop on discourse and dialogue.https://aclanthology.org/W01-1605[3] Christian Chiarcos (2014). Towards interoperable discourse annotation.Discourse features in the Ontologies of Linguistic Annotation Proceedingsof  the  Ninth  International  Conference  on  Language  Resources  andEvaluation (LREC'14). http://www.lrec-conf.org/proceedings/lrec2014/pdf/893_Paper.pdf[4] Conneau, A., Khandelwal,  K., Goyal,  N., Chaudhary, V., Wenzek, G.,Guzmán,  F.,  ...  &  Stoyanov,  V.  (2019).  Unsupervised  cross-lingualrepresentation learning at scale. arXiv preprint arXiv:1911.02116.[5] Das, D. & Taboada, M. (2019) Multiple Signals of Coherence Relations.Discourse [Online], 24: 1-38.[6] Das,  D.  (2014)  Signalling  of  Coherence  Relations  in  Discourse.  PhDdissertation. Simon Fraser University, Burnaby, Canada.https://summit.sfu.ca/item/14446[7] Fraser, B. (2009) An account of discourse markers, International reviewof Pragmatics 1(2), 293–320, Publisher: Brill[8] Hasselgren, A. (2002) Learner corpora and language testing: Small wordsas markers of learner fluency, Computer learner corpora, second languageacquisition  and  foreign  language  teaching,  143–174,  Publisher:  JohnBenjamins Amsterdam, The Netherlands.[9] Joulin, A., Grave, E., Bojanowski, P., Douze, M., Jegou, H., Mikolov, T.(2016)  Fasttext.zip:  Compressing  text  classification  models.  arXivpreprint arXiv:1612.03651[10] Knott, A. and R. Dale (1994) “Using Linguistic Phenomena to Motivatea Set of Coherence Relations”, in Discourse Processes, 18, (1), 35-62.[11] LINDAT/CLARIN-LT  repository  http://hdl.handle.net/20.500.11821/34(2021)[12] Maiya, A. S. (2020). ktrain: A low-code library for augmented machinelearning. arXiv preprint arXiv:2004.10703. 11. Maschler, Y. and Schiffrin,D.  (2015)  Discourse  markers:  Language,  meaning,  and  context,  Thehandbook  of  discourse  analysis,  1,  189-221.  Publisher:  Wiley  OnlineLibrary.12 G. Valūnaitė Oleškevičienė et al.[13] Prasad, R.; Dinesh, N.; Lee, A.; Miltsakaki, E.; Robaldo, L.; Joshi, A.and  Webber,  B.  (2008).  The  Penn  Discourse  TreeBank  2.0.  InProceedings of 6th International Conference on Language Resources andEvaluation (LREC 2008). https://aclanthology.org/L08-1093/[14] Sanders, T.; J. W. Spooren and Leo G. M. Noordman (1992) Toward aTaxonomy of Coherence Relations. Discourse Processes, 15, 1-35.[15] Schiffrin, D. (2001) Discourse markers: Language, meaning, and context,The handbook of discourse analysis 1, 54–75, Publisher: Wiley OnlineLibrary.[16] Silvano,  P.  (2011) Temporal  and rhetorical  relations:  the semantics  ofsentences  with  adverbial  subordination  in  European  Portuguese.  PhDThesis, University of Porto. https://repositorio-aberto.up.pt/handle/10216/56024?locale=pt[17] Wolf, T., Chaumond, J., Debut, L., Sanh, V., Delangue, C., Moi, A., ...& Rush, A. M. (2020, October). Transformers: State-of-the-art naturallanguage processing. In Proceedings of the 2020 Conference on EmpiricalMethods in Natural Language Processing: System Demonstrations (pp.38-45).

Speaker attitudes detection through discourse markers analysis [Extended abstract]

https://opus.bibliothek.uni-augsburg.de/opus4/files/103995/103995.pdf

Speaker attitudes detection through discourse markers analysis [Extended abstract]

Abstract

Similar works

Full text

Available Versions

OPUS Augsburg