Annotating arguments in a corpus of opinion articles

Interest in argument mining has resulted in an increasing number of argument annotated corpora. However, most focus on English texts with explicit argumentative discourse markers, such as persuasive essays or legal documents. Conversely, we report on the first extensive and consolidated Portuguese argument annotation project focused on opinion articles. We briefly describe the annotation guidelines based on a multi-layered process and analyze the manual annotations produced, highlighting the main challenges of this textual genre. We then conduct a comprehensive inter-annotator agreement analysis, including argumentative discourse units, their classes and relations, and resulting graphs. This analysis reveals that each of these aspects tackles very different kinds of challenges. We observe differences in annotator profiles, motivating our aim of producing a non-aggregated corpus containing the insights of every annotator. We note that the interpretation and identification of token-level arguments is challenging; nevertheless, tasks that focus on higher-level components of the argument structure can obtain considerable agreement. We lay down perspectives on corpus usage, exploiting its multi-faceted nature

Parsimonious Argument Annotations for Hate Speech Counter-narratives

We present an enrichment of the Hateval corpus of hate speech tweets (Basile et. al 2019) aimed to facilitate automated counter-narrative generation. Comparably to previous work (Chung et. al. 2019), manually written counter-narratives are associated to tweets. However, this information alone seems insufficient to obtain satisfactory language models for counter-narrative generation. That is why we have also annotated tweets with argumentative information based on Wagemanns (2016), that we believe can help in building convincing and effective counter-narratives for hate speech against particular groups. We discuss adequacies and difficulties of this annotation process and present several baselines for automatic detection of the annotated elements. Preliminary results show that automatic annotators perform close to human annotators to detect some aspects of argumentation, while others only reach low or moderate level of inter-annotator agreement

Malaysian learners’ argumentative writing in English: A contrastive, corpus-driven study

Research on learner English is by now an established sub-discipline in corpus linguistics, yet few studies exist on Malaysian learners. This thesis explores the difficulties that Malaysian learners of English face when producing argumentative essays, focussing on their overuse of particular linguistic features. WordSmith Tools (Scott, 2012) is used to analyse and compare two corpora: The Malaysian Corpus of Students’ Argumentative Writing (MCSAW): Version 2, consisting of 1,460 Malaysian students’ argumentative essays; and the Louvain Corpus of Native English Essays (LOCNESS), which is a corpus of native English essays written by British and American students and is used as a reference language variety here. The software enables analysis of keywords (words that are over-used in MCSCAW), collocates or surrounding words of the keywords, and concordances, which are used to examine the keywords in context. Crucially, it also allows examination of the ‘range’ of linguistic features (i.e. by how many students a feature is employed) – an under-used but crucial affordance of this software programme that is exploited in this thesis for down-sampling purposes. The thesis combines quantitative and qualitative corpus linguistic techniques, with keywords providing the starting point for in-depth qualitative analysis using concordancing. This corpus-driven analysis of MCSAW identifies typical features of the writing style of Malaysian learners’ writing of English, particularly the overuse of can and we (including the highly frequent bundle we can), and the lack of discourse-organising markers. Analysis of key words and key bundles is complemented with collocation analysis and concordancing of the highly frequent modal verb can as well as the highly frequent first person plural pronoun we, which both have a high range across the corpus. The concordances are carefully and systematically examined to explore the ways in which these over-used linguistic items are actually employed in their co-text by the Malaysian writers. While results show some similarities in both learner corpus and reference language variety, Malaysian learners tend to demonstrate higher writer visibility overall. One possible explanation lies in the influence of the national language (Malay). The thesis also identifies repeated sentences that occur in more than one essay, which implies either plagiarism on the learners’ part or a particular teaching strategy (templates or phrases that are provided to students). This finding has significant implications for corpus design (in terms of the need for more topic variation) as well as methodological significance (in terms of the advantages and disadvantages of using the ‘range’ feature for down-sampling), which are also discussed in this thesis. In sum, this thesis makes a new contribution to corpus linguistic research on learner English and will have implications for the development of teaching practices for Malaysian learners of English

Analyses and Comparisons of Three Lexical Features in Native and Nonnative Academic English Writing

Built upon the Contrastive Interlanguage Analysis (CIA) framework, this corpus-based research analyzes three lexical features (lexical diversity, lexical sophistication, and cohesion) in native and nonnative English writers\u27 academic writing and examines the potential differences in lexical performance 1) between native and nonnative English writers and 2) across all writers from various language backgrounds. The differences in lexical performance in academic writing between native and nonnative English writers and the unique characteristics of writers from different language backgrounds suggest the necessity of targeted academic writing instruction based upon learner needs. Using text length as the covariate, two Multivariate Analysis of Covariate (MANCOVA) were conducted with language background as the Independent Variable and the three lexical features as the Dependent Variables. The results revealed that nonnative English writers demonstrated significantly lower performance in lexical sophistication than did native English writers. In terms of the comparison between writers from different language backgrounds, the results suggested statistically significant differences in all three aspects of lexical features. Pedagogical implications for vocabulary instruction in academic writing for nonnative English writers include emphasizing the mastery of academic, low-frequency, and discipline-specific vocabulary. In addition, improving nonnative writers\u27 vocabulary size and lexical diversity can offer these learners more options to build cohesion in academic writing at a deeper level. Moreover, the results of this study highlight the wide but often under-considered variability within any language group as individual learner differences come into play, thereby downplaying the idea that writers of any given language tend to perform homogenously. Instructors should acknowledge the unique writing characteristics of different nonnative writers and their varied learner needs. Thus, targeted instruction is essential to provide effective enhancement to nonnative English writers\u27 lexical performance in academic writing

Argumentation models and their use in corpus annotation: practice, prospects, and challenges

The study of argumentation is transversal to several research domains, from philosophy to linguistics, from the law to computer science and artificial intelligence. In discourse analysis, several distinct models have been proposed to harness argumentation, each with a different focus or aim. To analyze the use of argumentation in natural language, several corpora annotation efforts have been carried out, with a more or less explicit grounding on one of such theoretical argumentation models. In fact, given the recent growing interest in argument mining applications, argument-annotated corpora are crucial to train machine learning models in a supervised way. However, the proliferation of such corpora has led to a wide disparity in the granularity of the argument annotations employed. In this paper, we review the most relevant theoretical argumentation models, after which we survey argument annotation projects closely following those theoretical models. We also highlight the main simplifications that are often introduced in practice. Furthermore, we glimpse other annotation efforts that are not so theoretically grounded but instead follow a shallower approach. It turns out that most argument annotation projects make their own assumptions and simplifications, both in terms of the textual genre they focus on and in terms of adapting the adopted theoretical argumentation model for their own agenda. Issues of compatibility among argument-annotated corpora are discussed by looking at the problem from a syntactical, semantic, and practical perspective. Finally, we discuss current and prospective applications of models that take advantage of argument-annotated corpora

Towards an authentic argumentation literacy test

A central goal of education is to improve argumentation literacy. How do we know how well this goal is achieved? Can we measure argumentation literacy? The present study is a preliminary step towards measuring the efficacy of education with regards to argumentation literacy. Tests currently in use to determine critical thinking skills are often similar to IQ-tests in that they predominantly measure logical and mathematical abilities. Thus, they may not measure the various other skills required in understanding authentic argumentation. To identify the elements of argumentation literacy, this exploratory study begins by surveying introductory textbooks within argumentation theory, critical thinking, and rhetoric. Eight main abilities have been identified. Then, the study outlines an Argumentation Literacy Test that would comprise these abilities suggested by the literature. Finally, the study presents results from a pilot of a version of such a test and discusses needs for further development

한국 대학생들의 논증적 에세이에 나타난 절과 구 복잡성의 발달

학위논문(석사) -- 서울대학교대학원 : 사범대학 외국어교육과(영어전공), 2023. 2. 오선영.영어 글쓰기 발달에 관한 연구들은 문법적 복잡성(grammatical complexity)을 학습자의 능숙도를 구별하는 중요한 지표로 인식하고 있다. 초기 연구들은 주로 절 복잡성(clausal complexity)에 기반해 문법적 복잡성을 측정하였지만, 최근 연구들은 구 복잡성(phrasal complexity)에 초점을 두고 있다. 이러한 변화는 절 복잡성이 일상 대화가 가진 특징으로 글쓰기의 초기 발달 단계를 나타내는 반면, 구 복잡성, 특히 명사구의 복잡성은 학문적 글(academic writing)이 가진 복잡성의 전형으로써 높은 수준의 발달 단계를 나타낸다는 인식에 기반하고 있다. 하지만 일부 연구들은 명사구의 복잡성이 글쓰기 능숙도와 큰 관련이 없다는 상반된 결과를 보이고 있는데, 이는 대부분의 연구들이 학습자 모국어가 문법적 복잡성에 미치는 영향을 고려하지 않고 다양한 모국어를 가진 학습자들에 의해 만들어진 코퍼스를 사용했기 때문일 수 있다. 이에 본 연구는 한국인 대학생들이 작성한 글을 분석하여 절과 구의 복잡성이 글쓰기 능숙도와 연관성이 있는지 살펴보고, 그러한 연관성에 크게 기여한 복잡성 특징들을 바탕으로 문법적 복잡성의 발달 패턴을 추정하고자 하였다. 또한 학생들의 글을 질적으로 분석하여, 특정 복잡성 특징을 구현할 때 자주 쓰이는 어휘와 오류 빈도 및 유형을 파악함으로써 능숙도 집단 간의 차이를 더 자세히 묘사하고자 하였다. 본 연구에 사용된 코퍼스는 연세 영어 학습자 코퍼스(Yonsei English Learner Corpus, YELC 2011)에서 추출한 234개의 논증적 에세이로 구성되어 있으며, 이는 CEFR에 기반하여 초급, 중급, 고급의 글쓰기 능숙도를 나타내는 세 개의 하위 코퍼스로 구분되었다. 품사 태깅된 코퍼스를 바탕으로 정규표현식(regular expressions)을 사용하여, Biber et al. (2011)이 제안한 발달단계에 있는 9개의 절 복잡성 특징과 8개의 구 복잡성 특징을 추출하여 각각의 빈도를 계산하였다. 피어슨 카이제곱검정(a Pearson Chi-square test) 결과, 글쓰기 능숙도가 절과 구의 복잡성과 유의한 연관성이 있다는 결론이 도출되었다. 사후검정으로 잔차 분석(a residual analysis)을 수행한 결과, 특히 5개 복잡성 특징이 이러한 연관성에 크게 기여했음이 밝혀졌다. 주목할 만한 발견은 각 능숙도 집단의 주요 복잡성 특징이 Biber et al. (2011)이 제안한 발달단계와 일치하며 따라서 한국인 대학생의 발달 패턴이 두 개의 매개변수, 즉 (1) 구조적 형태와 (2) 통사적 기능에 의해 설명될 수 있다는 점이다. 즉, 한국 대학생들의 문법적 복잡성은 (i) 절의 구성 성분으로 기능하는 정형 종속절(finite dependent clauses functioning as clause constituents)인 부사절의 빈번한 사용에서 (ii) 명사구의 구성 성분으로 기능하는 정형 종속절(finite clause types function as NP constituents)인 WH 관계절에 대한 의존을 거쳐 (iii) 명사구의 구성 성분으로 기능하는 종속구(dependent phrasal structures functioning as noun phrase constituents)인 of 전치사구에 대한 선호로 발달하는 것으로 나타났다. 예상과 달리, 명사의 선수식어(premodifier)로 사용되는 형용사 및 명사의 빈도는 글쓰기 능숙도와 큰 연관성이 없는 것으로 나타났다. 이에 관해 학생들의 글을 질적 분석한 결과, 첫째, 초급 수준의 글은 쓰기 지시문(writing prompts)에 제시된 형용사+명사 조합을 반복적으로 사용하는 경향을 보였다. 둘째, 명사+명사 구조와 관련한 오류가 능숙도가 높아질수록 현저히 낮아지는 경향을 보였다. 마지막으로, 보어절(complement clauses)과 관련해서는 모든 능숙도 수준의 학생들이 매우 한정적인 종류의 통제 명사(controlling nouns)를 사용했으며, 학문적인 글 보다는 일상 대화에서 쓰이는 통제 동사(controlling verbs)를 사용하였다. 이러한 연구 결과는 크게 세가지 교육적 함의를 시사한다. 첫째, 경험적으로 도출된 문법적 복잡성의 발달 단계를 상세한 평가 척도 설명자(rating scale descriptors) 개발과 보다 맞춤화 된 수업 설계를 위해 활용해야 한다. 둘째, 학문적인 글에서 보어절과 함께 자주 사용되는 통제 명사 및 동사에 대한 교실 수업을 통해, 학습자들이 문법적 구조를 학문적인 어휘로 실현할 수 있도록 해야 한다. 마지막으로, 특히 명사를 선수식하는 명사 및 관계대명사절의 사용에 있어 학습자의 글에서 자주 발견되는 오류를 시정함으로써, 문법 구조 사용에 대한 정확성을 향상시켜야 한다.Studies that explore L2 writing development identify grammatical complexity as a primary discriminator for different proficiency levels of L2 writers. In the 1990s, grammatical complexity in L2 writing was often measured by clausal complexity, but the kind of complexity that has recently received particular attention is phrasal complexity. Such a move follows the recognition that clausal complexity represents the complexity of conversation and beginning levels of writing development, whereas phrasal complexity, specifically noun phrase complexity, represents the complexity of academic writing and advanced developmental levels. Some L2 writing studies, however, have yielded conflicting results, showing that phrasal features as noun modifiers have little predictive power for writing quality. One possible reason underlying these inconsistent results might be that most studies in this area have used corpus data from learners of heterogenous L1 backgrounds with no consideration for the significant effect of L1 on the use of complexity features in L2 writing. Thus, this study analyzed essay samples produced only by L1 Korean writers to investigate whether clausal and phrasal complexity is associated with L2 writing proficiency and, if so, what developmental patterns can be observed based on complexity features that contribute substantially to the association. A qualitative analysis of student writing was followed up to provide a detailed description of proficiency-level differences, especially with respect to lexical realizations and error types associated with specific complexity features. The corpus used in the present study contained 234 argumentative essays written by first-year college students, including 78 low-rated essays (A1 and A1+ levels of the CEFR), 78 mid-rated essays (B1 and B1+ levels of the CEFR), and 78 high-rated essays (B2+, C1, and C2 levels of the CEFR). Drawing on Biber et al.s (2011) developmental index, the nine clausal and eight phrasal complexity features were extracted from the tagged corpus using regular expressions to measure the frequency of each feature. The result of a Pearson Chi-square test demonstrated a statistically significant association between the three proficiency levels and the use of clausal and phrasal complexity features. The post-hoc residual analysis revealed five complexity features with great contribution to the association: finite adverbial clause, noun complement clause, WH relative clause, prepositional phrase (of), and prepositional phrase (other). Especially noteworthy is the finding that the main source of complexity at each proficiency level agrees with its corresponding developmental stage reported by Biber et al. (2011), and thus, developmental patterns for Korean college students are successfully explained by two parameters: (1) structural form (finite dependent clauses vs. dependent phrases) and (2) syntactic function (clause constituents vs. noun phrase constituents). Specifically, the development proceeds from (i) clausal complexity mainly via finite adverbial clauses (i.e., finite dependent clauses functioning as clause constituents); through (ii) the intermediate stage of heavy reliance on WH relative clauses (i.e., finite clause types functioning as noun phrase constituents); to finally (iii) phrasal complexity primarily via prepositional phrases (of) (i.e., phrasal structures functioning as noun phrase constituents). Surprisingly, premodifying adjectives and nouns were found to have no significant association with L2 writing proficiency despite being noun-modifying phrasal features. The subsequent qualitative analysis of student writing, however, illustrated greater proficiency of the highly rated essays in using these features in two regards. First, the lower-rated essays drew much more heavily on adjective-noun sequences presented in writing prompts than the higher-rated essays. Second, the number of errors in the composition of noun-noun sequences noticeably decreased in the higher-rated essays. The qualitative observation concerning that-complement clauses, on the other hand, identified the reliance on a limited set of controlling nouns and conversational styles of controlling verbs in student writing across proficiency levels. Three main pedagogical implications are provided based on the findings: (i) the use of empirically derived developmental stages to create detailed rating scale descriptors and provide more customized writing courses on the use of complexity features; (ii) the need for classroom instruction on common academic controlling nouns and verbs used in that complement clauses given the importance of academically oriented lexical realizations of grammatical structures; and (iii) the need to address recurrent errors, particularly in terms of using premodifying nouns and relative clauses.CHAPTER 1. INTRODUCTION 1 1.1 Background of the Study 1 1.2 Purpose of the Study 4 1.3 Research Questions 5 1.4 Organization of the Thesis 6 CHAPTER 2. LITERATURE REVIEW 8 2.1 Grammatical Complexity in L2 Writing 8 2.1.1 Definition of Grammatical Complexity 9 2.1.2 Grammatical Complexity in L2 Writing Studies 13 2.2 Criticism of Traditional Measures of Grammatical Complexity 15 2.2.1 Reductiveness and Redundancy of Length- and Subordination-based Measures 16 2.2.2 Inappropriateness of the T-unit Approach to the Assessment of Writing Development 21 2.3 Measures of Grammatical Complexity in L2 Writing 24 2.3.1 Clausal and Phrasal Complexity in Relation to L2 Writing Development 25 2.3.2 Studies on Clausal and Phrasal Complexity in L2 Writing 31 2.4 Variation in the Use of Grammatical Complexity Features 36 2.4.1 The Effect of L1 Background 37 2.4.2 The Effect of Genre 43 2.4.3 The Effect of Timing Condition 46 CHAPTER 3. METHODOLOGY 50 3.1 Learner Corpus 50 3.1.1 Description of YELC 2011 50 3.1.2 Description of a Subset of YELC 2011 used in the Study 53 3.2 Grammatical Complexity Measures 55 3.3 Corpus Tagging and Automatic Extraction 59 3.4 Data Analysis 65 CHAPTER 4. RESULTS AND DISCUSSION 70 4.1 Descriptive Statistics 70 4.2 The Association between L2 Writing Proficiency and Grammatical Complexity 76 4.3 The Developmental Patterns of Grammatical Complexity 77 4.4 The Grammatical Complexity Features with Great Contribution to the Association 84 4.4.1 Finite Adverbial Clauses 84 4.4.2 Prepositional Phrases as Nominal Postmodifiers 92 4.4.3 WH Relative Clauses 100 4.4.4 Finite Complement Clauses Controlled by Nouns 106 4.5 The Grammatical Complexity Features with Little Contribution to the Association 112 4.5.1 Premodifying Adjectives 113 4.5.2 Nouns as Nominal Premodifiers 120 4.5.3 Finite Complement Clauses Controlled by Verbs or Adjectives 125 CHAPTER 5. CONCLUSION 136 5.1 Major Findings 136 5.2 Pedagogical Implications 139 5.3 Limitations and Prospect for Future Research 142 REFERENCES 145 APPENDICES 161 ABSTRACT IN KOREAN 165석