Argumentation Element Annotation Modeling using XLNet

This study demonstrates the effectiveness of XLNet, a transformer-based language model, for annotating argumentative elements in persuasive essays. XLNet's architecture incorporates a recurrent mechanism that allows it to model long-term dependencies in lengthy texts. Fine-tuned XLNet models were applied to three datasets annotated with different schemes - a proprietary dataset using the Annotations for Revisions and Reflections on Writing (ARROW) scheme, the PERSUADE corpus, and the Argument Annotated Essays (AAE) dataset. The XLNet models achieved strong performance across all datasets, even surpassing human agreement levels in some cases. This shows XLNet capably handles diverse annotation schemes and lengthy essays. Comparisons between the model outputs on different datasets also revealed insights into the relationships between the annotation tags. Overall, XLNet's strong performance on modeling argumentative structures across diverse datasets highlights its suitability for providing automated feedback on essay organization.Comment: 28 page

The implementation of dialogue-based pedagogy to improve written argumentation amongst secondary school students in Malaysia

The purpose of this study is to find solutions on how to improve secondary school students’ persuasive argumentative English essay writing. The participants of this study are groups of ESL students aged 13 and 17 who live and study in a sub-urban area in Malaysia. All students and teachers converse amongst themselves using the Malay language on a daily basis while English language is merely used during classroom interaction time. Not only do they have very little opportunity to communicate using English language in their daily lives and for academic purposes, they also have limited opportunity to learn how to argue persuasively in their English classroom. Thus, they have difficulties in writing two-sided argumentative essays in English. The teaching-to-the-test culture has taken its toll on students’ writing performance when writing argumentative essays. In order to help students to score well in examination, teachers often overlook the need to teach critical thinking skills for the English subject. They focus solely on writing narrative essays as these essays require less critical thinking skill from the students. The Design-Based Research is employed to solve this problem of writing persuasive argumentative essays. Based on the pre-intervention essays written by the participants, it is believed that their difficulties are because of two major factors; insufficient English language skills and no exposure to persuasive argumentation skills. The initial design framework asserts that students should improve their persuasive argumentative essay writing if they are initially exposed to face-to-face group argumentation. However, the findings from the exploratory study revealed that face-to-face group argumentation is unmanageable in the context studied. Hence, an online learning intervention was considered to support secondary school students to improve their written argument. It was developed underpinned by design principles based on Exploratory Talk to achieve persuasive argumentation. The prototype online intervention was tested and developed through a series of iterations. Findings from Iteration 1 show that only a small number of students manage to write two-sided essays because most of them have an extreme attitude when writing about an issue and display a lack of positive transfer from group to individual argumentation. Prior to Iteration 2, the prototype intervention was adapted to tackle the extreme attitude and negative transfer issues by highlighting five elements: face-to-face classroom practice, focus more on three main ground rules, argument game, role of teachers during group argumentation and the use of argument map during the post-intervention essay writing. The findings demonstrate that all students in the second iteration wrote argumentative essays which are more persuasive. The final design framework developed in this study suggests a design framework that could be used by future researchers and ESL teachers at secondary school level who are interested in improving students’ persuasive argumentative essays

Getting engaged: dialogistic positioning in novice academic discussion writing

1This chapter builds on a proposal put forward in Swain (2007a), that in effective academic argumentation, a very important role is played by the ENGAGEMENT system (White, 1998, 2003; Martin & White, 2005) of appraisal theory (Feez et al, 2008; Martin, 2000; White, 2005; Martin & White, 2005). The analysis for engagement of a corpus of non-native speaker undergraduate discussion writing in English showed that engagement resources are key to successful writing. Engagement resources are implicated in evaluative coherence; the degree to which the writer engages with the referenced arguments, coherently with the authorial position adopted; the inclusion of other viewpoints, and the mediation of attitude. From this it follows that the engagement framework has implications for EAP course content and methodology. The framework includes expressions and structures either ignored in traditional EAP programmes or taught with little attention to their dialogic, rhetorical functionality. Focus on engagement enables awareness of a fuller range of resources for persuasion and of their dialogic functions than is traditionally envisaged, providing help with perhaps one of the most challenging forms of academic writing for students.This chapter builds on a proposal put forward in Swain (2007a), that in effective academic argumentation, a very important role is played by the ENGAGEMENT system (White, 1998, 2003; Martin & White, 2005) of appraisal theory (Feez et al, 2008; Martin, 2000; White, 2005; Martin & White, 2005). The analysis for engagement of a corpus of non-native speaker undergraduate discussion writing in English showed that engagement resources are key to successful writing. Engagement resources are implicated in evaluative coherence; the degree to which the writer engages with the referenced arguments, coherently with the authorial position adopted; the inclusion of other viewpoints, and the mediation of attitude. From this it follows that the engagement framework has implications for EAP course content and methodology. The framework includes expressions and structures either ignored in traditional EAP programmes or taught with little attention to their dialogic, rhetorical functionality. Focus on engagement enables awareness of a fuller range of resources for persuasion and of their dialogic functions than is traditionally envisaged, providing help with perhaps one of the most challenging forms of academic writing for students.nonemixedSwain; E.A.Swain, ELIZABETH ANN

Rubrics and Revision: What are the Effects of 3\u3csup\u3erd\u3c/sup\u3e Graders Using Rubrics to Self-Assess or Peer-Assess Drafts of Writing?

The purpose of this quasi-experimental research thesis was to determine the effects of rubric-referenced peer-revision and self-assessment on the writing drafts of 3rd grade students. A convenience sample of students in existing classrooms engaged in two persuasive writing assignments. The first assignment established a baseline score for comparison purposes. During the second assignment, a peer-revision group and self-assessment group received different interventions that focused on revision guided by a rubric. A third control group did not receive an intervention. Student opinions toward the usefulness of the treatments were also gathered through a questionnaire that was delivered after the writing assignments were complete. The utilization of rubrics to assist peers in revision had a statistically significant, positive effect on student scores during the second persuasive writing assignment. The treatment of rubric-referenced self-assessment did not have an overall positive effect on student scores during the second assignment. The control group’s scores decreased slightly on the second assignment. Almost every student in the peer-revision group thought the treatment was beneficial for student writing. In comparison, a little more than half the students in the self-assessment group considered the treatment to be useful in helping them achieve higher scores or become more proficient writers

Automatic text scoring using neural networks

Automated Text Scoring (ATS) provides a cost-effective and consistent alternative to human marking. However, in order to achieve good performance, the predictive features of the system need to be manually engineered by human experts. We introduce a model that forms word representations by learning the extent to which specific words contribute to the text’s score. Using Long-Short Term Memory networks to represent the meaning of texts, we demonstrate that a fully automated framework is able to achieve excellent results over similar approaches. In an attempt to make our results more interpretable, and inspired by recent advances in visualizing neural networks, we introduce a novel method for identifying the regions of the text that the model has found more discriminative.This is the accepted manuscript. It is currently embargoed pending publication

Argumentation Mining in User-Generated Web Discourse

The goal of argumentation mining, an evolving research field in computational linguistics, is to design methods capable of analyzing people's argumentation. In this article, we go beyond the state of the art in several ways. (i) We deal with actual Web data and take up the challenges given by the variety of registers, multiple domains, and unrestricted noisy user-generated Web discourse. (ii) We bridge the gap between normative argumentation theories and argumentation phenomena encountered in actual data by adapting an argumentation model tested in an extensive annotation study. (iii) We create a new gold standard corpus (90k tokens in 340 documents) and experiment with several machine learning methods to identify argument components. We offer the data, source codes, and annotation guidelines to the community under free licenses. Our findings show that argumentation mining in user-generated Web discourse is a feasible but challenging task.Comment: Cite as: Habernal, I. & Gurevych, I. (2017). Argumentation Mining in User-Generated Web Discourse. Computational Linguistics 43(1), pp. 125-17