Argumentative Writing Support by means of Natural Language Processing

Persuasive essay writing is a powerful pedagogical tool for teaching argumentation skills. So far, the provision of feedback about argumentation has been considered a manual task since automated writing evaluation systems are not yet capable of analyzing written arguments. Computational argumentation, a recent research field in natural language processing, has the potential to bridge this gap and to enable novel argumentative writing support systems that automatically provide feedback about the merits and defects of written arguments. The automatic analysis of natural language arguments is, however, subject to several challenges. First of all, creating annotated corpora is a major impediment for novel tasks in natural language processing. At the beginning of this research, it has been mostly unknown whether humans agree on the identification of argumentation structures and the assessment of arguments in persuasive essays. Second, the automatic identification of argumentation structures involves several interdependent and challenging subtasks. Therefore, considering each task independently is not sufficient for identifying consistent argumentation structures. Third, ordinary arguments are rarely based on logical inference rules and are hardly ever in a standardized form which poses additional challenges to human annotators and computational methods. To approach these challenges, we start by investigating existing argumentation theories and compare their suitability for argumentative writing support. We derive an annotation scheme that models arguments as tree structures. For the first time, we investigate whether human annotators agree on the identification of argumentation structures in persuasive essays. We show that human annotators can reliably apply our annotation scheme to persuasive essays with substantial agreement. As a result of this annotation study, we introduce a unique corpus annotated with fine-grained argumentation structures at the discourse-level. Moreover, we pre- sent a novel end-to-end approach for parsing argumentation structures. We identify the boundaries of argument components using sequence labeling at the token level and propose a novel joint model that globally optimizes argument component types and argumentative relations for identifying consistent argumentation structures. We show that our model considerably improves the performance of local base classifiers and significantly outperforms challenging heuristic baselines. In addition, we introduce two approaches for assessing the quality of natural language arguments. First, we introduce an approach for identifying myside biases which is a well-known tendency to ignore opposing arguments when formulating arguments. Our experimental results show that myside biases can be recognized with promising accuracy using a combination of lexical features, syntactic features and features based on adversative transitional phrases. Second, we investigate for the first time the characteristics of insufficiently supported arguments. We show that insufficiently supported arguments frequently exhibit specific lexical indicators. Moreover, our experimental results indicate that convolutional neural networks significantly outperform several challenging baselines

Parsing Argumentation Structures in Persuasive Essays

In this article, we present a novel approach for parsing argumentation structures. We identify argument components using sequence labeling at the token level and apply a new joint model for detecting argumentation structures. The proposed model globally optimizes argument component types and argumentative relations using integer linear programming. We show that our model considerably improves the performance of base classifiers and significantly outperforms challenging heuristic baselines. Moreover, we introduce a novel corpus of persuasive essays annotated with argumentation structures. We show that our annotation scheme and annotation guidelines successfully guide human annotators to substantial agreement. This corpus and the annotation guidelines are freely available for ensuring reproducibility and to encourage future research in computational argumentation.Comment: Under review in Computational Linguistics. First submission: 26 October 2015. Revised submission: 15 July 201

THE "POWER" OF TEXT PRODUCTION ACTIVITY IN COLLABORATIVE MODELING : NINE RECOMMENDATIONS TO MAKE A COMPUTER SUPPORTED SITUATION WORK

Language is not a direct translation of a speaker’s or writer’s knowledge or intentions. Various complex processes and strategies are involved in serving the needs of the audience: planning the message, describing some features of a model and not others, organizing an argument, adapting to the knowledge of the reader, meeting linguistic constraints, etc. As a consequence, when communicating about a model, or about knowledge, there is a complex interaction between knowledge and language. In this contribution, we address the question of the role of language in modeling, in the specific case of collaboration over a distance, via electronic exchange of written textual information. What are the problems/dimensions a language user has to deal with when communicating a (mental) model? What is the relationship between the nature of the knowledge to be communicated and linguistic production? What is the relationship between representations and produced text? In what sense can interactive learning systems serve as mediators or as obstacles to these processes

ANNOTATION MODEL FOR LOANWORDS IN INDONESIAN CORPUS: A LOCAL GRAMMAR FRAMEWORK

There is a considerable number for loanwords in Indonesian language as it has been, or even continuously, in contact with other languages. The contact takes place via different media; one of them is via machine readable medium. As the information in different languages can be obtained by a mouse click these days, the contact becomes more and more intense. This paper aims at proposing an annotation model and lexical resource for loanwords in Indonesian. The lexical resource is applied to a corpus by a corpus processing software called UNITEX. This software works under local grammar framewor

An exploratory study into automated précis grading

Automated writing evaluation is a popular research field, but the main focus has been on evaluating argumentative essays. In this paper, we consider a different genre, namely précis texts. A précis is a written text that provides a coherent summary of main points of a spoken or written text. We present a corpus of English précis texts which all received a grade assigned by a highly-experienced English language teacher and were subsequently annotated following an exhaustive error typology. With this corpus we trained a machine learning model which relies on a number of linguistic, automatic summarization and AWE features. Our results reveal that this model is able to predict the grade of précis texts with only a moderate error margin

Argumentation Mining in User-Generated Web Discourse

The goal of argumentation mining, an evolving research field in computational linguistics, is to design methods capable of analyzing people's argumentation. In this article, we go beyond the state of the art in several ways. (i) We deal with actual Web data and take up the challenges given by the variety of registers, multiple domains, and unrestricted noisy user-generated Web discourse. (ii) We bridge the gap between normative argumentation theories and argumentation phenomena encountered in actual data by adapting an argumentation model tested in an extensive annotation study. (iii) We create a new gold standard corpus (90k tokens in 340 documents) and experiment with several machine learning methods to identify argument components. We offer the data, source codes, and annotation guidelines to the community under free licenses. Our findings show that argumentation mining in user-generated Web discourse is a feasible but challenging task.Comment: Cite as: Habernal, I. & Gurevych, I. (2017). Argumentation Mining in User-Generated Web Discourse. Computational Linguistics 43(1), pp. 125-17

THE STRATEGY OF THE TEXT AND THE STRUCTURAL RELATIONSTO EXERCISE SUNDANESE CRITICS’ IDEOLOGICAL HEGEMONY

The action of mind control in Media is executed to reproduce dominance and hegemony. This mind control, however, should be performed less resist and even find “natural”. Van Dijk in Schiffrin (2001:357) argues discursive, a function of the structures and strategies of text, involve in mind control. To perform it, the use of particular strategy may trigger the use of structural relation. In reality, how Ajip Rosidi acted to control Sundaneses may lead to the questions: (1)cwhat textual strategy is applied in the discourse, and (2) what structural relations are developed to reproduce Sundanese critics’ ideological hegemony

An Exploratory Application of Rhetorical Structure Theory to Detect Coherence Errors in L2 English Writing: Possible Implications for Automated Writing Evaluation Software

This paper presents an initial attempt to examine whether Rhetorical Structure Theory (RST) (Mann & Thompson, 1988) can be fruitfully applied to the detection of the coherence errors made by Taiwanese low-intermediate learners of English. This investigation is considered warranted for three reasons. First, other methods for bottom-up coherence analysis have proved ineffective (e.g., Watson Todd et al., 2007). Second, this research provides a preliminary categorization of the coherence errors made by first language (L1) Chinese learners of English. Third, second language discourse errors in general have received little attention in applied linguistic research. The data are 45 written samples from the LTTC English Learner Corpus, a Taiwanese learner corpus of English currently under construction. The rationale of this study is that diagrams which violate some of the rules of RST diagram formation will point to coherence errors. No reliability test has been conducted since this work is at an initial stage. Therefore, this study is exploratory and results are preliminary. Results are discussed in terms of the practicality of using this method to detect coherence errors, their possible consequences about claims for a typical inductive content order in the writing of L1 Chinese learners of English, and their potential implications for Automated Writing Evaluation (AWE) software, since discourse organization is one of the essay characteristics assessed by this software. In particular, the extent to which the kinds of errors detected through the RST analysis match those located by Criterion (Burstein, Chodorow, & Leachock, 2004), a well-known AWE software by Educational Testing Service (ETS), is discussed