122,344 research outputs found
Exploring the Potential of Large Language Models in Computational Argumentation
Computational argumentation has become an essential tool in various fields,
including artificial intelligence, law, and public policy. It is an emerging
research field in natural language processing (NLP) that attracts increasing
attention. Research on computational argumentation mainly involves two types of
tasks: argument mining and argument generation. As large language models (LLMs)
have demonstrated strong abilities in understanding context and generating
natural language, it is worthwhile to evaluate the performance of LLMs on
various computational argumentation tasks. This work aims to embark on an
assessment of LLMs, such as ChatGPT, Flan models and LLaMA2 models, under
zero-shot and few-shot settings within the realm of computational
argumentation. We organize existing tasks into 6 main classes and standardise
the format of 14 open-sourced datasets. In addition, we present a new benchmark
dataset on counter speech generation, that aims to holistically evaluate the
end-to-end performance of LLMs on argument mining and argument generation.
Extensive experiments show that LLMs exhibit commendable performance across
most of these datasets, demonstrating their capabilities in the field of
argumentation. We also highlight the limitations in evaluating computational
argumentation and provide suggestions for future research directions in this
field
Mining arguments in scientific abstracts: Application to argumentative quality assessment
Argument mining consists in the automatic identification of argumentative structures in natural language, a task that has been recognized as particularly challenging in the scientific domain. In this work we propose SciARG, a new annotation scheme, and apply it to the identification of argumentative units and relations in abstracts in two scientific disciplines: computational linguistics and biomedicine, which allows us to assess the applicability of our scheme to different knowledge fields. We use our annotated corpus to train and evaluate argument mining models in various experimental settings, including single and multi-task learning. We investigate the possibility of leveraging existing annotations, including discourse relations and rhetorical roles of sentences, to improve the performance of argument mining models. In particular, we explore the potential offered by a sequential transfer- learning approach in which supplementary training tasks are used to fine-tune pre-trained parameter-rich language models. Finally, we analyze the practical usability of the automatically-extracted components and relations for the prediction of argumentative quality dimensions of scientific abstracts.Agencia Nacional de Investigación e InnovaciónMinisterio de Economía, Industria y Competitividad (España
Theoretical foundations for illocutionary structure parsing
Illocutionary structure in real language use is intricate and complex, and nowhere more so than in argument and debate. Identifying this structure without any theoretical scaffolding is extremely challenging even for humans. New work in Inference Anchoring Theory has provided significant advances in such scaffolding which are helping to allow the analytical challenges of argumentation structure to be tackled. This paper demonstrates how these advances can also pave the way to automated and semi-automated research in understanding the structure of natural debate.
This paper is the extended version of the paper presented at the 11th International Conference on Computational Models of Natural Argument (CMNA 2013), 14 June 2013, Rome, Italy. It reports on the initial steps of a project on argument mining from dialogue. Note that since then the corpus size and the annotation scheme have evolved, however, the method presented here is still valid and the project has developed accordingly
Mining Legal Arguments in Court Decisions
Identifying, classifying, and analyzing arguments in legal discourse has been
a prominent area of research since the inception of the argument mining field.
However, there has been a major discrepancy between the way natural language
processing (NLP) researchers model and annotate arguments in court decisions
and the way legal experts understand and analyze legal argumentation. While
computational approaches typically simplify arguments into generic premises and
claims, arguments in legal research usually exhibit a rich typology that is
important for gaining insights into the particular case and applications of law
in general. We address this problem and make several substantial contributions
to move the field forward. First, we design a new annotation scheme for legal
arguments in proceedings of the European Court of Human Rights (ECHR) that is
deeply rooted in the theory and practice of legal argumentation research.
Second, we compile and annotate a large corpus of 373 court decisions (2.3M
tokens and 15k annotated argument spans). Finally, we train an argument mining
model that outperforms state-of-the-art models in the legal NLP domain and
provide a thorough expert-based evaluation. All datasets and source codes are
available under open lincenses at
https://github.com/trusthlt/mining-legal-arguments.Comment: to appear in Artificial Intelligence and La
The Argument Reasoning Comprehension Task: Identification and Reconstruction of Implicit Warrants
Reasoning is a crucial part of natural language argumentation. To comprehend
an argument, one must analyze its warrant, which explains why its claim follows
from its premises. As arguments are highly contextualized, warrants are usually
presupposed and left implicit. Thus, the comprehension does not only require
language understanding and logic skills, but also depends on common sense. In
this paper we develop a methodology for reconstructing warrants systematically.
We operationalize it in a scalable crowdsourcing process, resulting in a freely
licensed dataset with warrants for 2k authentic arguments from news comments.
On this basis, we present a new challenging task, the argument reasoning
comprehension task. Given an argument with a claim and a premise, the goal is
to choose the correct implicit warrant from two options. Both warrants are
plausible and lexically close, but lead to contradicting claims. A solution to
this task will define a substantial step towards automatic warrant
reconstruction. However, experiments with several neural attention and language
models reveal that current approaches do not suffice.Comment: Accepted as NAACL 2018 Long Paper; see details on the front pag
- …