Argumentation models and their use in corpus annotation: practice, prospects, and challenges

The study of argumentation is transversal to several research domains, from philosophy to linguistics, from the law to computer science and artificial intelligence. In discourse analysis, several distinct models have been proposed to harness argumentation, each with a different focus or aim. To analyze the use of argumentation in natural language, several corpora annotation efforts have been carried out, with a more or less explicit grounding on one of such theoretical argumentation models. In fact, given the recent growing interest in argument mining applications, argument-annotated corpora are crucial to train machine learning models in a supervised way. However, the proliferation of such corpora has led to a wide disparity in the granularity of the argument annotations employed. In this paper, we review the most relevant theoretical argumentation models, after which we survey argument annotation projects closely following those theoretical models. We also highlight the main simplifications that are often introduced in practice. Furthermore, we glimpse other annotation efforts that are not so theoretically grounded but instead follow a shallower approach. It turns out that most argument annotation projects make their own assumptions and simplifications, both in terms of the textual genre they focus on and in terms of adapting the adopted theoretical argumentation model for their own agenda. Issues of compatibility among argument-annotated corpora are discussed by looking at the problem from a syntactical, semantic, and practical perspective. Finally, we discuss current and prospective applications of models that take advantage of argument-annotated corpora

ChangeMyView Through Concessions: Do Concessions Increase Persuasion?

In Discourse Studies concessions are considered among those argumentative strategies that increase persuasion. We aim to empirically test this hypothesis by calculating the distribution of argumentative concessions in persuasive vs. non-persuasive comments from the the ChangeMyView subreddit. This constitutes a challenging task since concessions do not always bear an argumentative role and are expressed through polysemous lexical markers. Drawing from a theoretically-informed typology of concessions, we first conduct a crowdsourcing task to label a set of polysemous lexical markers as introducing an argumentative concession relation or not. Second, we present a self-training method to automatically identify argumentative concessions using linguistically motivated features. While we achieve a moderate F1 of 57.4% via the self-training method, our subsequent error analysis highlights that the self training method is able to generalize and identify other types of concessions that are argumentative, but were not considered in the annotation guidelines. Our findings from the manual labeling and the classification experiments indicate that the type of argumentative concessions we investigated is almost equally likely to be used in winning and losing arguments. While this result seems to contradict theoretical assumptions, we provide some reasons related to the ChangeMyView subreddit

Detecting Influencers in Social Media Discussions

In the past decade we have been privileged to witness the creation and revolution of social media on the World Wide Web. The abundance of content available on the web allows us to analyze the way people interact and the roles they play in a conversation on a large scale. One such role is influencer in the conversation. Detecting influence can be useful for successful advertisement strategies, detecting terrorist leaders and political campaigning. We explore influence in discussion forums, weblogs, and micro-blogs using several components that have been found to be indicators of influence. Our components are author traits, agreement, claims, argumentation, persuasion, credibility, and certain dialog patterns. In the first portion of this thesis we describe each of our system components. Each of these components is motivated by social science through Robert Cialdini’s “Weapons of Influence” [Cialdini, 2007]. The weapons of influence are Reciprocation, Commitment and Consistency, Social Proof, Liking, Authority, and Scarcity. We then show the method and experiments for classifying each component. In the second part of this thesis we classify influencers across five online genres and analyze which features are most indicative of influencers in each genre. The online genres we explore are Wikipedia Talk Pages, LiveJournal weblogs, Political Forum discussions, Create Debate debate discussions, and Twitter microblog conversations. First, we describe a rich suite of features that were generated using each of the system components. Then, we describe our experiments and results including using domain adaptation to exploit the data from multiple online genres. Finally, we also provide a detailed analysis of a single weapon of influence, social proof, and its impact in detecting influence in Wikipedia Talk Pages. This provides a single example of the usefulness of providing comprehensive components in the detection of influence. The contributions of this thesis include a system for predicting who the influencers are in online discussion forums. We provide an evaluation of a rich set of features inspired by social science. In our system, each feature set used to detect influence is complex and computed by a system component. This allows us to provide a detailed analysis as to why the person was chosen as an influencer. We also provide a comparison of differences across several online discussion datasets and exploit the differences across the different genres to provide further improvements in influence detection

ChangeMyView Through Concessions: Do Concessions Increase Persuasion?

In Discourse Studies concessions are considered among those argumentative strategies that increase persuasion. We aim to empirically test this hypothesis by calculating the distribution of argumentative concessions in persuasive vs. non-persuasive comments from the the ChangeMyView subreddit. This constitutes a challenging task since concessions do not always bear an argumentative role and are expressed through polysemous lexical markers. Drawing from a theoretically-informed typology of concessions, we first conduct a crowdsourcing task to label a set of polysemous lexical markers as introducing an argumentative concession relation or not. Second, we present a selftraining method to automatically identify argumentative concessions using linguistically motivated features. While we achieve a moderate F1 of 57.4% via the self-training method, our subsequent error analysis highlights that the self training method is able to generalize and identify other types of concessions that are argumentative, but were not considered in the annotation guidelines. Our findings from the manual labeling and the classification experiments indicate that the type of argumentative concessions we investigated is almost equally likely to be used in winning and losing arguments. While this result seems to contradict theoretical assumptions, we provide some reasons related to the ChangeMyView subreddit

A Survey of Available Corpora For Building Data-Driven Dialogue Systems: The Journal Version

During the past decade, several areas of speech and language understanding have witnessed substantial breakthroughs from the use of data-driven models. In the area of dialogue systems, the trend is less obvious, and most practical systems are still built through significant engineering and expert knowledge. Nevertheless, several recent results suggest that data-driven approaches are feasible and quite promising. To facilitate research in this area, we have carried out a wide survey of publicly available datasets suitable for data-driven learning of dialogue systems. We discuss important characteristics of these datasets, how they can be used to learn diverse dialogue strategies, and their other potential uses. We also examine methods for transfer learning between datasets and the use of external knowledge. Finally, we discuss appropriate choice of evaluation metrics for the learning objective