The purpose of this study is to deploy a novel methodology for classifying argumentative support (or evidence) in arguments. The methodology shows that Tree Kernel can discriminate between different types of argumentative evidence with high scores, while keeping a good generalization. Moreover, the results of two different Tree Kernels are evaluated

Liga, Davide

peer reviewedThe purpose of this study is to deploy a novel methodology for classifying argumentative support (or evidence) in arguments. The methodology shows that Tree Kernel can discriminate between different types of argumentative evidence with high scores, while keeping a good generalization. Moreover, the results of two different Tree Kernels are evaluated

Open Repository and Bibliography - Luxembourg

COMPARING TREE KERNELS PERFORMANCES IN ARGUMENTATIVE EVIDENCE CLASSIFICATIONDavide Liga11 CIRSFID, Università degli Studi di Bologna, (e-mail: davide.liga2@unibo.it)ABSTRACT: The purpose of this study is to deploy a novel methodology for clas-sifying argumentative support (or evidence) in arguments. The methodology showsthat Tree Kernel can discriminate between different types of argumentative evidencewith high scores, while keeping a good generalization. Moreover, the results of twodifferent Tree Kernels are evaluated.KEYWORDS: argument mining, argumentation, tree kernels, evidence detection.1 Introduction, the Argument Mining pipelineArgument Mining is relatively new field in the scientific community and sev-eral works have been written about this topic in the last few years (Cabrio &Villata, 2018, Lippi & Torroni, 2015). Broadly, its aim is to detect argumenta-tive units from data and predict their relations. The achievement of this aim isnot trivial and involves the resolution of multiple problems. In fact, ArgumentMining can be seen as a a multifaceted problem and it is often consideredas a pipeline. For example, Cabrio & Villata (Cabrio & Villata, 2018) de-scribed it as a pipeline composed of two steps, where the first step involves theidentification of arguments and the second involves the prediction of argumentrelations. The first step includes both the classification argumentative vs non-argumentative and the identification of the arguments’ components (claims,premises, etc.) along with their boundaries. The second step comprises pre-dicting the nature of argument relations (e.g. supports, attacks) and the linksbetween evidences and claims. The two steps are strictly dependent on the un-derlying argumentative model (e.g. the Waltonian claim/evidence dichotomy).In this paper, a further step is considered, which involves fitting the achievedargumentative units into an Argument Schemes model, e.g. Walton’s classifi-cation of Argument Schemes (Walton et al. , 2008). To achieve this aim, it iscrucial to create classifiers capable of differentiating among different kinds ofargumentative evidence (e.g. argument from Expert Opinion, argument from292Example). The proposed methodology is based on a Tree Kernel approach ableto discriminate between different kinds of argumentative support.2 Related WorksThis work presents a method for classifying evidence typology within argu-ments using Tree Kernels (Moschitti, 2006), since being able to classify dif-ferent kinds of support is crucial when dealing with Argument Schemes.The advantage of Tree Kernels is the possibility to calculate similaritiesbetween different tree-structured data instead of engineering sophisticated fea-tures. Tree Kernels have already been used successfully in several NLP-relatedworks. However, the application of Tree Kernel in the domain of ArgumentMining has been relatively limited. One of the first implementations was pre-sented by Rooney et Al. (Rooney et al. , 2012). Three years later, Lippi andTorroni suggested to exploit the ability of Tree Kernels of leveraging structuralinformation to detect arguments (which can be considered the first step in theabove-mentioned Argument Mining pipeline) (Lippi & Torroni, 2015).This work is the continuation of a previous work (Liga, 2019) which aimedto classify argumentative support. A similar work (Liga & Palmirani, 2019)aimed to to classify argumentative opposition. Both these studies show thatcombining Tree Kernels and TFIDF vectorization can be a good strategy forthis kind of classification. Particularly, the present approach tries to discrim-inate between two different kinds of evidence (or premise), comparing twodifferent Tree Kernel functions.3 MethodologyFollowing the method in Liga, 2019, two important Argument Mining datasetshave been considered: the first (DS1) is taken from Al Khatib et al. (Al Khatibet al. , 2016) the second (DS2) from Aharoni et al. (Aharoni et al. , 2014).These two datasets have been built for different tasks but they share a verysimilar labelling system, which is the reason why the can be used jointly. Moreprecisely, DS1 and DS2 classify argumentative texts depending on three com-mon labels (i.e. Study/Statistics, Expert/Testimony, Anecdote). In particular,only the first two labels have been considered, with the aim of classifying evi-dences from study and from expert.Two groups of classifiers were created using KeLP (Filice et al. , 2015), thefirst group was trained on DS1 while the second on DS2. For each classifier,a combination of a Linear Kernel and a Tree Kernel was employed, using a293GROUP 1 Performance on DS1TFIDF + PTK TFIDF + SPTKP R F1 P R F1Study 0.90 0.87 0.88 0.91 0.89 0.90Expert 0.89 0.91 0.90 0.90 0.92 0.91Average F1 (macro) 0.89 0.91Performance on DS2Study 0.74 0.68 0.71 0.78 0.66 0.72Expert 0.76 0.80 0.78 0.75 0.85 0.80Average F1 (macro) 0.75 0.76GROUP 2 Performance on DS2TFIDF + PTK TFIDF + SPTKP R F1 P R F1Study 0.69 0.69 0.69 0.69 0.69 0.69Expert 0.74 0.74 0.74 0.74 0.74 0.74Average F1 (macro) 0.72 0.72Performance on DS1Study 0.83 0.80 0.82 0.82 0.80 0.81Expert 0.86 0.87 0.86 0.85 0.87 0.86Average F1 (macro) 0.84 0.84Table 1. Results of the two groups of classifiers (P=precision, R=recall, F1=F1 score)TFIDF vectorization and a GRCT (Grammatical Relation Centered Tree) rep-resentation. For the choice of the Tree Kernel function, two strategies havebeen attempted: the first deploys a Partial Tree Kernel (PTK, Moschitti, 2006),while the second deploys a Smoothed Partial Tree Kernel (SPTK, Croce et al., 2011). To be sure that the classifiers were able to generalize, they were testedboth on DS1 and on DS2 to detect whether sentences where an evidence from“Study/Statistics” or from “Testimony/Expert”.4 ResultsAs can be seen from Table 1, SPTKs outperform PTKs in group 1, while theirperformances in group 2 are mostly equal. Importantly, both Partial Tree Ker-nels and Smoothed Partial Tree Kernels keep a high degree of generalization,which is one of the main reasons why this methodology can be valuable formany classification problems in the Argumentation Mining pipeline.294ReferencesAHARONI, E., POLNAROV, A., LAVEE, T., HERSHCOVICH, D., LEVY, R.,RINOTT, R., GUTFREUND, D., & SLONIM, N. 2014. A benchmarkdataset for automatic detection of claims and evidence in the context ofcontroversial topics. Pages 64–68 of: Proceedings of the First Workshopon Argumentation Mining.AL KHATIB, K., WACHSMUTH, H., KIESEL, J., HAGEN, M., & STEIN,B. 2016. A news editorial corpus for mining argumentation strategies.Pages 3433–3443 of: Proceedings of COLING 2016, the 26th Interna-tional Conference on Computational Linguistics: Technical Papers.CABRIO, E., & VILLATA, S. 2018. Five Years of Argument Mining: a Data-driven Analysis. Pages 5427–5433 of: IJCAI.CROCE, D., MOSCHITTI, A., & BASILI, R. 2011. Semantic convolution ker-nels over dependency trees: smoothed partial tree kernel. Pages 2013–2016 of: Proceedings of the 20th ACM international conference on Infor-mation and knowledge management. ACM.FILICE, S., CASTELLUCCI, G., CROCE, D., & BASILI, R. 2015. Kelp: akernel-based learning platform for natural language processing. Proceed-ings of ACL-IJCNLP 2015 System Demonstrations, 19–24.LIGA, D. 2019. Argumentative Evidences Classification and ArgumentScheme Detection Using Tree Kernels. Pages 92–97 of: Proceedingsof the 6th Workshop on Argument Mining.LIGA, D., & PALMIRANI, M. 2019. Detecting “Slippery Slope” and otherargumentative stances of opposition using Tree Kernels in monologic dis-course. In: Rules and Reasoning. Third International Joint Conference,RuleML+RR 2019.LIPPI, M., & TORRONI, P. 2015. Argument mining: A machine learningperspective. Pages 163–176 of: International Workshop on Theory andApplications of Formal Argumentation. Springer.MOSCHITTI, A. 2006. Efficient convolution kernels for dependency and con-stituent syntactic trees. Pages 318–329 of: European Conference on Ma-chine Learning. Springer.ROONEY, N., WANG, H., & BROWNE, F. 2012. Applying kernel methodsto argumentation mining. In: Twenty-Fifth International FLAIRS Confer-ence.WALTON, D., REED, C., & MACAGNO, F. 2008. Argumentation schemes.Cambridge University Press.295

Comparing Tree Kernels performances in argumentative evidence classification

https://orbilu.uni.lu/bitstream/10993/52144/1/Comparing%20Tree%20Kernels%20performances%20in%20argumentative%20evidence%20classification.pdf

Comparing Tree Kernels performances in argumentative evidence classification

Abstract

Similar works

Full text

Available Versions

Open Repository and Bibliography - Luxembourg