EFFECTS OF ELABORATIVE INTERROGATION AFTER READING BELIEF-INCONSISTENT ARGUMENTS AND NEED FOR COGNITION ON ARGUMENTATION AND TOPIC BELIEFS

Constructing quality argumentation to justify one’s own beliefs on a topic is important both for a thorough topic understanding and the development of argumentation writing skills. Also, one’s change or retention of topic beliefs should be based on quality argumentation, such that the belief can be considered rational. The purpose of this study was to test whether a cognitive strategy, elaborative interrogation, can improve the understanding of belief-inconsistent arguments on a controversial topic and then improve argumentation quality, as well as result in reflective belief change. Elaborative interrogation is a cognitive strategy which prompts individuals to answer “why” questions on the to-be-learned information. The present study also examined the role of individuals’ need for cognition in argumentation and its role in the relationship between using elaborative interrogation and quality of argumentation. This study used a mixed model pretest-posttest experimental design with random assignment to three experimental conditions (elaborative interrogation treatment condition, summary control condition, and no-processing control condition) to test three hypotheses on effects of elaborative interrogation and need for cognition. It was hypothesized individuals who used elaborative interrogation strategy when reading belief-inconsistent arguments would demonstrate improvement in quality of argumentation (Hypothesis 1) and reflective belief change (Hypothesis 2) after reading, whereas individuals who did not use this strategy would not. Argumentation quality and topic beliefs were measured before and after the experimental manipulation to examine pre-post changes, if any. It was also hypothesized high need for cognition would be associated with high quality of argumentation (Hypothesis 3). Based on the experimental results, Hypotheses 1 and 2 were confirmed. Hypothesis 3 was rejected. In the end, implications of the findings about each hypothesis are discussed, along with possible cognitive mechanisms underlying these findings. Contributions of this study also are summarized, highlighting the connection between the psychology literature on cognitive biases and the education literature on learning strategies. Finally, limitations of the study are discussed, followed by suggestions for future research. Advisor: Roger Brunin

Argumentation Mining in User-Generated Web Discourse

The goal of argumentation mining, an evolving research field in computational linguistics, is to design methods capable of analyzing people's argumentation. In this article, we go beyond the state of the art in several ways. (i) We deal with actual Web data and take up the challenges given by the variety of registers, multiple domains, and unrestricted noisy user-generated Web discourse. (ii) We bridge the gap between normative argumentation theories and argumentation phenomena encountered in actual data by adapting an argumentation model tested in an extensive annotation study. (iii) We create a new gold standard corpus (90k tokens in 340 documents) and experiment with several machine learning methods to identify argument components. We offer the data, source codes, and annotation guidelines to the community under free licenses. Our findings show that argumentation mining in user-generated Web discourse is a feasible but challenging task.Comment: Cite as: Habernal, I. & Gurevych, I. (2017). Argumentation Mining in User-Generated Web Discourse. Computational Linguistics 43(1), pp. 125-17

Volume 58, Issue 1, Summer 2022 Speaker & Gavel

Complete digitized issue (Volume 58, issue 1, Summer 2022) of Speaker & Gavel

Learning to Engage With Wicked Problems

Complex, societal problems can be overwhelming. Maybe better avoid them. This contribution shows how a cloud-based learning technology—the Reflect! platform—can be used to practice a particular strategy for dealing with so-called wicked problems. By providing a learning experience that is close to collaborative problem-solving in real life, students can gain the self-confidence needed to engage constructively with wicked problems. The approach presented is an example of how philosophy can contribute to general education. After discussing the notion of wicked problems and what is required to cope with them, this article provides information that should be useful for readers who want to include a focus on wicked problems in their teaching: first, a discussion of how the work of learners can be assessed—with examples that demonstrate what is expected—and, second, the results of a survey-based assessment of the Reflect! learning experience from learners’ points of view

Persuasive Gaming in Context

The rapid developments in new communication technologies have facilitated the popularization of digital games, which has translated into an exponential growth of the game industry in recent decades. The ubiquitous presence of digital games has resulted in an expansion of the applications of these games from mere entertainment purposes to a great variety of serious purposes. In this edited volume, we narrow the scope of attention by focusing on what game theorist Ian Bogost has called 'persuasive games', that is, gaming practices that combine the dissemination of information with attempts to engage players in particular attitudes and behaviors.This volume offers a multifaceted reflection on persuasive gaming, that is, on the process of these particular games being played by players. The purpose is to better understand when and how digital games can be used for persuasion by further exploring persuasive games and some other kinds of persuasive playful interaction as well. The book critically integrates what has been accomplished in separate research traditions to offer a multidisciplinary approach to understanding persuasive gaming that is closely linked to developments in the industry by including the exploration of relevant case studies

The impact of task type on learners\u27 argumentation, participation and collaboration in online discussions.

Task type is a fundamental element that directs learners’ interactions in collaboration. The study grounds its design in McGrath’s (1984) Group Task Circumplex and examines students’ online behaviors, processes of argumentation, and collaboration. Students were asked to solve an authentic organizational challenge in a five-day online discussion in a blended (face-to-face and online) undergraduate business course. Two kinds of tasks were given: a task with an open-ended question, and a task with two contrasting alternatives. Twenty-three groups (107 students) agreed to participate; the content of their posts and participation (click-stream) data were collected. The results show that the groups given an open-ended question participated more actively in reviewing and reading activities; they also challenged others more often and provided more supporting reasons and evidence, but there seemed to be an unequal distribution of efforts among group members in the time they spent reviewing and the length of posts they made

Defining and Assessing Critical Thinking: toward an automatic analysis of HiEd students’ written texts

L'obiettivo principale di questa tesi di dottorato è testare, attraverso due studi empirici, l'affidabilità di un metodo volto a valutare automaticamente le manifestazioni del Pensiero Critico (CT) nei testi scritti da studenti universitari. Gli studi empirici si sono basati su una review critica della letteratura volta a proporre una nuova classificazione per sistematizzare le diverse definizioni di CT e i relativi approcci teorici. La review esamina anche la relazione tra le diverse definizioni di CT e i relativi metodi di valutazione. Dai risultati emerge la necessità di concentrarsi su misure aperte per la valutazione del CT e di sviluppare strumenti automatici basati su tecniche di elaborazione del linguaggio naturale (NLP) per superare i limiti attuali delle misure aperte, come l’attendibilità e i costi di scoring. Sulla base di una rubrica sviluppata e implementata dal gruppo di ricerca del Centro di Didattica Museale – Università di Roma Tre (CDM) per la valutazione e l'analisi dei livelli di CT all'interno di risposte aperte (Poce, 2017), è stato progettato un prototipo per la misurazione automatica di alcuni indicatori di CT. Il primo studio empirico condotto su un gruppo di 66 docenti universitari mostra livelli di affidabilità soddisfacenti della rubrica di valutazione, mentre la valutazione effettuata dal prototipo non era sufficientemente attendibile. I risultati di questa sperimentazione sono stati utilizzati per capire come e in quali condizioni il modello funziona meglio. La seconda indagine empirica era volta a capire quali indicatori del linguaggio naturale sono maggiormente associati a sei sottodimensioni del CT, valutate da esperti in saggi scritti in lingua italiana. Lo studio ha utilizzato un corpus di 103 saggi pre-post di studenti universitari di laurea magistrale che hanno frequentato il corso di "Pedagogia sperimentale e valutazione scolastica". All'interno del corso, sono state proposte due attività per stimolare il CT degli studenti: la valutazione delle risorse educative aperte (OER) (obbligatoria e online) e la progettazione delle OER (facoltativa e in modalità blended). I saggi sono stati valutati sia da valutatori esperti, considerando sei sotto-dimensioni del CT, sia da un algoritmo che misura automaticamente diversi tipi di indicatori del linguaggio naturale. Abbiamo riscontrato un'affidabilità interna positiva e un accordo tra valutatori medio-alto. I livelli di CT degli studenti sono migliorati in modo significativo nel post-test. Tre indicatori del linguaggio naturale sono 5 correlati in modo significativo con il punteggio totale di CT: la lunghezza del corpus, la complessità della sintassi e la funzione di peso tf-idf (term frequency–inverse document frequency). I risultati raccolti durante questo dottorato hanno implicazioni sia teoriche che pratiche per la ricerca e la valutazione del CT. Da un punto di vista teorico, questa tesi mostra sovrapposizioni inesplorate tra diverse tradizioni, prospettive e metodi di studio del CT. Questi punti di contatto potrebbero costituire la base per un approccio interdisciplinare e la costruzione di una comprensione condivisa di CT. I metodi di valutazione automatica possono supportare l’uso di misure aperte per la valutazione del CT, specialmente nell'insegnamento online. Possono infatti facilitare i docenti e i ricercatori nell'affrontare la crescente presenza di dati linguistici prodotti all'interno di piattaforme educative (es. Learning Management Systems). A tal fine, è fondamentale sviluppare metodi automatici per la valutazione di grandi quantità di dati che sarebbe impossibile analizzare manualmente, fornendo agli insegnanti e ai valutatori un supporto per il monitoraggio e la valutazione delle competenze dimostrate online dagli studenti.The main goal of this PhD thesis is to test, through two empirical studies, the reliability of a method aimed at automatically assessing Critical Thinking (CT) manifestations in Higher Education students’ written texts. The empirical studies were based on a critical review aimed at proposing a new classification for systematising different CT definitions and their related theoretical approaches. The review also investigates the relationship between the different adopted CT definitions and CT assessment methods. The review highlights the need to focus on open-ended measures for CT assessment and to develop automatic tools based on Natural Language Processing (NLP) technique to overcome current limitations of open-ended measures, such as reliability and costs. Based on a rubric developed and implemented by the Center for Museum Studies – Roma Tre University (CDM) research group for the evaluation and analysis of CT levels within open-ended answers (Poce, 2017), a NLP prototype for the automatic measurement of CT indicators was designed. The first empirical study was carried out on a group of 66 university teachers. The study showed satisfactory reliability levels of the CT evaluation rubric, while the evaluation carried out by the prototype was not yet sufficiently reliable. The results were used to understand how and under what conditions the model works better. The second empirical investigation was aimed at understanding which NLP features are more associated with six CT sub-dimensions as assessed by human raters in essays written in the Italian language. The study used a corpus of 103 students’ pre-post essays who attended a Master's Degree module in “Experimental Education and School Assessment” to assess students' CT levels. Within the module, we proposed two activities to stimulate students' CT: Open Educational Resources (OERs) assessment (mandatory and online) and OERs design (optional and blended). The essays were assessed both by expert evaluators, considering six CT sub-dimensions, and by an algorithm that automatically calculates different kinds of NLP features. The study shows a positive internal reliability and a medium to high inter-coder agreement in expert evaluation. Students' CT levels improved significantly in the post-test. Three NLP indicators significantly correlate with CT total score: the Corpus Length, the Syntax Complexity, and an adapted measure of Term Frequency- Inverse Document Frequency. The results collected during this PhD have both theoretical and practical implications for CT research and assessment. From a theoretical perspective, this thesis shows unexplored similarities among different CT traditions, perspectives, and study methods. These similarities could be exploited to open up an interdisciplinary dialogue among experts and build up a shared understanding of CT. Automatic assessment methods can enhance the use of open-ended measures for CT assessment, especially in online teaching. Indeed, they can support teachers and researchers to deal with the growing presence of linguistic data produced within educational 4 platforms. To this end, it is pivotal to develop automatic methods for the evaluation of large amounts of data which would be impossible to analyse manually, providing teachers an