Argument education in higher education: A validation study

Argument education can play an important role in higher education for leadership development and responding to increasing calls for post-secondary accountability. But to do so, argumentation teachers, scholars, and practitioners need to develop a clearer definition and research agenda for the purposes of teaching and assessing argumentation. The research conducted here contributes to this project by first establishing a definitional construct and observable behaviors associated with learning and practicing argumentation. Second, an argument education assessment instrument was created based off of the literature-supported definition of argumentation. Third, debate and argument education subject matter experts reviewed the definition, behaviors, and assessment instrument. Fourth, the newly developed instrument was administered to undergraduate college students over the course of three studies (n=949) to collect evidence testing whether the instrument may be used in a reliable and valid way to assess the learning of argumentation. Finally, the author concluded that the data suggests that the instrument may be used for assessing argument education, but further research is needed to improve the evidence for reliability and validity of the instrument’s use. Furthermore, the data collected from assessing argument education provides important implications for how argumentation is defined and assessed within an educational context and what role argument education may play in leadership development

Walton's types of argumentation dialogues as classroom discourse sequences

UID/FIL/00183/2013 DL 57/2016/CP1453/CT0066Dialogic argumentation has thus far been proposed as a way to analyse, understand, and promote meaningful classroom interactions. However, currently there is a lack of systematic proposals for conceptualising argumentation dialogue goals as part of teachers' pedagogical repertoire. Our main goal is to operationalise an existing framework of argumentation dialogue types, the one proposed by argumentation theorist Douglas Walton. To do so, we first identify a set of epistemic criteria for meaningful, from an argumentation point of view, discursive interactions, which we use as ‘framing indicators’ to enrich Walton's existing typology of four argumentation dialogues (information-seeking, inquiry, discovery, persuasion). We applied the resulting pragmatic framework to teacher-student interactions found in 20 transcripts of both science and social sciences secondary education lessons. We found that affordances for these four types of dialogues were also present in teacher-student discourse, where the implied argumentation goal was not fulfilled. We discuss these findings in terms of the need to be able to identify the dialogic potentiality and accountability within teacher-student interactions so that the argumentative potential of these interactions can be fulfilled, resulting in productive classroom discourse within secondary education classroom settings.publishersversionepub_ahead_of_prin

Learning analytics and psychophysiology : understanding the learning process in a STEM game

This study focuses on the exploration of player experience in educational games and its potential impact on predicting learning outcomes. Specifically, the research aims to investigate the connection psychophysiology data, obtained through a summative study involving nine participants, and the results of a learning analytics model derived from a larger field test. The study incorporates eye tracking and electrodermal activity data to gain insights into the predictive power of this data. Through the analysis of player experience data, the study sheds light on the factors that contribute to effective educational game design. By examining the eye tracking and EDA data, the researchers explored the participants' engagement levels, attention patterns, and emotional arousal during gameplay. These findings revealed a connection between spikes of visual attention and EDA during interactions with character faces as well as in game cinematics. In conclusion, the outcomes of this study provide valuable insights for future educational game designers. By understanding the relationship between user experience indicators and learning analytics, designers can tailor game elements to enhance engagement, attention, and emotional arousal, ultimately leading to improved learning outcomes. The integration of eye tracking and EDA data in user experience studies adds a new dimension to the evaluation and design of educational games. The findings pave the way for future research in the field and highlight the importance of considering user experience as a crucial factor in educational game design and development.Includes bibliographical references

Argumentation Mining in User-Generated Web Discourse

The goal of argumentation mining, an evolving research field in computational linguistics, is to design methods capable of analyzing people's argumentation. In this article, we go beyond the state of the art in several ways. (i) We deal with actual Web data and take up the challenges given by the variety of registers, multiple domains, and unrestricted noisy user-generated Web discourse. (ii) We bridge the gap between normative argumentation theories and argumentation phenomena encountered in actual data by adapting an argumentation model tested in an extensive annotation study. (iii) We create a new gold standard corpus (90k tokens in 340 documents) and experiment with several machine learning methods to identify argument components. We offer the data, source codes, and annotation guidelines to the community under free licenses. Our findings show that argumentation mining in user-generated Web discourse is a feasible but challenging task.Comment: Cite as: Habernal, I. & Gurevych, I. (2017). Argumentation Mining in User-Generated Web Discourse. Computational Linguistics 43(1), pp. 125-17

Conceptualizing and supporting awareness of collaborative argumentation

In this thesis, we introduce “Argue(a)ware”. This is a concept for an instructional group awareness tool which aims at supporting social interactions in co-located computer-supported collaborative argumentation settings. Argue(a)ware is designed to support the social interactions in the content (i.e., task-related) and in the relational (i.e., social and interpersonal) space of co-located collaborative argumentation (Barron, 2003). The support for social interactions in the content space of collaboration is facilitated with the use of collaborative scripts for argumentation (i.e., instructions and scaffolds of argument construction) as well with the use of an argument mapping tool (i.e., visualization of argumentation outcomes in a form of diagrams) (Stegmann, Weinberger, & Fischer, 2007; van Gelder, 2013). The support for social interactions in the relational space of collaboration is facilitated with the use of different awareness mechanisms from the CSCL and the CSCW research fields (i.e., monitoring, mirroring and awareness notification tools). In this thesis, we examined how different awareness mechanisms facilitate the regulation of collaborative processes in the relational space of collaborative argumentation. Moreover, we studied how they affect the perceived team effectiveness (i.e., process outcome) and group performance (i.e., learning outcome) in the content space of collaboration. Thereby, we studied also the effects of the design of the awareness mechanisms on the application of the mechanisms and the user experience with them. In line with the design-based research paradigm, we attempted to simultaneously improve and study the effect of Argue(a)ware on collaborative argumentation (Herrington, McKenney, Reeves & Oliver, 2007). Through a series of design-based research studies we tested and refined the prototypes of the instructional group awareness tool. Moreover, we studied the ecological validity of dominant awareness and instructional theories in the context of co-located computer-supported collaborative argumentation. The underlying premise of the Argue(a)ware tool is that a combination of awareness and instructional support will result in increased awareness of collaboration, which will, in turn, mediate the regulation of collaborative processes. Moreover, we assume that successful regulation of collaboration will result in high perceived team effectiveness and the group performance in turn. In the first phase of development of the Argue(a)ware tool, we built support of the content space of collaborative argumentation with argument scaffold elements in a pedagogical face-to-face macro-script and an argument mapping tool. Furthermore, we extended the use of the script for supporting the relational space of collaboration by embedding awareness prompts for reflecting on collaboration during regular breaks in the script. Following, we designed two variations of the same pedagogical face-to-face macro-script which differ with respect to the type of group awareness prompts they used for supporting the relational space of collaboration i.e. behavioral and social. Upon designing the two script variations, we conducted a longitudinal, multiple-case study with ten groups of Media Informatics master students (n = 28, in groups of three or two, group=case, 4 sessions x70 min, Behavioural Awareness Script group= 5, Social Awareness Script group =5.) where each group was conceptualized as a case. Students collaborated every time for arguing to solve one different ill-structured problem and for transferring their arguments in the argument mapping tool Rationale. Thereby, we intended to investigate the effects of different awareness prompts on (a) collaborative metacognitive processes i.e., regulation, reflection, and evaluation (b) the relation between collaborative metacognitive processes and the quality of collaborative argumentation as well as (c) the impact of the two script variations on perceived team effectiveness and (d) what was experience with the different parts of the script variations in the two groups and how this fits into the design framework by Buder (2011). The quantitative analysis of argument outcomes from the groups yield no significant difference between the groups that worked with the BAS and the SAS variations. No significant difference between the script variations with respect to the results from the team effectiveness questionnaires was found either. Prompts for regulating collaboration processes were found to be the most successfully and consistently applied ones, especially in the most successful cases from both script variations and have influenced the argumentation outcomes. The awareness prompts afforded an explicit feedback display format (e.g. assessment of participation levels of self- and others) through discussion (Buder, 2011). The prompted explicit feedback display format (i.e., ratings of one’s self and of others) was criticized for running only on subjective awareness information on participation, contribution efforts and performance in the role. This resulted in evaluation apprehension phenomena (Cottrell, 1972) and evaluation bias (i.e., users may have not assessed themselves or others frankly) (Ghadirian et al., 2016). The awareness prompts for reflection and evaluation did reveal frictions in the plan making process (i.e., dropping out of the plan for collaboration) in the least successful groups. Problems with group dynamics (i.e., free-loading and presence of dominance) but were not powerful enough to trigger the desired changes in the behaviors of the students. The prompts for evaluating the collaboration in both script variations had no apparent connection to argumentation outcomes. The results indicated that dominant presence phenomena inhibited substantive argumentation in the least successful groups. They also indicated that the role-assignment influenced the group dynamics by helping student’s making clear the labor division in the group. In the second phase of development of the Argue(a)ware tool, the focus is on structuring and regulating social interactions in the relational space of collaborative argumentation by means of scripted roles and role-based awareness scaffolds. We designed support for mirroring participation in the role (i.e., a role-based awareness visualization) and support for monitoring participation, coordination and collaboration efforts in the role (i.e., self-assessment questionnaire). Moreover, we designed additional support for guiding participation in the role i.e., role-based reminders as notifications on smartwatches. In a between-subjects study, ten groups of three university students each (n = 30, Mage =22y, mixed educational backgrounds, 1x90min) worked with two variants of the Argue(a)ware for arguing to solve one ill-structured problem and transferring their arguments in the argument mapping tool Rationale. Next, to that, students should monitor their progress in their role with the role-based awareness visualization and the self-assessment questionnaire with the basic awareness support (role-based awareness visualization with the intermediate self-assessment) and the enhanced awareness support (additional role-based awareness reminders). Half of the groups worked only with the role-based awareness visualization and the self-assessment questionnaire (Basic Awareness Condition-BAC) while the other half groups received additional text-based awareness notifications via smartwatches that were sent to students privately (Enhanced Awareness Condition- EAC). Thereby, we tested the use of different degrees of awareness support in the two conditions with respect to their impact on a) self-perceived awareness of performance in the role and of collaboration and coordination efforts (measured with the same questionnaire at two time points), b) on perceive team effectiveness, c) group performance. We hypothesized that students in EAC will perform better thanks to the additional awareness reminders that increased the directivity and influenced their awareness in the role. The mixed methods analysis revealed that the awareness reminders, when perceived on time, succeeded in guiding collaboration (i.e., resulted in more role-specific behaviors). Students in the EAC condition improved their awareness over time (between the two measurements). These results indicated that enhanced awareness support in the form of additional guidance through awareness reminders can boost the awareness of students’ performance in the role as well as the awareness of their coordination and collaboration efforts over time by directing them back to the mirroring and monitoring tools. Moreover, students in EAC exhibited higher perceived team effectiveness than the students in BAC. However, no significant differences in building of shared mental models or performing in mutual performance monitoring were found between the groups. However, students in BAC and EAC did not differ significantly with respect to the formal correctness or evidence sufficiency of their group argumentation outcomes. Moreover, technical difficulties with the smartphones used as delivery devices for the awareness reminders (i.e., low vibration modus) hindered the timely perception of the reminders and thus their effect on participation. Finally, the questionnaire on the experience with the different parts of Argue(a)ware system indicated the need for exploring further media for supporting the awareness reminders to avoid the overwhelming effects of the multiple displays of the system and enhancing higher perceptiveness of the reminders with low interruption costs for other group members. The rather high satisfaction with the use of the role-based awareness visualization and the positive comments on the motivating aspects of monitoring how the personal success contributes to the group performance indicate that the group mirror succeeded in making group norms visible to group members in a non-obtrusive way. The high interpersonal comparability of performances without moderating the group ‘s interaction directly in the basic awareness condition was proven to be the favored design approach compared to the combination of group mirror and awareness reminders in the enhance awareness condition. In the third phase of development of Argue(a)ware, we focused on designing and testing different notification modes on different ubiquitous mobile devices for facilitating the next prototype of a notification system for role-based awareness reminders. Thereby, the aim of the system was again to guide students’ active participation in collaborative argumentation. More specifically, we focused on raising students’ attention to the reminders and triggering a prompter reaction to the contents of the reminders whilst avoiding a high interruption cost for the primary task (i.e., arguing for solving the problem at hand) in the group. These goals were translated into design challenges for the design of the role-based awareness notification system. The system should afford low interruptions, high reaction and high comprehension of notifications. Notification systems with this particular configuration of IRC values are known as "secondary display" systems (McCrickard et al., 2003). Next, we designed three low-fidelity prototypes for a role-based notification system for delivering awareness reminders: The first ran on a smartwatch and afforded text-based information with vibration and light notification modalities. The second ran on smartphone and afforded text-based information with vibrotactile and light-based notification modalities. Finally, the third prototype run on a smart-ring which afforded graphical- based (i.e. abstract light) information with and light and vibration notification modalities. To test the suitability of these prototypes for acting as “secondary display” systems, we conducted a within-subjects user study where three university students (n= 3, Mage=28, mixed educational background) argued for solving three different problem cases and producing an argument map in each of the three consecutive meetings (max 90min) in the Argue(a)ware instructional system. Students were assigned the roles of writer, corrector and devil`s advocate and were instructed to maintain the same role across the three meetings. In each meeting, students worked with a different role-based awareness notification prototype, where they received a notification indicating their balloon is not growing bigger after five minutes of not exhibiting any role-specific behaviors. The role-based awareness notification prototypes aimed at introducing timely interventions which would prompt students to check on their own progress in the role and the group progress as visualized by the role-based awareness visualization on the large display. Ultimately, this should prompt them to reflect on the awareness information from the visualization and adapt their behaviors to the desired behavior standards over time. Results showed that students perceived the notifications from all media mostly based on vibration cues. Thereby, the vibration cues on the wrist (smartwatch) were considered the least disruptive to the main task compared to the vibration cues on finger (smartwatch) and the vibration cues on the desk (smartphone). Students also declared that vibration cues on wrist prompted the fastest reaction i.e., attending to notification by interacting with the smartwatch. These results indicate that vibration cues on the wrist can be a suitable notification mechanism for increasing the perceived urgency of the message and prompting the reaction on it without causing great distraction to the main task, as studies previous studies showed before (Pielot, Church, & deOliveira, 2013; Hernández-Leo, Balestrini, Nieves & Blat, 2012). Based on very limited qualitative data on light as notification modality and awareness representation type no inferences could be made about its influence on the cost of interruption, reaction and comprehension parameters comprehensiveness. The qualitative and quantitative data on the experience with different media as awareness notification systems indicate that smartwatches may be the most suitable medium for acting as awareness notification medium with a “secondary display” IRC configuration (low-high-high). However, this inference needs to be tested in terms of a follow up study. In the next study, the great limitations of study (limited data due to low power and mal-structured measurement instruments) need to be repaired. Finally, the focus should be on comparing notification modalities of one medium (e.g., smartphone) based on a larger set of participants and with the use of objective measurements for the IRC parameter values (Chewar, McCrickard & Sutcliffe, 2004). Finally, we draw conclusions based on the findings from the three studies with respect to the role of awareness mechanisms for facilitating collaborative processes and outcomes and provide replicable and generalizable design principles. These principles are formed as heuristic statements and are subject to refinement by further research (Bell, Hoadley, & Linn, 2004; Van den Akker, 1999). We conclude with the limitations of the study and ideas for future work with Argue(a)ware

The Dramatic Arc in the Development of Argumentation Skills of Upper Secondary School Students in Geography Education

Geography education can facilitate learners' critical thinking and argumentation skills to make well-reasoned decisions on social and environmental issues. This study reports on a geography course consisting of 18 lessons, each of them 75 min, designed to afford intensive practice in argumentation to upper secondary school students (n = 21) and following the dramatic arc. The study produces examples of different developmental pathways of upper secondary school students' argumentation during the geography course. In this qualitative case study, the data were collected from learning diaries and analyzed using content analysis following ARRA-analysis (Analysis of Reasoning, Rhetorics and Argumentation), which is based on Toulmin's argumentation model. The results indicated that most of the students developed justified arguments and composed clear claims and relevant rhetorical modes such as qualifications, rhetorical questions and rebuttals. Justification categories that were mainly used were backings, grounds and warrants. However, some students had difficulties in recognizing the main claim and arguments. The students developed their argumentation skills following the dramatic arc. They possessed the prerequisites for argumentative reasoning and writing but needed further practice in analytical and critical writing.Peer reviewe

The Dramatic Arc in the Development of Argumentation Skills of Upper Secondary School Students in Geography Education

Geography education can facilitate learners' critical thinking and argumentation skills to make well-reasoned decisions on social and environmental issues. This study reports on a geography course consisting of 18 lessons, each of them 75 min, designed to afford intensive practice in argumentation to upper secondary school students (n = 21) and following the dramatic arc. The study produces examples of different developmental pathways of upper secondary school students' argumentation during the geography course. In this qualitative case study, the data were collected from learning diaries and analyzed using content analysis following ARRA-analysis (Analysis of Reasoning, Rhetorics and Argumentation), which is based on Toulmin's argumentation model. The results indicated that most of the students developed justified arguments and composed clear claims and relevant rhetorical modes such as qualifications, rhetorical questions and rebuttals. Justification categories that were mainly used were backings, grounds and warrants. However, some students had difficulties in recognizing the main claim and arguments. The students developed their argumentation skills following the dramatic arc. They possessed the prerequisites for argumentative reasoning and writing but needed further practice in analytical and critical writing.Peer reviewe

Arguement in the humanities: A knowledge based approach

In this thesis I have a threefold purpose. I will attempt: (a) to present a generic design for a tool - the Argument Support Program - which can be of use in supporting the reasoning of archaeologists (and others especially, but not exclusively, in the humanities); (b) I will present a model of argumentation and debate as the theoretical orientation within which the model is developed; and, (c) I will suggest that this approach is a natural development of several strands of research within the artificial intelligence community. A tripartite model of argument is presented in terms of arguers, the argument structure produced and the argument domain or field. This model subsumes reasoning, interpretation and argument exchange or debate. It is maintained, further, that while this model is generally applicable, specific domains have particular styles of argument. The notion of argument style is discussed in terms of the types of reasoning used. The related concept of relevance in argument is discussed in terms of the specific tokens of these types which may be used in a particular argument. It is argued that archaeology is characterized, at least in part, by the use of argument by analogy and argument from theoretical principles or models. A design for a generic program - the Argument Support Program (ASP) - based on the theoretical principles is delineated. Details of the partial implementation of the model as a constrained debater in the domain of archaeology (ASP for archaeology or ASParch) are presented. Example runs which illustrate how the characterizing features of archaeology are dealt with are also presented as are examples of the various domain and system knowledge bases needed. The application of ASPs to other domains and areas such as literary criticism, legal reasoning and Darwinian theory is discussed. In the final chapter, the achievements and inadequacies of this research are summarized, possible reasons are presented for the inadequacies in the resulting system and future directions discussed