Scaffolding School Pupils’ Scientific Argumentation with Evidence-Based Dialogue Maps

This chapter reports pilot work investigating the potential of Evidence-based Dialogue Mapping to scaffold young teenagers’ scientific argumentation. Our research objective is to better understand pupils’ usage of dialogue maps created in Compendium to write scientific ex-planations. The participants were 20 pupils, 12-13 years old, in a summer science course for “gifted and talented” children in the UK. Through qualitative analysis of three case studies, we investigate the value of dialogue mapping as a mediating tool in the scientific reasoning process during a set of learning activities. These activities were published in an online learning envi-ronment to foster collaborative learning. Pupils mapped their discussions in pairs, shared maps via the online forum and in plenary discussions, and wrote essays based on their dialogue maps. This study draws on these multiple data sources: pupils’ maps in Compendium, writings in science and reflective comments about the uses of mapping for writing. Our analysis highlights the diversity of ways, both successful and unsuccessful, in which dialogue mapping was used by these young teenagers

Conceptualizing and supporting awareness of collaborative argumentation

In this thesis, we introduce “Argue(a)ware”. This is a concept for an instructional group awareness tool which aims at supporting social interactions in co-located computer-supported collaborative argumentation settings. Argue(a)ware is designed to support the social interactions in the content (i.e., task-related) and in the relational (i.e., social and interpersonal) space of co-located collaborative argumentation (Barron, 2003). The support for social interactions in the content space of collaboration is facilitated with the use of collaborative scripts for argumentation (i.e., instructions and scaffolds of argument construction) as well with the use of an argument mapping tool (i.e., visualization of argumentation outcomes in a form of diagrams) (Stegmann, Weinberger, & Fischer, 2007; van Gelder, 2013). The support for social interactions in the relational space of collaboration is facilitated with the use of different awareness mechanisms from the CSCL and the CSCW research fields (i.e., monitoring, mirroring and awareness notification tools). In this thesis, we examined how different awareness mechanisms facilitate the regulation of collaborative processes in the relational space of collaborative argumentation. Moreover, we studied how they affect the perceived team effectiveness (i.e., process outcome) and group performance (i.e., learning outcome) in the content space of collaboration. Thereby, we studied also the effects of the design of the awareness mechanisms on the application of the mechanisms and the user experience with them. In line with the design-based research paradigm, we attempted to simultaneously improve and study the effect of Argue(a)ware on collaborative argumentation (Herrington, McKenney, Reeves & Oliver, 2007). Through a series of design-based research studies we tested and refined the prototypes of the instructional group awareness tool. Moreover, we studied the ecological validity of dominant awareness and instructional theories in the context of co-located computer-supported collaborative argumentation. The underlying premise of the Argue(a)ware tool is that a combination of awareness and instructional support will result in increased awareness of collaboration, which will, in turn, mediate the regulation of collaborative processes. Moreover, we assume that successful regulation of collaboration will result in high perceived team effectiveness and the group performance in turn. In the first phase of development of the Argue(a)ware tool, we built support of the content space of collaborative argumentation with argument scaffold elements in a pedagogical face-to-face macro-script and an argument mapping tool. Furthermore, we extended the use of the script for supporting the relational space of collaboration by embedding awareness prompts for reflecting on collaboration during regular breaks in the script. Following, we designed two variations of the same pedagogical face-to-face macro-script which differ with respect to the type of group awareness prompts they used for supporting the relational space of collaboration i.e. behavioral and social. Upon designing the two script variations, we conducted a longitudinal, multiple-case study with ten groups of Media Informatics master students (n = 28, in groups of three or two, group=case, 4 sessions x70 min, Behavioural Awareness Script group= 5, Social Awareness Script group =5.) where each group was conceptualized as a case. Students collaborated every time for arguing to solve one different ill-structured problem and for transferring their arguments in the argument mapping tool Rationale. Thereby, we intended to investigate the effects of different awareness prompts on (a) collaborative metacognitive processes i.e., regulation, reflection, and evaluation (b) the relation between collaborative metacognitive processes and the quality of collaborative argumentation as well as (c) the impact of the two script variations on perceived team effectiveness and (d) what was experience with the different parts of the script variations in the two groups and how this fits into the design framework by Buder (2011). The quantitative analysis of argument outcomes from the groups yield no significant difference between the groups that worked with the BAS and the SAS variations. No significant difference between the script variations with respect to the results from the team effectiveness questionnaires was found either. Prompts for regulating collaboration processes were found to be the most successfully and consistently applied ones, especially in the most successful cases from both script variations and have influenced the argumentation outcomes. The awareness prompts afforded an explicit feedback display format (e.g. assessment of participation levels of self- and others) through discussion (Buder, 2011). The prompted explicit feedback display format (i.e., ratings of one’s self and of others) was criticized for running only on subjective awareness information on participation, contribution efforts and performance in the role. This resulted in evaluation apprehension phenomena (Cottrell, 1972) and evaluation bias (i.e., users may have not assessed themselves or others frankly) (Ghadirian et al., 2016). The awareness prompts for reflection and evaluation did reveal frictions in the plan making process (i.e., dropping out of the plan for collaboration) in the least successful groups. Problems with group dynamics (i.e., free-loading and presence of dominance) but were not powerful enough to trigger the desired changes in the behaviors of the students. The prompts for evaluating the collaboration in both script variations had no apparent connection to argumentation outcomes. The results indicated that dominant presence phenomena inhibited substantive argumentation in the least successful groups. They also indicated that the role-assignment influenced the group dynamics by helping student’s making clear the labor division in the group. In the second phase of development of the Argue(a)ware tool, the focus is on structuring and regulating social interactions in the relational space of collaborative argumentation by means of scripted roles and role-based awareness scaffolds. We designed support for mirroring participation in the role (i.e., a role-based awareness visualization) and support for monitoring participation, coordination and collaboration efforts in the role (i.e., self-assessment questionnaire). Moreover, we designed additional support for guiding participation in the role i.e., role-based reminders as notifications on smartwatches. In a between-subjects study, ten groups of three university students each (n = 30, Mage =22y, mixed educational backgrounds, 1x90min) worked with two variants of the Argue(a)ware for arguing to solve one ill-structured problem and transferring their arguments in the argument mapping tool Rationale. Next, to that, students should monitor their progress in their role with the role-based awareness visualization and the self-assessment questionnaire with the basic awareness support (role-based awareness visualization with the intermediate self-assessment) and the enhanced awareness support (additional role-based awareness reminders). Half of the groups worked only with the role-based awareness visualization and the self-assessment questionnaire (Basic Awareness Condition-BAC) while the other half groups received additional text-based awareness notifications via smartwatches that were sent to students privately (Enhanced Awareness Condition- EAC). Thereby, we tested the use of different degrees of awareness support in the two conditions with respect to their impact on a) self-perceived awareness of performance in the role and of collaboration and coordination efforts (measured with the same questionnaire at two time points), b) on perceive team effectiveness, c) group performance. We hypothesized that students in EAC will perform better thanks to the additional awareness reminders that increased the directivity and influenced their awareness in the role. The mixed methods analysis revealed that the awareness reminders, when perceived on time, succeeded in guiding collaboration (i.e., resulted in more role-specific behaviors). Students in the EAC condition improved their awareness over time (between the two measurements). These results indicated that enhanced awareness support in the form of additional guidance through awareness reminders can boost the awareness of students’ performance in the role as well as the awareness of their coordination and collaboration efforts over time by directing them back to the mirroring and monitoring tools. Moreover, students in EAC exhibited higher perceived team effectiveness than the students in BAC. However, no significant differences in building of shared mental models or performing in mutual performance monitoring were found between the groups. However, students in BAC and EAC did not differ significantly with respect to the formal correctness or evidence sufficiency of their group argumentation outcomes. Moreover, technical difficulties with the smartphones used as delivery devices for the awareness reminders (i.e., low vibration modus) hindered the timely perception of the reminders and thus their effect on participation. Finally, the questionnaire on the experience with the different parts of Argue(a)ware system indicated the need for exploring further media for supporting the awareness reminders to avoid the overwhelming effects of the multiple displays of the system and enhancing higher perceptiveness of the reminders with low interruption costs for other group members. The rather high satisfaction with the use of the role-based awareness visualization and the positive comments on the motivating aspects of monitoring how the personal success contributes to the group performance indicate that the group mirror succeeded in making group norms visible to group members in a non-obtrusive way. The high interpersonal comparability of performances without moderating the group ‘s interaction directly in the basic awareness condition was proven to be the favored design approach compared to the combination of group mirror and awareness reminders in the enhance awareness condition. In the third phase of development of Argue(a)ware, we focused on designing and testing different notification modes on different ubiquitous mobile devices for facilitating the next prototype of a notification system for role-based awareness reminders. Thereby, the aim of the system was again to guide students’ active participation in collaborative argumentation. More specifically, we focused on raising students’ attention to the reminders and triggering a prompter reaction to the contents of the reminders whilst avoiding a high interruption cost for the primary task (i.e., arguing for solving the problem at hand) in the group. These goals were translated into design challenges for the design of the role-based awareness notification system. The system should afford low interruptions, high reaction and high comprehension of notifications. Notification systems with this particular configuration of IRC values are known as "secondary display" systems (McCrickard et al., 2003). Next, we designed three low-fidelity prototypes for a role-based notification system for delivering awareness reminders: The first ran on a smartwatch and afforded text-based information with vibration and light notification modalities. The second ran on smartphone and afforded text-based information with vibrotactile and light-based notification modalities. Finally, the third prototype run on a smart-ring which afforded graphical- based (i.e. abstract light) information with and light and vibration notification modalities. To test the suitability of these prototypes for acting as “secondary display” systems, we conducted a within-subjects user study where three university students (n= 3, Mage=28, mixed educational background) argued for solving three different problem cases and producing an argument map in each of the three consecutive meetings (max 90min) in the Argue(a)ware instructional system. Students were assigned the roles of writer, corrector and devil`s advocate and were instructed to maintain the same role across the three meetings. In each meeting, students worked with a different role-based awareness notification prototype, where they received a notification indicating their balloon is not growing bigger after five minutes of not exhibiting any role-specific behaviors. The role-based awareness notification prototypes aimed at introducing timely interventions which would prompt students to check on their own progress in the role and the group progress as visualized by the role-based awareness visualization on the large display. Ultimately, this should prompt them to reflect on the awareness information from the visualization and adapt their behaviors to the desired behavior standards over time. Results showed that students perceived the notifications from all media mostly based on vibration cues. Thereby, the vibration cues on the wrist (smartwatch) were considered the least disruptive to the main task compared to the vibration cues on finger (smartwatch) and the vibration cues on the desk (smartphone). Students also declared that vibration cues on wrist prompted the fastest reaction i.e., attending to notification by interacting with the smartwatch. These results indicate that vibration cues on the wrist can be a suitable notification mechanism for increasing the perceived urgency of the message and prompting the reaction on it without causing great distraction to the main task, as studies previous studies showed before (Pielot, Church, & deOliveira, 2013; Hernández-Leo, Balestrini, Nieves & Blat, 2012). Based on very limited qualitative data on light as notification modality and awareness representation type no inferences could be made about its influence on the cost of interruption, reaction and comprehension parameters comprehensiveness. The qualitative and quantitative data on the experience with different media as awareness notification systems indicate that smartwatches may be the most suitable medium for acting as awareness notification medium with a “secondary display” IRC configuration (low-high-high). However, this inference needs to be tested in terms of a follow up study. In the next study, the great limitations of study (limited data due to low power and mal-structured measurement instruments) need to be repaired. Finally, the focus should be on comparing notification modalities of one medium (e.g., smartphone) based on a larger set of participants and with the use of objective measurements for the IRC parameter values (Chewar, McCrickard & Sutcliffe, 2004). Finally, we draw conclusions based on the findings from the three studies with respect to the role of awareness mechanisms for facilitating collaborative processes and outcomes and provide replicable and generalizable design principles. These principles are formed as heuristic statements and are subject to refinement by further research (Bell, Hoadley, & Linn, 2004; Van den Akker, 1999). We conclude with the limitations of the study and ideas for future work with Argue(a)ware

THE "POWER" OF TEXT PRODUCTION ACTIVITY IN COLLABORATIVE MODELING : NINE RECOMMENDATIONS TO MAKE A COMPUTER SUPPORTED SITUATION WORK

Language is not a direct translation of a speaker’s or writer’s knowledge or intentions. Various complex processes and strategies are involved in serving the needs of the audience: planning the message, describing some features of a model and not others, organizing an argument, adapting to the knowledge of the reader, meeting linguistic constraints, etc. As a consequence, when communicating about a model, or about knowledge, there is a complex interaction between knowledge and language. In this contribution, we address the question of the role of language in modeling, in the specific case of collaboration over a distance, via electronic exchange of written textual information. What are the problems/dimensions a language user has to deal with when communicating a (mental) model? What is the relationship between the nature of the knowledge to be communicated and linguistic production? What is the relationship between representations and produced text? In what sense can interactive learning systems serve as mediators or as obstacles to these processes

Content analysis: What are they talking about?

Quantitative content analysis is increasingly used to surpass surface level analyses in Computer-Supported Collaborative Learning (e.g., counting messages), but critical reflection on accepted practice has generally not been reported. A review of CSCL conference proceedings revealed a general vagueness in definitions of units of analysis. In general, arguments for choosing a unit were lacking and decisions made while developing the content analysis procedures were not made explicit. In this article, it will be illustrated that the currently accepted practices concerning the ‘unit of meaning’ are not generally applicable to quantitative content analysis of electronic communication. Such analysis is affected by ‘unit boundary overlap’ and contextual constraints having to do with the technology used. The analysis of e-mail communication required a different unit of analysis and segmentation procedure. This procedure proved to be reliable, and the subsequent coding of these units for quantitative analysis yielded satisfactory reliabilities. These findings have implications and recommendations for current content analysis practice in CSCL research

Software engineering whispers: The effect of textual vs. graphical software design descriptions on software design communication

Context:\ua0Software\ua0engineering\ua0is a social and collaborative activity. Communicating and sharing knowledge between\ua0software\ua0developers requires much effort. Hence, the quality of\ua0communication\ua0plays an important role in influencing project success. To better understand the\ua0effect\ua0of\ua0communication\ua0on project success, more in-depth empirical studies investigating this phenomenon are needed. Objective: We investigate the\ua0effect\ua0of using a\ua0graphical\ua0versus\ua0textual\ua0design\ua0description\ua0on co-located\ua0software\ua0design\ua0communication. Method: Therefore, we conducted a family of experiments involving a mix of 240\ua0software\ua0engineering\ua0students from four universities. We examined how different\ua0design\ua0representations (i.e.,\ua0graphical\ua0vs.\ua0textual) affect the ability to Explain, Understand, Recall, and Actively Communicate knowledge. Results: We found that the\ua0graphical\ua0design\ua0description\ua0is better than the\ua0textual\ua0in promoting Active Discussion between developers and improving the Recall of\ua0design\ua0details. Furthermore, compared to its unaltered version, a well-organized and motivated\ua0textual\ua0design\ua0description–that is used for the same amount of time–enhances the recall of\ua0design\ua0details and increases the amount of active discussions at the cost of reducing the perceived quality of explaining

Conceptualizing and supporting awareness of collaborative argumentation

In this thesis, we introduce “Argue(a)ware”. This is a concept for an instructional group awareness tool which aims at supporting social interactions in co-located computer-supported collaborative argumentation settings. Argue(a)ware is designed to support the social interactions in the content (i.e., task-related) and in the relational (i.e., social and interpersonal) space of co-located collaborative argumentation (Barron, 2003). The support for social interactions in the content space of collaboration is facilitated with the use of collaborative scripts for argumentation (i.e., instructions and scaffolds of argument construction) as well with the use of an argument mapping tool (i.e., visualization of argumentation outcomes in a form of diagrams) (Stegmann, Weinberger, & Fischer, 2007; van Gelder, 2013). The support for social interactions in the relational space of collaboration is facilitated with the use of different awareness mechanisms from the CSCL and the CSCW research fields (i.e., monitoring, mirroring and awareness notification tools). In this thesis, we examined how different awareness mechanisms facilitate the regulation of collaborative processes in the relational space of collaborative argumentation. Moreover, we studied how they affect the perceived team effectiveness (i.e., process outcome) and group performance (i.e., learning outcome) in the content space of collaboration. Thereby, we studied also the effects of the design of the awareness mechanisms on the application of the mechanisms and the user experience with them. In line with the design-based research paradigm, we attempted to simultaneously improve and study the effect of Argue(a)ware on collaborative argumentation (Herrington, McKenney, Reeves & Oliver, 2007). Through a series of design-based research studies we tested and refined the prototypes of the instructional group awareness tool. Moreover, we studied the ecological validity of dominant awareness and instructional theories in the context of co-located computer-supported collaborative argumentation. The underlying premise of the Argue(a)ware tool is that a combination of awareness and instructional support will result in increased awareness of collaboration, which will, in turn, mediate the regulation of collaborative processes. Moreover, we assume that successful regulation of collaboration will result in high perceived team effectiveness and the group performance in turn. In the first phase of development of the Argue(a)ware tool, we built support of the content space of collaborative argumentation with argument scaffold elements in a pedagogical face-to-face macro-script and an argument mapping tool. Furthermore, we extended the use of the script for supporting the relational space of collaboration by embedding awareness prompts for reflecting on collaboration during regular breaks in the script. Following, we designed two variations of the same pedagogical face-to-face macro-script which differ with respect to the type of group awareness prompts they used for supporting the relational space of collaboration i.e. behavioral and social. Upon designing the two script variations, we conducted a longitudinal, multiple-case study with ten groups of Media Informatics master students (n = 28, in groups of three or two, group=case, 4 sessions x70 min, Behavioural Awareness Script group= 5, Social Awareness Script group =5.) where each group was conceptualized as a case. Students collaborated every time for arguing to solve one different ill-structured problem and for transferring their arguments in the argument mapping tool Rationale. Thereby, we intended to investigate the effects of different awareness prompts on (a) collaborative metacognitive processes i.e., regulation, reflection, and evaluation (b) the relation between collaborative metacognitive processes and the quality of collaborative argumentation as well as (c) the impact of the two script variations on perceived team effectiveness and (d) what was experience with the different parts of the script variations in the two groups and how this fits into the design framework by Buder (2011). The quantitative analysis of argument outcomes from the groups yield no significant difference between the groups that worked with the BAS and the SAS variations. No significant difference between the script variations with respect to the results from the team effectiveness questionnaires was found either. Prompts for regulating collaboration processes were found to be the most successfully and consistently applied ones, especially in the most successful cases from both script variations and have influenced the argumentation outcomes. The awareness prompts afforded an explicit feedback display format (e.g. assessment of participation levels of self- and others) through discussion (Buder, 2011). The prompted explicit feedback display format (i.e., ratings of one’s self and of others) was criticized for running only on subjective awareness information on participation, contribution efforts and performance in the role. This resulted in evaluation apprehension phenomena (Cottrell, 1972) and evaluation bias (i.e., users may have not assessed themselves or others frankly) (Ghadirian et al., 2016). The awareness prompts for reflection and evaluation did reveal frictions in the plan making process (i.e., dropping out of the plan for collaboration) in the least successful groups. Problems with group dynamics (i.e., free-loading and presence of dominance) but were not powerful enough to trigger the desired changes in the behaviors of the students. The prompts for evaluating the collaboration in both script variations had no apparent connection to argumentation outcomes. The results indicated that dominant presence phenomena inhibited substantive argumentation in the least successful groups. They also indicated that the role-assignment influenced the group dynamics by helping student’s making clear the labor division in the group. In the second phase of development of the Argue(a)ware tool, the focus is on structuring and regulating social interactions in the relational space of collaborative argumentation by means of scripted roles and role-based awareness scaffolds. We designed support for mirroring participation in the role (i.e., a role-based awareness visualization) and support for monitoring participation, coordination and collaboration efforts in the role (i.e., self-assessment questionnaire). Moreover, we designed additional support for guiding participation in the role i.e., role-based reminders as notifications on smartwatches. In a between-subjects study, ten groups of three university students each (n = 30, Mage =22y, mixed educational backgrounds, 1x90min) worked with two variants of the Argue(a)ware for arguing to solve one ill-structured problem and transferring their arguments in the argument mapping tool Rationale. Next, to that, students should monitor their progress in their role with the role-based awareness visualization and the self-assessment questionnaire with the basic awareness support (role-based awareness visualization with the intermediate self-assessment) and the enhanced awareness support (additional role-based awareness reminders). Half of the groups worked only with the role-based awareness visualization and the self-assessment questionnaire (Basic Awareness Condition-BAC) while the other half groups received additional text-based awareness notifications via smartwatches that were sent to students privately (Enhanced Awareness Condition- EAC). Thereby, we tested the use of different degrees of awareness support in the two conditions with respect to their impact on a) self-perceived awareness of performance in the role and of collaboration and coordination efforts (measured with the same questionnaire at two time points), b) on perceive team effectiveness, c) group performance. We hypothesized that students in EAC will perform better thanks to the additional awareness reminders that increased the directivity and influenced their awareness in the role. The mixed methods analysis revealed that the awareness reminders, when perceived on time, succeeded in guiding collaboration (i.e., resulted in more role-specific behaviors). Students in the EAC condition improved their awareness over time (between the two measurements). These results indicated that enhanced awareness support in the form of additional guidance through awareness reminders can boost the awareness of students’ performance in the role as well as the awareness of their coordination and collaboration efforts over time by directing them back to the mirroring and monitoring tools. Moreover, students in EAC exhibited higher perceived team effectiveness than the students in BAC. However, no significant differences in building of shared mental models or performing in mutual performance monitoring were found between the groups. However, students in BAC and EAC did not differ significantly with respect to the formal correctness or evidence sufficiency of their group argumentation outcomes. Moreover, technical difficulties with the smartphones used as delivery devices for the awareness reminders (i.e., low vibration modus) hindered the timely perception of the reminders and thus their effect on participation. Finally, the questionnaire on the experience with the different parts of Argue(a)ware system indicated the need for exploring further media for supporting the awareness reminders to avoid the overwhelming effects of the multiple displays of the system and enhancing higher perceptiveness of the reminders with low interruption costs for other group members. The rather high satisfaction with the use of the role-based awareness visualization and the positive comments on the motivating aspects of monitoring how the personal success contributes to the group performance indicate that the group mirror succeeded in making group norms visible to group members in a non-obtrusive way. The high interpersonal comparability of performances without moderating the group ‘s interaction directly in the basic awareness condition was proven to be the favored design approach compared to the combination of group mirror and awareness reminders in the enhance awareness condition. In the third phase of development of Argue(a)ware, we focused on designing and testing different notification modes on different ubiquitous mobile devices for facilitating the next prototype of a notification system for role-based awareness reminders. Thereby, the aim of the system was again to guide students’ active participation in collaborative argumentation. More specifically, we focused on raising students’ attention to the reminders and triggering a prompter reaction to the contents of the reminders whilst avoiding a high interruption cost for the primary task (i.e., arguing for solving the problem at hand) in the group. These goals were translated into design challenges for the design of the role-based awareness notification system. The system should afford low interruptions, high reaction and high comprehension of notifications. Notification systems with this particular configuration of IRC values are known as "secondary display" systems (McCrickard et al., 2003). Next, we designed three low-fidelity prototypes for a role-based notification system for delivering awareness reminders: The first ran on a smartwatch and afforded text-based information with vibration and light notification modalities. The second ran on smartphone and afforded text-based information with vibrotactile and light-based notification modalities. Finally, the third prototype run on a smart-ring which afforded graphical- based (i.e. abstract light) information with and light and vibration notification modalities. To test the suitability of these prototypes for acting as “secondary display” systems, we conducted a within-subjects user study where three university students (n= 3, Mage=28, mixed educational background) argued for solving three different problem cases and producing an argument map in each of the three consecutive meetings (max 90min) in the Argue(a)ware instructional system. Students were assigned the roles of writer, corrector and devil`s advocate and were instructed to maintain the same role across the three meetings. In each meeting, students worked with a different role-based awareness notification prototype, where they received a notification indicating their balloon is not growing bigger after five minutes of not exhibiting any role-specific behaviors. The role-based awareness notification prototypes aimed at introducing timely interventions which would prompt students to check on their own progress in the role and the group progress as visualized by the role-based awareness visualization on the large display. Ultimately, this should prompt them to reflect on the awareness information from the visualization and adapt their behaviors to the desired behavior standards over time. Results showed that students perceived the notifications from all media mostly based on vibration cues. Thereby, the vibration cues on the wrist (smartwatch) were considered the least disruptive to the main task compared to the vibration cues on finger (smartwatch) and the vibration cues on the desk (smartphone). Students also declared that vibration cues on wrist prompted the fastest reaction i.e., attending to notification by interacting with the smartwatch. These results indicate that vibration cues on the wrist can be a suitable notification mechanism for increasing the perceived urgency of the message and prompting the reaction on it without causing great distraction to the main task, as studies previous studies showed before (Pielot, Church, & deOliveira, 2013; Hernández-Leo, Balestrini, Nieves & Blat, 2012). Based on very limited qualitative data on light as notification modality and awareness representation type no inferences could be made about its influence on the cost of interruption, reaction and comprehension parameters comprehensiveness. The qualitative and quantitative data on the experience with different media as awareness notification systems indicate that smartwatches may be the most suitable medium for acting as awareness notification medium with a “secondary display” IRC configuration (low-high-high). However, this inference needs to be tested in terms of a follow up study. In the next study, the great limitations of study (limited data due to low power and mal-structured measurement instruments) need to be repaired. Finally, the focus should be on comparing notification modalities of one medium (e.g., smartphone) based on a larger set of participants and with the use of objective measurements for the IRC parameter values (Chewar, McCrickard & Sutcliffe, 2004). Finally, we draw conclusions based on the findings from the three studies with respect to the role of awareness mechanisms for facilitating collaborative processes and outcomes and provide replicable and generalizable design principles. These principles are formed as heuristic statements and are subject to refinement by further research (Bell, Hoadley, & Linn, 2004; Van den Akker, 1999). We conclude with the limitations of the study and ideas for future work with Argue(a)ware

Legal knowledge-based systems: new directions in system design

This thesis examines and critiques the concept of 'legal knowledge-based’ systems. Work on legal knowledge-based systems is dominated by work in 'artificial intelligence and law’. It seeks to automate the application of law and to automate the solution of legal problems. Automation however, has proved elusive. In contrast to such automation, this thesis proposes the creation of legal knowledge-based systems based on the concept of augmentation of legal work. Focusing on systems that augment legal work opens new possibilities for system creation and use. To inform how systems might augment legal work, this thesis examines philosophy, psychology and legal theory for information they provide on how processes of legal reasoning operate. It is argued that, in contrast to conceptions of law adopted in artificial intelligence and law, 'sensemaking' provides a useful perspective with which to create systems. It is argued that visualisation, and particularly diagrams, are an important and under considered element of reasoning and that producing systems that support diagramming of processes of legal reasoning would provide useful support for legal work. This thesis reviews techniques for diagramming aspects of sensemaking. In particular this thesis examines standard methods for diagramming arguments and methods for diagramming reasoning. These techniques are applied in the diagramming of legal judgments. A review is conducted of systems that have been constructed to support the construction of diagrams of argument and reasoning. Drawing upon these examinations, this thesis highlights the necessity of appropriate representations for supporting reasoning. The literature examining diagramming for reasoning support provides little discussion of appropriate representations. This thesis examines theories of representation for insight they can provide into the design of appropriate representations. It is concluded that while the theories of representation that are examined do not determine what amounts to a good representation, guidelines for the design and choice of representations can be distilled. These guidelines cannot map the class of legal knowledge-based systems that augment legal sensemaking, they can however, be used to explore this class and to inform construction of systems

Towards adaptive argumentation learning systems : theoretical and practical considerations in the design of argumentation learning systems

This dissertation addresses four issues of pivotal importance in realizing the promises of adaptive argumentation learning systems: (1) User interface: How can argumentation user interfaces be designed to effectively structure and support problem solving, peer interaction, and learning? (2) Software architecture: How can software architectures of adaptive argumentation learning systems be designed to be employable across different argumentation domains and application scenarios in a flexible and cost-effective manner? (3) Diagnostics: How can user behavior be analyzed, automatically and accurately, to drive automated adaptations and help generation? (4) Adaptation: How can strategies for automated adaptation and support be designed to promote problem solving, peer interaction, and learning in an optimal fashion? Regarding issue (1), this dissertation investigates argument diagrams and structured discussion interfaces, two areas of focal interest in argumentation learning research during the past decades. The foundation for such structuring approaches is given by theories of learning and teaching with knowledge representations (theory of representational guidance) and collaboration scripts (script theory of guidance in computer-supported collaborative learning). This dissertation brings these two strands of research together and presents a computer-based learning environment that combines both approaches to support students in conducting high-quality discussions of controversial texts. An empirical study confirms that this combined approach has positive impact on the quality of discussions, thus, underpins the theoretical basis of the approach. Regarding issue (2), this dissertation presents a software framework for enhancing argumentation systems with adaptive support mechanisms. Adaptive support functionality of past argumentation systems has been tailored to particular domains and application scenarios. A novel software framework is presented that abstracts from the specific demands of different domains and application scenarios to provide a more general approach. The approach comprises an extensive configuration subsystem that allows the flexible definition of intelligent software agents, that is, software components able to reason and act autonomously to help students engage in fruitful learning activities. A graphical authoring tool has been conceptualized and implemented to simplify the process of defining and administering software agents beyond what has been achieved with the provided framework system. Among other things, the authoring tool allows, for the first time, specifying relevant patterns in argument diagrams using a graphical language. Empirical results indicate the high potential of the authoring approach but also challenges for future research. Regarding issue (3), the dissertation investigates two alternative approaches to automatically analyzing argumentation learning activities: the knowledge-driven and the data-driven analysis method. The knowledge-driven approach utilizes a pattern search component to identify relevant structures in argument diagrams based on declarative pattern specifications. The capabilities and appropriateness of this approach are demonstrated through three exemplary applications, for which pedagogically relevant patterns have been defined and implemented within the component. The approach proves particularly useful for patterns of limited complexity in scenarios with sufficient expert knowledge available. The data-driven approach is based on machine learning techniques, which have been employed to induce computational classifiers for important aspects of graphical online discussions, such as off-topic contributions, reasoned claims, and question-answer interactions. Validation results indicate that this approach can be realistically used even for complex classification tasks involving natural language. This research constitutes the first investigation on the use of machine learning techniques to analyze diagram-based educational discussions. The dissertation concludes with discussing the four addressed research challenges in the broader context of existing theories and empirical results. The pros and cons of different options in the design of argumentation learning systems are juxtaposed; areas for future research are identified. This final part of the dissertation gives researchers and practitioners a synopsis of the current state of the art in the design of argumentation learning systems and its theoretical and empirical underpinning. Special attention is paid to issue (4), with an in-depth discussion of existing adaptation approaches and corresponding empirical results.Diese Dissertationsschrift behandelt die folgenden vier Fragestellungen, welche bei der Realisierung adaptiver Argumentationssysteme von zentraler Bedeutung sind: (1) Benutzerschnittstelle: Wie müssen Benutzerschnittstellen beschaffen sein, um Problemlöse-, Kooperations- und Lernprozesse effektiv zu strukturieren und zu unterstützen? (2) Softwarearchitektur: Wie können die Funktionalitäten eines adaptiven Argumentationslernsystems in eine Softwarearchitektur abgebildet werden, welche flexibel und mit angemessenem Aufwand in verschiedenen Bereichen und Szenarien einsetzbar ist? (3) Diagnostik: Wie kann Benutzerverhalten automatisch und mit hoher Genauigkeit analysiert werden, um automatisierte Anpassungen und Hilfestellungen effektiv zu steuern? (4) Adaption: Wie sollten automatisierte Anpassungen und Hilfestellungen ausgestaltet werden, um Problemlöse-, Kooperations- und Lernprozesse optimal zu unterstützen? Hinsichtlich Fragestellung (1) untersucht diese Arbeit Argumentationsdiagramme und strukturierte Onlinediskussionen, zwei Schwerpunkte der Forschung zu Lernsystemen für Argumentation der vergangenen Jahre. Die Grundlage solcher Strukturierungsansätze bilden Theorien zum Lehren und Lernen mit Wissensrepräsentationen (theory of representational guidance) und Kooperationsskripten (script theory of guidance in computer-supported collaborative learning). Diese Arbeit führt beide Forschungsstränge in einer neuartigen Lernumgebung zusammen, die beide Ansätze vereint, um Lernende beim Diskutieren kontroverser Texte zu unterstützen. Eine empirische Untersuchung zeigt, dass sich dieser kombinierte Ansatz positiv auf die Diskussionsqualität auswirkt und bekräftigt damit die zu Grunde liegenden theoretischen Annahmen. Hinsichtlich Fragestellung (2) stellt diese Arbeit ein Software-Rahmensystem zur Bereitstellung adaptiver Unterstützungsmechanismen in Argumentationssystemen vor. Das Rahmensystem abstrahiert von domänen- und anwendungsspezifischen Besonderheiten und stellt damit einen generelleren Ansatz im Vergleich zu früheren Systemen dar. Der Ansatz umfasst ein umfangreiches Konfigurationssystem zur Definition intelligenter Softwareagenten, d. h. Softwarekomponenten, die eigeständig schlussfolgern und handeln, um Lernprozesse zu unterstützen. Um das Definieren und Administrieren von Softwareagenten über das bereitgestellte Rahmensystem hinaus zu vereinfachen, wurde ein grafisches Autorenwerkzeug konzipiert und entwickelt. Unter anderem erlaubt dieses erstmals, relevante Muster in Argumentationsdiagrammen ohne Programmierung mittels einer grafischen Sprache zu spezifizieren. Empirische Befunde zeigen neben dem hohen Potential des Ansatzes auch die Notwendigkeit weiterführender Forschung. Hinsichtlich Fragestellung (3) untersucht diese Arbeit zwei alternative Ansätze zur automatisierten Analyse von Lernaktivitäten im Bereich Argumentation: die wissensbasierte und die datenbasierte Analysemethodik. Der wissensbasierte Ansatz wurde mittels einer Softwarekomponente zur Mustersuche in Argumentationsdiagrammen umgesetzt, welche auf Grundlage deklarativer Musterbeschreibungen arbeitet. Die Möglichkeiten und Eignung des Ansatzes werden anhand von drei Beispielszenarien demonstriert, für die verschiedenartige, pädagogisch relevante Muster innerhalb der entwickelten Softwarekomponente definiert wurden. Der Ansatz erweist sich insbesondere als nützlich für Muster eingeschränkter Komplexität in Szenarien, für die Expertenwissen in ausreichendem Umfang verfügbar ist. Der datenbasierte Ansatz wurde mittels maschineller Lernverfahren umgesetzt. Mit deren Hilfe wurden Klassifikationsroutinen zur Analyse zentraler Aspekte von Onlinediskussionen, wie beispielsweise themenfremde Beiträge, begründete Aussagen und Frage-Antwort-Interaktionen, algorithmisch hergeleitet. Validierungsergebnisse zeigen, dass sich dieser Ansatz selbst für komplexe Klassifikationsprobleme eignet, welche die Berücksichtigung natürlicher Sprache erfordern. Dies ist die erste Arbeit zum Einsatz maschineller Lernverfahren zur Analyse von diagrammbasierten Lerndiskussionen. Die Arbeit schließt mit einer Diskussion des aktuellen Forschungsstands hinsichtlich der vier Fragestellungen im breiteren Kontext existierender Theorien und empirischer Befunde. Die Vor- und Nachteile verschiedener Optionen für die Gestaltung von Lernsystemen für Argumentation werden gegenübergestellt und zukünftige Forschungsfelder vorgeschlagen. Dieser letzte Teil der Arbeit bietet Forschern und Anwendern einen umfassenden Überblick des aktuellen Forschungsstands bezüglich des Designs computerbasierter Argumentationslernsysteme und den zugrunde liegenden lehr- und lerntheoretischen Erkenntnissen. Insbesondere wird auf Fragestellung (4) vertiefend eingegangen und bisherige Adaptionsansätze einschließlich entsprechender empirischer Befunde erörtert