Good for learning, bad for motivation? A meta-analysis on the effects of computer-supported collaboration scripts

Scripting computer-supported collaborative learning has been shown to greatly enhance learning, but is often criticized for hindering learners’ agency and thus undermining learners’ motivation. Beyond that, what makes some CSCL scripts particularly effective for learning is still a conundrum. This meta-analysis synthesizes the results of 53 primary studies that experimentally compared the effect of learning with a CSCL script to unguided collaborative learning on at least one of the variables motivation, domain learning, and collaboration skills. Overall, 5616 learners enrolled in K-12, higher education, or professional development participated in the included studies. The results of a random-effects meta-analysis show that learning with CSCL scripts leads to a non-significant positive effect on motivation (Hedges’ g = 0.13), a small positive effect (Hedges’ g = 0.24) on domain learning and a medium positive effect (Hedges’ g = 0.72) on collaboration skills. Additionally, the meta-analysis shows how scaffolding single particular collaborative activities and scaffolding a combination of collaborative activities affects the effectiveness of CSCL scripts and that synergistic or differentiated scaffolding is hard to achieve. This meta-analysis offers the first counterevidence against the widespread criticism that CSCL scripts have negative motivational effects. Furthermore, the findings can be taken as evidence for the robustness of the positive effects on domain learning and collaboration skills

Collaboration scripts - a conceptual analysis

This article presents a conceptual analysis of collaboration scripts used in face-to-face and computer-mediated collaborative learning. Collaboration scripts are scaffolds that aim to improve collaboration through structuring the interactive processes between two or more learning partners. Collaboration scripts consist of at least five components: (a) learning objectives, (b) type of activities, (c) sequencing, (d) role distribution, and (e) type of representation. These components serve as a basis for comparing prototypical collaboration script approaches for face-to-face vs. computer-mediated learning. As our analysis reveals, collaboration scripts for face-to-face learning often focus on supporting collaborators in engaging in activities that are specifically related to individual knowledge acquisition. Scripts for computer-mediated collaboration are typically concerned with facilitating communicative-coordinative processes that occur among group members. The two lines of research can be consolidated to facilitate the design of collaboration scripts, which both support participation and coordination, as well as induce learning activities closely related to individual knowledge acquisition and metacognition. In addition, research on collaboration scripts needs to consider the learners’ internal collaboration scripts as a further determinant of collaboration behavior. The article closes with the presentation of a conceptual framework incorporating both external and internal collaboration scripts

Effects of differently sequenced classroom scripts on transformative and regulative processes in inquiry learning

Kooperatives Forschendes Lernen hat sich empirisch als ein effektiver Instruktionsansatz für die Förderung des naturwissenschaftlichen Denkens bewährt. Obwohl Forschung zur Orchestrierung von Sozialformen im Unterricht zeigt, dass diese einen wichtigen Einfluss auf die Qualität von Lernprozessen, wie Kommunikations- und Interaktionsprozessen, und damit auf die Lernergebnisse von Gruppe und einzelnen Lernenden hat, wurde im Bereich des Forschenden Lernens die Verteilung und Abfolge von individuellen und kooperativen Lernaktivitäten bislang jedoch kaum untersucht. Basierend auf Erkenntnissen zu Scaffolding, Fading, Productive Failure und dem ICAP-Rahmenmodell wird in der vorliegenden Arbeit der Einfluss zweier Unterrichtsskripts auf die transformativen und regulativen Prozesse des forschenden Lernens bei Individuen und Gruppen untersucht. Das eine Unterrichtsskript sieht die Abfolge „Plenum-Kleingruppe-Individuum“ vor (PKI-Skript), das andere wechselt vom Plenum über die individuelle Ebene zur Kleingruppenebene (PIK-Skript). Transformationsprozesse beziehen sich dabei auf wissensgenerierende Prozesse, während regulative Prozesse meta-kognitive Prozesse darstellen. Deskriptiv zeigten sich unterschieden zwischen den beiden Bedingungen: Lernende mit dem PKI-Skript zeigten mehr und intensivere individuelle transformative Prozesse, z.B. während bei der Datenauswertung und beim wissenschaftlichen Schlussfolgern. Lernende mit dem PIK-Skript zeigten hingegen mehr transformative und regulative Prozessen auf der Gruppenebene. Lernende, die mit diesem Skript arbeiteten, zeigten mehr und intensivere Grounding-Aktivitäten, die das gemeinsame Verständnis und das Entstehen eines Common Ground förderten. Dementsprechend zeigten sich hier auch häufiger intensivere transformative Prozesse auf der Gruppenebene.Collaborative inquiry learning has been empirically proven to be an effective instructional approach to foster students’ scientific literacy. However, there is little research on the coordination of individual and collaborative activities during inquiry learning which could shape the quality of communication and interaction, and consequentially, individual and group learning outcomes. Research has indicated that classroom orchestration (i.e., distribution and sequencing of activities) could have profound effect on learning processes and outcomes. Premised on theories of scaffolding, fading, productive failure and the ICAP (interactive, constructive, active and passive) framework on different activity types, this study investigates the effects of two differently sequenced classroom scripts on the individual and group transformative and regulative processes in inquiry learning. Transformative processes refers to processes that yield knowledge and regulative processes are meta-cognitive processes. Descriptive statistics suggest that the Plenary-Small Group-Individual (PSI) script transition facilitated better individual engagement in transformative processes such as generating of evidence and the drawing of conclusions, whereas the Plenary-Individual-Small Group (PIS) script condition fostered better transformative and regulative processes for the group. Establishing shared understanding and forging common grounds through grounding and high-level grounding was more prevalent in this script condition, which also accounted for more occurrences of high-level transformative processes at the group level

Conceptualizing and supporting awareness of collaborative argumentation

In this thesis, we introduce “Argue(a)ware”. This is a concept for an instructional group awareness tool which aims at supporting social interactions in co-located computer-supported collaborative argumentation settings. Argue(a)ware is designed to support the social interactions in the content (i.e., task-related) and in the relational (i.e., social and interpersonal) space of co-located collaborative argumentation (Barron, 2003). The support for social interactions in the content space of collaboration is facilitated with the use of collaborative scripts for argumentation (i.e., instructions and scaffolds of argument construction) as well with the use of an argument mapping tool (i.e., visualization of argumentation outcomes in a form of diagrams) (Stegmann, Weinberger, & Fischer, 2007; van Gelder, 2013). The support for social interactions in the relational space of collaboration is facilitated with the use of different awareness mechanisms from the CSCL and the CSCW research fields (i.e., monitoring, mirroring and awareness notification tools). In this thesis, we examined how different awareness mechanisms facilitate the regulation of collaborative processes in the relational space of collaborative argumentation. Moreover, we studied how they affect the perceived team effectiveness (i.e., process outcome) and group performance (i.e., learning outcome) in the content space of collaboration. Thereby, we studied also the effects of the design of the awareness mechanisms on the application of the mechanisms and the user experience with them. In line with the design-based research paradigm, we attempted to simultaneously improve and study the effect of Argue(a)ware on collaborative argumentation (Herrington, McKenney, Reeves & Oliver, 2007). Through a series of design-based research studies we tested and refined the prototypes of the instructional group awareness tool. Moreover, we studied the ecological validity of dominant awareness and instructional theories in the context of co-located computer-supported collaborative argumentation. The underlying premise of the Argue(a)ware tool is that a combination of awareness and instructional support will result in increased awareness of collaboration, which will, in turn, mediate the regulation of collaborative processes. Moreover, we assume that successful regulation of collaboration will result in high perceived team effectiveness and the group performance in turn. In the first phase of development of the Argue(a)ware tool, we built support of the content space of collaborative argumentation with argument scaffold elements in a pedagogical face-to-face macro-script and an argument mapping tool. Furthermore, we extended the use of the script for supporting the relational space of collaboration by embedding awareness prompts for reflecting on collaboration during regular breaks in the script. Following, we designed two variations of the same pedagogical face-to-face macro-script which differ with respect to the type of group awareness prompts they used for supporting the relational space of collaboration i.e. behavioral and social. Upon designing the two script variations, we conducted a longitudinal, multiple-case study with ten groups of Media Informatics master students (n = 28, in groups of three or two, group=case, 4 sessions x70 min, Behavioural Awareness Script group= 5, Social Awareness Script group =5.) where each group was conceptualized as a case. Students collaborated every time for arguing to solve one different ill-structured problem and for transferring their arguments in the argument mapping tool Rationale. Thereby, we intended to investigate the effects of different awareness prompts on (a) collaborative metacognitive processes i.e., regulation, reflection, and evaluation (b) the relation between collaborative metacognitive processes and the quality of collaborative argumentation as well as (c) the impact of the two script variations on perceived team effectiveness and (d) what was experience with the different parts of the script variations in the two groups and how this fits into the design framework by Buder (2011). The quantitative analysis of argument outcomes from the groups yield no significant difference between the groups that worked with the BAS and the SAS variations. No significant difference between the script variations with respect to the results from the team effectiveness questionnaires was found either. Prompts for regulating collaboration processes were found to be the most successfully and consistently applied ones, especially in the most successful cases from both script variations and have influenced the argumentation outcomes. The awareness prompts afforded an explicit feedback display format (e.g. assessment of participation levels of self- and others) through discussion (Buder, 2011). The prompted explicit feedback display format (i.e., ratings of one’s self and of others) was criticized for running only on subjective awareness information on participation, contribution efforts and performance in the role. This resulted in evaluation apprehension phenomena (Cottrell, 1972) and evaluation bias (i.e., users may have not assessed themselves or others frankly) (Ghadirian et al., 2016). The awareness prompts for reflection and evaluation did reveal frictions in the plan making process (i.e., dropping out of the plan for collaboration) in the least successful groups. Problems with group dynamics (i.e., free-loading and presence of dominance) but were not powerful enough to trigger the desired changes in the behaviors of the students. The prompts for evaluating the collaboration in both script variations had no apparent connection to argumentation outcomes. The results indicated that dominant presence phenomena inhibited substantive argumentation in the least successful groups. They also indicated that the role-assignment influenced the group dynamics by helping student’s making clear the labor division in the group. In the second phase of development of the Argue(a)ware tool, the focus is on structuring and regulating social interactions in the relational space of collaborative argumentation by means of scripted roles and role-based awareness scaffolds. We designed support for mirroring participation in the role (i.e., a role-based awareness visualization) and support for monitoring participation, coordination and collaboration efforts in the role (i.e., self-assessment questionnaire). Moreover, we designed additional support for guiding participation in the role i.e., role-based reminders as notifications on smartwatches. In a between-subjects study, ten groups of three university students each (n = 30, Mage =22y, mixed educational backgrounds, 1x90min) worked with two variants of the Argue(a)ware for arguing to solve one ill-structured problem and transferring their arguments in the argument mapping tool Rationale. Next, to that, students should monitor their progress in their role with the role-based awareness visualization and the self-assessment questionnaire with the basic awareness support (role-based awareness visualization with the intermediate self-assessment) and the enhanced awareness support (additional role-based awareness reminders). Half of the groups worked only with the role-based awareness visualization and the self-assessment questionnaire (Basic Awareness Condition-BAC) while the other half groups received additional text-based awareness notifications via smartwatches that were sent to students privately (Enhanced Awareness Condition- EAC). Thereby, we tested the use of different degrees of awareness support in the two conditions with respect to their impact on a) self-perceived awareness of performance in the role and of collaboration and coordination efforts (measured with the same questionnaire at two time points), b) on perceive team effectiveness, c) group performance. We hypothesized that students in EAC will perform better thanks to the additional awareness reminders that increased the directivity and influenced their awareness in the role. The mixed methods analysis revealed that the awareness reminders, when perceived on time, succeeded in guiding collaboration (i.e., resulted in more role-specific behaviors). Students in the EAC condition improved their awareness over time (between the two measurements). These results indicated that enhanced awareness support in the form of additional guidance through awareness reminders can boost the awareness of students’ performance in the role as well as the awareness of their coordination and collaboration efforts over time by directing them back to the mirroring and monitoring tools. Moreover, students in EAC exhibited higher perceived team effectiveness than the students in BAC. However, no significant differences in building of shared mental models or performing in mutual performance monitoring were found between the groups. However, students in BAC and EAC did not differ significantly with respect to the formal correctness or evidence sufficiency of their group argumentation outcomes. Moreover, technical difficulties with the smartphones used as delivery devices for the awareness reminders (i.e., low vibration modus) hindered the timely perception of the reminders and thus their effect on participation. Finally, the questionnaire on the experience with the different parts of Argue(a)ware system indicated the need for exploring further media for supporting the awareness reminders to avoid the overwhelming effects of the multiple displays of the system and enhancing higher perceptiveness of the reminders with low interruption costs for other group members. The rather high satisfaction with the use of the role-based awareness visualization and the positive comments on the motivating aspects of monitoring how the personal success contributes to the group performance indicate that the group mirror succeeded in making group norms visible to group members in a non-obtrusive way. The high interpersonal comparability of performances without moderating the group ‘s interaction directly in the basic awareness condition was proven to be the favored design approach compared to the combination of group mirror and awareness reminders in the enhance awareness condition. In the third phase of development of Argue(a)ware, we focused on designing and testing different notification modes on different ubiquitous mobile devices for facilitating the next prototype of a notification system for role-based awareness reminders. Thereby, the aim of the system was again to guide students’ active participation in collaborative argumentation. More specifically, we focused on raising students’ attention to the reminders and triggering a prompter reaction to the contents of the reminders whilst avoiding a high interruption cost for the primary task (i.e., arguing for solving the problem at hand) in the group. These goals were translated into design challenges for the design of the role-based awareness notification system. The system should afford low interruptions, high reaction and high comprehension of notifications. Notification systems with this particular configuration of IRC values are known as "secondary display" systems (McCrickard et al., 2003). Next, we designed three low-fidelity prototypes for a role-based notification system for delivering awareness reminders: The first ran on a smartwatch and afforded text-based information with vibration and light notification modalities. The second ran on smartphone and afforded text-based information with vibrotactile and light-based notification modalities. Finally, the third prototype run on a smart-ring which afforded graphical- based (i.e. abstract light) information with and light and vibration notification modalities. To test the suitability of these prototypes for acting as “secondary display” systems, we conducted a within-subjects user study where three university students (n= 3, Mage=28, mixed educational background) argued for solving three different problem cases and producing an argument map in each of the three consecutive meetings (max 90min) in the Argue(a)ware instructional system. Students were assigned the roles of writer, corrector and devil`s advocate and were instructed to maintain the same role across the three meetings. In each meeting, students worked with a different role-based awareness notification prototype, where they received a notification indicating their balloon is not growing bigger after five minutes of not exhibiting any role-specific behaviors. The role-based awareness notification prototypes aimed at introducing timely interventions which would prompt students to check on their own progress in the role and the group progress as visualized by the role-based awareness visualization on the large display. Ultimately, this should prompt them to reflect on the awareness information from the visualization and adapt their behaviors to the desired behavior standards over time. Results showed that students perceived the notifications from all media mostly based on vibration cues. Thereby, the vibration cues on the wrist (smartwatch) were considered the least disruptive to the main task compared to the vibration cues on finger (smartwatch) and the vibration cues on the desk (smartphone). Students also declared that vibration cues on wrist prompted the fastest reaction i.e., attending to notification by interacting with the smartwatch. These results indicate that vibration cues on the wrist can be a suitable notification mechanism for increasing the perceived urgency of the message and prompting the reaction on it without causing great distraction to the main task, as studies previous studies showed before (Pielot, Church, & deOliveira, 2013; Hernández-Leo, Balestrini, Nieves & Blat, 2012). Based on very limited qualitative data on light as notification modality and awareness representation type no inferences could be made about its influence on the cost of interruption, reaction and comprehension parameters comprehensiveness. The qualitative and quantitative data on the experience with different media as awareness notification systems indicate that smartwatches may be the most suitable medium for acting as awareness notification medium with a “secondary display” IRC configuration (low-high-high). However, this inference needs to be tested in terms of a follow up study. In the next study, the great limitations of study (limited data due to low power and mal-structured measurement instruments) need to be repaired. Finally, the focus should be on comparing notification modalities of one medium (e.g., smartphone) based on a larger set of participants and with the use of objective measurements for the IRC parameter values (Chewar, McCrickard & Sutcliffe, 2004). Finally, we draw conclusions based on the findings from the three studies with respect to the role of awareness mechanisms for facilitating collaborative processes and outcomes and provide replicable and generalizable design principles. These principles are formed as heuristic statements and are subject to refinement by further research (Bell, Hoadley, & Linn, 2004; Van den Akker, 1999). We conclude with the limitations of the study and ideas for future work with Argue(a)ware

Facilitating Computer-Supported Collaborative Learning with Question-Asking Scripting Activity and its Effects on Students’ Conceptual Understanding and Critical Thinking in Science

Computer-supported collaborative learning (CSCL) provides an environment that enhances social interaction and shared knowledge construction among students. However, limited research has examined CSCL reinforced by question-asking scripting activity. This research investigated the effects of CSCL with question-asking scripting activity on the development of conceptual understanding and critical thinking in science. Moreover, the research design was a three-group pre-test-post-test quasi-experimental study in which the research sample were 106 Grade 7 students. The only difference between the control and the experimental groups was the exposure to CSCL. The experimental groups were exposed to CSCL approaches: one without scripting while one was exposed with question-asking scripting activity. Results revealed that CSCL approaches significantly affected the development of students’ conceptual understanding and critical thinking. Specifically, CSCL with scripting stimulated more intellectual discussion which allowed learners to deepen lesson comprehension and improve their critical thinking skills. Insights on the innovations through technology integration, collaborative inquiry learning, and question-asking activity to enhance science education were also discussed. The findings of this study have important implications for future practice

Effects of differently sequenced classroom scripts on transformative and regulative processes in inquiry learning

Kooperatives Forschendes Lernen hat sich empirisch als ein effektiver Instruktionsansatz für die Förderung des naturwissenschaftlichen Denkens bewährt. Obwohl Forschung zur Orchestrierung von Sozialformen im Unterricht zeigt, dass diese einen wichtigen Einfluss auf die Qualität von Lernprozessen, wie Kommunikations- und Interaktionsprozessen, und damit auf die Lernergebnisse von Gruppe und einzelnen Lernenden hat, wurde im Bereich des Forschenden Lernens die Verteilung und Abfolge von individuellen und kooperativen Lernaktivitäten bislang jedoch kaum untersucht. Basierend auf Erkenntnissen zu Scaffolding, Fading, Productive Failure und dem ICAP-Rahmenmodell wird in der vorliegenden Arbeit der Einfluss zweier Unterrichtsskripts auf die transformativen und regulativen Prozesse des forschenden Lernens bei Individuen und Gruppen untersucht. Das eine Unterrichtsskript sieht die Abfolge „Plenum-Kleingruppe-Individuum“ vor (PKI-Skript), das andere wechselt vom Plenum über die individuelle Ebene zur Kleingruppenebene (PIK-Skript). Transformationsprozesse beziehen sich dabei auf wissensgenerierende Prozesse, während regulative Prozesse meta-kognitive Prozesse darstellen. Deskriptiv zeigten sich unterschieden zwischen den beiden Bedingungen: Lernende mit dem PKI-Skript zeigten mehr und intensivere individuelle transformative Prozesse, z.B. während bei der Datenauswertung und beim wissenschaftlichen Schlussfolgern. Lernende mit dem PIK-Skript zeigten hingegen mehr transformative und regulative Prozessen auf der Gruppenebene. Lernende, die mit diesem Skript arbeiteten, zeigten mehr und intensivere Grounding-Aktivitäten, die das gemeinsame Verständnis und das Entstehen eines Common Ground förderten. Dementsprechend zeigten sich hier auch häufiger intensivere transformative Prozesse auf der Gruppenebene.Collaborative inquiry learning has been empirically proven to be an effective instructional approach to foster students’ scientific literacy. However, there is little research on the coordination of individual and collaborative activities during inquiry learning which could shape the quality of communication and interaction, and consequentially, individual and group learning outcomes. Research has indicated that classroom orchestration (i.e., distribution and sequencing of activities) could have profound effect on learning processes and outcomes. Premised on theories of scaffolding, fading, productive failure and the ICAP (interactive, constructive, active and passive) framework on different activity types, this study investigates the effects of two differently sequenced classroom scripts on the individual and group transformative and regulative processes in inquiry learning. Transformative processes refers to processes that yield knowledge and regulative processes are meta-cognitive processes. Descriptive statistics suggest that the Plenary-Small Group-Individual (PSI) script transition facilitated better individual engagement in transformative processes such as generating of evidence and the drawing of conclusions, whereas the Plenary-Individual-Small Group (PIS) script condition fostered better transformative and regulative processes for the group. Establishing shared understanding and forging common grounds through grounding and high-level grounding was more prevalent in this script condition, which also accounted for more occurrences of high-level transformative processes at the group level

Conceptualizing and supporting awareness of collaborative argumentation

In this thesis, we introduce “Argue(a)ware”. This is a concept for an instructional group awareness tool which aims at supporting social interactions in co-located computer-supported collaborative argumentation settings. Argue(a)ware is designed to support the social interactions in the content (i.e., task-related) and in the relational (i.e., social and interpersonal) space of co-located collaborative argumentation (Barron, 2003). The support for social interactions in the content space of collaboration is facilitated with the use of collaborative scripts for argumentation (i.e., instructions and scaffolds of argument construction) as well with the use of an argument mapping tool (i.e., visualization of argumentation outcomes in a form of diagrams) (Stegmann, Weinberger, & Fischer, 2007; van Gelder, 2013). The support for social interactions in the relational space of collaboration is facilitated with the use of different awareness mechanisms from the CSCL and the CSCW research fields (i.e., monitoring, mirroring and awareness notification tools). In this thesis, we examined how different awareness mechanisms facilitate the regulation of collaborative processes in the relational space of collaborative argumentation. Moreover, we studied how they affect the perceived team effectiveness (i.e., process outcome) and group performance (i.e., learning outcome) in the content space of collaboration. Thereby, we studied also the effects of the design of the awareness mechanisms on the application of the mechanisms and the user experience with them. In line with the design-based research paradigm, we attempted to simultaneously improve and study the effect of Argue(a)ware on collaborative argumentation (Herrington, McKenney, Reeves & Oliver, 2007). Through a series of design-based research studies we tested and refined the prototypes of the instructional group awareness tool. Moreover, we studied the ecological validity of dominant awareness and instructional theories in the context of co-located computer-supported collaborative argumentation. The underlying premise of the Argue(a)ware tool is that a combination of awareness and instructional support will result in increased awareness of collaboration, which will, in turn, mediate the regulation of collaborative processes. Moreover, we assume that successful regulation of collaboration will result in high perceived team effectiveness and the group performance in turn. In the first phase of development of the Argue(a)ware tool, we built support of the content space of collaborative argumentation with argument scaffold elements in a pedagogical face-to-face macro-script and an argument mapping tool. Furthermore, we extended the use of the script for supporting the relational space of collaboration by embedding awareness prompts for reflecting on collaboration during regular breaks in the script. Following, we designed two variations of the same pedagogical face-to-face macro-script which differ with respect to the type of group awareness prompts they used for supporting the relational space of collaboration i.e. behavioral and social. Upon designing the two script variations, we conducted a longitudinal, multiple-case study with ten groups of Media Informatics master students (n = 28, in groups of three or two, group=case, 4 sessions x70 min, Behavioural Awareness Script group= 5, Social Awareness Script group =5.) where each group was conceptualized as a case. Students collaborated every time for arguing to solve one different ill-structured problem and for transferring their arguments in the argument mapping tool Rationale. Thereby, we intended to investigate the effects of different awareness prompts on (a) collaborative metacognitive processes i.e., regulation, reflection, and evaluation (b) the relation between collaborative metacognitive processes and the quality of collaborative argumentation as well as (c) the impact of the two script variations on perceived team effectiveness and (d) what was experience with the different parts of the script variations in the two groups and how this fits into the design framework by Buder (2011). The quantitative analysis of argument outcomes from the groups yield no significant difference between the groups that worked with the BAS and the SAS variations. No significant difference between the script variations with respect to the results from the team effectiveness questionnaires was found either. Prompts for regulating collaboration processes were found to be the most successfully and consistently applied ones, especially in the most successful cases from both script variations and have influenced the argumentation outcomes. The awareness prompts afforded an explicit feedback display format (e.g. assessment of participation levels of self- and others) through discussion (Buder, 2011). The prompted explicit feedback display format (i.e., ratings of one’s self and of others) was criticized for running only on subjective awareness information on participation, contribution efforts and performance in the role. This resulted in evaluation apprehension phenomena (Cottrell, 1972) and evaluation bias (i.e., users may have not assessed themselves or others frankly) (Ghadirian et al., 2016). The awareness prompts for reflection and evaluation did reveal frictions in the plan making process (i.e., dropping out of the plan for collaboration) in the least successful groups. Problems with group dynamics (i.e., free-loading and presence of dominance) but were not powerful enough to trigger the desired changes in the behaviors of the students. The prompts for evaluating the collaboration in both script variations had no apparent connection to argumentation outcomes. The results indicated that dominant presence phenomena inhibited substantive argumentation in the least successful groups. They also indicated that the role-assignment influenced the group dynamics by helping student’s making clear the labor division in the group. In the second phase of development of the Argue(a)ware tool, the focus is on structuring and regulating social interactions in the relational space of collaborative argumentation by means of scripted roles and role-based awareness scaffolds. We designed support for mirroring participation in the role (i.e., a role-based awareness visualization) and support for monitoring participation, coordination and collaboration efforts in the role (i.e., self-assessment questionnaire). Moreover, we designed additional support for guiding participation in the role i.e., role-based reminders as notifications on smartwatches. In a between-subjects study, ten groups of three university students each (n = 30, Mage =22y, mixed educational backgrounds, 1x90min) worked with two variants of the Argue(a)ware for arguing to solve one ill-structured problem and transferring their arguments in the argument mapping tool Rationale. Next, to that, students should monitor their progress in their role with the role-based awareness visualization and the self-assessment questionnaire with the basic awareness support (role-based awareness visualization with the intermediate self-assessment) and the enhanced awareness support (additional role-based awareness reminders). Half of the groups worked only with the role-based awareness visualization and the self-assessment questionnaire (Basic Awareness Condition-BAC) while the other half groups received additional text-based awareness notifications via smartwatches that were sent to students privately (Enhanced Awareness Condition- EAC). Thereby, we tested the use of different degrees of awareness support in the two conditions with respect to their impact on a) self-perceived awareness of performance in the role and of collaboration and coordination efforts (measured with the same questionnaire at two time points), b) on perceive team effectiveness, c) group performance. We hypothesized that students in EAC will perform better thanks to the additional awareness reminders that increased the directivity and influenced their awareness in the role. The mixed methods analysis revealed that the awareness reminders, when perceived on time, succeeded in guiding collaboration (i.e., resulted in more role-specific behaviors). Students in the EAC condition improved their awareness over time (between the two measurements). These results indicated that enhanced awareness support in the form of additional guidance through awareness reminders can boost the awareness of students’ performance in the role as well as the awareness of their coordination and collaboration efforts over time by directing them back to the mirroring and monitoring tools. Moreover, students in EAC exhibited higher perceived team effectiveness than the students in BAC. However, no significant differences in building of shared mental models or performing in mutual performance monitoring were found between the groups. However, students in BAC and EAC did not differ significantly with respect to the formal correctness or evidence sufficiency of their group argumentation outcomes. Moreover, technical difficulties with the smartphones used as delivery devices for the awareness reminders (i.e., low vibration modus) hindered the timely perception of the reminders and thus their effect on participation. Finally, the questionnaire on the experience with the different parts of Argue(a)ware system indicated the need for exploring further media for supporting the awareness reminders to avoid the overwhelming effects of the multiple displays of the system and enhancing higher perceptiveness of the reminders with low interruption costs for other group members. The rather high satisfaction with the use of the role-based awareness visualization and the positive comments on the motivating aspects of monitoring how the personal success contributes to the group performance indicate that the group mirror succeeded in making group norms visible to group members in a non-obtrusive way. The high interpersonal comparability of performances without moderating the group ‘s interaction directly in the basic awareness condition was proven to be the favored design approach compared to the combination of group mirror and awareness reminders in the enhance awareness condition. In the third phase of development of Argue(a)ware, we focused on designing and testing different notification modes on different ubiquitous mobile devices for facilitating the next prototype of a notification system for role-based awareness reminders. Thereby, the aim of the system was again to guide students’ active participation in collaborative argumentation. More specifically, we focused on raising students’ attention to the reminders and triggering a prompter reaction to the contents of the reminders whilst avoiding a high interruption cost for the primary task (i.e., arguing for solving the problem at hand) in the group. These goals were translated into design challenges for the design of the role-based awareness notification system. The system should afford low interruptions, high reaction and high comprehension of notifications. Notification systems with this particular configuration of IRC values are known as "secondary display" systems (McCrickard et al., 2003). Next, we designed three low-fidelity prototypes for a role-based notification system for delivering awareness reminders: The first ran on a smartwatch and afforded text-based information with vibration and light notification modalities. The second ran on smartphone and afforded text-based information with vibrotactile and light-based notification modalities. Finally, the third prototype run on a smart-ring which afforded graphical- based (i.e. abstract light) information with and light and vibration notification modalities. To test the suitability of these prototypes for acting as “secondary display” systems, we conducted a within-subjects user study where three university students (n= 3, Mage=28, mixed educational background) argued for solving three different problem cases and producing an argument map in each of the three consecutive meetings (max 90min) in the Argue(a)ware instructional system. Students were assigned the roles of writer, corrector and devil`s advocate and were instructed to maintain the same role across the three meetings. In each meeting, students worked with a different role-based awareness notification prototype, where they received a notification indicating their balloon is not growing bigger after five minutes of not exhibiting any role-specific behaviors. The role-based awareness notification prototypes aimed at introducing timely interventions which would prompt students to check on their own progress in the role and the group progress as visualized by the role-based awareness visualization on the large display. Ultimately, this should prompt them to reflect on the awareness information from the visualization and adapt their behaviors to the desired behavior standards over time. Results showed that students perceived the notifications from all media mostly based on vibration cues. Thereby, the vibration cues on the wrist (smartwatch) were considered the least disruptive to the main task compared to the vibration cues on finger (smartwatch) and the vibration cues on the desk (smartphone). Students also declared that vibration cues on wrist prompted the fastest reaction i.e., attending to notification by interacting with the smartwatch. These results indicate that vibration cues on the wrist can be a suitable notification mechanism for increasing the perceived urgency of the message and prompting the reaction on it without causing great distraction to the main task, as studies previous studies showed before (Pielot, Church, & deOliveira, 2013; Hernández-Leo, Balestrini, Nieves & Blat, 2012). Based on very limited qualitative data on light as notification modality and awareness representation type no inferences could be made about its influence on the cost of interruption, reaction and comprehension parameters comprehensiveness. The qualitative and quantitative data on the experience with different media as awareness notification systems indicate that smartwatches may be the most suitable medium for acting as awareness notification medium with a “secondary display” IRC configuration (low-high-high). However, this inference needs to be tested in terms of a follow up study. In the next study, the great limitations of study (limited data due to low power and mal-structured measurement instruments) need to be repaired. Finally, the focus should be on comparing notification modalities of one medium (e.g., smartphone) based on a larger set of participants and with the use of objective measurements for the IRC parameter values (Chewar, McCrickard & Sutcliffe, 2004). Finally, we draw conclusions based on the findings from the three studies with respect to the role of awareness mechanisms for facilitating collaborative processes and outcomes and provide replicable and generalizable design principles. These principles are formed as heuristic statements and are subject to refinement by further research (Bell, Hoadley, & Linn, 2004; Van den Akker, 1999). We conclude with the limitations of the study and ideas for future work with Argue(a)ware

The impact of 'computer-supported collaborative inquiry' for science learning in secondary education

Social Sciences, Ghent Universitynrpages: 264status: publishe