Multilevel analysis in CSCL Research

Janssen, J., Erkens, G., Kirschner, P. A., & Kanselaar, G. (2011). Multilevel analysis in CSCL research. In S. Puntambekar, G. Erkens, & C. Hmelo-Silver (Eds.), Analyzing interactions in CSCL: Methods, approaches and issues (pp. 187-205). New York: Springer. doi:10.1007/978-1-4419-7710-6_9CSCL researchers are often interested in the processes that unfold between learners in online learning environments and the outcomes that stem from these interactions. However, studying collaborative learning processes is not an easy task. Researchers have to make quite a few methodological decisions such as how to study the collaborative process itself (e.g., develop a coding scheme or a questionnaire), on the appropriate unit of analysis (e.g., the individual or the group), and which statistical technique to use (e.g., descriptive statistics, analysis of variance, correlation analysis). Recently, several researchers have turned to multilevel analysis (MLA) to answer their research questions (e.g., Cress, 2008; De Wever, Van Keer, Schellens, & Valcke, 2007; Dewiyanti, Brand-Gruwel, Jochems, & Broers, 2007; Schellens, Van Keer, & Valcke, 2005; Strijbos, Martens, Jochems, & Broers, 2004; Stylianou-Georgiou, Papanastasiou, & Puntambekar, chapter #). However, CSCL studies that apply MLA analysis still remain relatively scarce. Instead, many CSCL researchers continue to use ‘traditional’ statistical techniques (e.g., analysis of variance, regression analysis), although these techniques may not be appropriate for what is being studied. An important aim of this chapter is therefore to explain why MLA is often necessary to correctly answer the questions CSCL researchers address. Furthermore, we wish to highlight the consequences of failing to use MLA when this is called for, using data from our own studies

Assessment of e-learners' temporal patterns in an online collaborative writing task

E-learners are generally adults with work and family constraints who get involved in the virtual campus looking for temporary academic flexibility. However, they are often confronted with collaborative learning activities which lead to additional organizational efforts by reducing their individual time flexibility. In this paper, we argue that time is a major variable in Computer Supported Collaborative Learning (CSCL) activities, and that assessing students' use of time in these situations can help educational designers to propose adequate time scripting to plan these educational activities.This case study presents an exploratory analysis of time patterns for 15 groups of students (n=66), involved in a collaborative writing task. The results reveal that (a), e-learners' time-on-task increased since the beginning of the activity, (b), they work more during week days than during weekends and (c), they tend to work during "conventional" hours of the day. The identification of these patterns is the first step toward the development of new methodologies and computer-supported tools to enhance organisation of time and social aspects in CSCL

Fostering complex learning-task performance through scripting student use of computer supported representational tools

This study investigated whether scripting student use of computer supported representational tools fostered students’ collaborative performance of a complex business-economics problem. Scripting the problem-solving process sequenced and made its phase-related part-task demands explicit, namely (1) determining core concepts, (2) proposing multiple solutions, and (3) coming to a final solution. The representational tools facilitated students in constructing specific representations of the domain (i.e., conceptual, causal, or mathematical) and were each suited for carrying out the part-task demands of a specific phase. Student groups in four experimental conditions had to carry out all part-tasks in a predefined order, but differed in the representational tool(s) they received during their collaborative problem-solving process. In three mismatch conditions, student groups received either a conceptual, causal, or simulation representational tool which supported them in only carrying out one of the three part-tasks. In the match condition, student groups received the three representational tools in the specified order, each matching the part-task demands of a specific problem phase. The results revealed that student groups in the match condition constructed more task-appropriate representations and had more elaborated and meaningful discussions about the domain. As a consequence, those student groups performed better on the complex learning-task. However, similar results were obtained by student groups who only received a representational tool for constructing causal representations for all part-tasks

Fostering complex learning-task performance through scripting student use of computer supported representational tools

Slof, B., Erkens, G., Kirschner, P. A., Janssen, J., & Phielix, C. (2010). Fostering complex learning-task performance through scripting student use of computer supported representational tools. Computers & Education, 55(4), 1707-1720.This study investigated whether scripting student use of computer supported representational tools fostered students’ collaborative performance of a complex business-economics problem. Scripting the problem-solving process sequenced and made its phase-related part-task demands explicit, namely (1) determining core concepts, (2) proposing multiple solutions, and (3) coming to a final solution. The representational tools facilitated students in constructing specific representations of the domain (i.e., conceptual, causal, or mathematical) and were each suited for carrying out the part-task demands of a specific phase. Student groups in four experimental conditions had to carry out all part-tasks in a predefined order, but differed in the representational tool(s) they received during their collaborative problem-solving process. In three mismatch conditions, student groups received either a conceptual, causal, or simulation representational tool which supported them in only carrying out one of the three part-tasks. In the match condition, student groups received the three representational tools in the specified order, each matching the part-task demands of a specific problem phase. The results revealed that student groups in the match condition constructed more task-appropriate representations and had more elaborated and meaningful discussions about the domain. As a consequence, those student groups performed better on the complex learning-task. However, similar results were obtained by student groups who only received a representational tool for constructing causal representations for all part-tasks

Guiding students’ online complex learning-task behavior through representational scripting

Slof, B., Erkens, G., Kirschner, P. A., Jaspers, J. G. M., & Janssen, J. (2010). Guiding students’ online complex learning-task behavior through representational scripting. Computers in Human Behavior, 26(5), 927-939. doi:10.1016/j.chb.2010.02.2007This study investigated the effects of representational scripting on students’ collaborative performance of a complex business-economics problem. The scripting structured the learning-task into three part-tasks, namely (1) determining core concepts and relating them to the problem, (2) proposing multiple solutions to the problem, and (3) coming to a final solution to the problem. Each provided representation (i.e., conceptual, causal, or simulation) was suited for carrying out a specific part-task. It was hypothesized that providing part-task congruent support would guide student interaction towards better learning-task performance. Groups in four experimental conditions had to carry out the part-tasks in a predefined order, but differed in the representation they received. In three mismatch conditions, groups only received one of the representations and were, thus, only supported in carrying out one of the part-tasks. In the match condition, groups received all three representations in the specified order (i.e., representational scripting). The results indicate that groups in the match condition had more elaborated discussions about the content of the knowledge domain (i.e., concepts, solutions and relations) and were better able to share and to negotiate about their knowledge. As a consequence, these groups performed better on the learning-task. However, these differences were not obtained for groups receiving only a causal representation of the domain

Open(ed) Classroom – Who cares?

Social Media werden zunehmend auch im Rahmen formaler Lernsettings eingesetzt. Teilweise werden hierbei bereits die Möglichkeiten von Social Media zur Öffnung von Lernsettings über die ursprünglichen Adressaten hinaus genutzt. Durch diese Öffnung von Lernräumen entstehen neue Anreize zum sozialen Austausch. Doch was geschieht, wenn immer mehr Individuen am Austausch teilhaben (können)? Werden die neuen potentiellen Austauschpartner wahrgenommen? Und bedeuten «mehr Austauschpartner» gleichzeitig auch grösseren Lernerfolg? Um Antworten auf diese Fragen zu finden, wurde eine theoretisch hergeleitete Wahrnehmungstaxonomie entwickelt, welche in einem ersten Setting überprüft wurde. Die Ergebnisse der Untersuchung geben Hinweise darauf, dass Lernende in geöffneten Lernsettings die neuen potentiellen Austauschpartner wahrnehmen. Die weiteren Dimensionen der Taxonomie, Perspektive und Group Awareness, tragen ebenfalls signifikant zum Ergebnis bei. Dieser Artikel skizziert die theoretische Herleitung der Wahrnehmungstaxonomie, fasst die zentralen Untersuchungsergebnisse zusammen und gibt Impulse für Folgestudien.Social media have become an integral part of formal learning settings. To some extent the possibilities of social media for opening (academic) classrooms are already used. Through this opening of learning spaces new incentives for social exchange arise. But what happens when more and more individuals can partake in the social exchange? Are these new exchange partners perceived? And implies «more exchange partners» greater learning success at the same time? To find answers to these questions, a theoretically derived awareness taxonomy was developed, which was tested in a first setting. The results of the study provide evidence that learners in open learning settings perceive the new exchange partners. The other dimensions of the taxonomy, perspective and group awareness, also contribute significantly to the findings. This article outlines the theoretical derivation of an awareness taxonomy, summarizes the key findings and provides impetus for further studies

Social Metacognition in Computer Supported Collaborative Learning Environments: A Comparison in an Online and Face-To-Face Information Literacy Course

Because of the advances in technology for education, online learning has become more prominent, especially in higher education. Computer-supported collaborative learning (CSCL) seems especially promising in allowing students to work together in ways that they have not been able to before, both face-to-face and online. Instructors use CSCL to engage students and to increase learning. CSCL requires that students regulate each other’s learning through social metacognition; this allows the group as a whole to make use of the knowledge and skills of the group as they learn. The purpose of this study was to compare the social metacognition of students in a CSCL environment for a face-to-face information literacy course and an online information literacy course. This allowed for the development of the Social Metacognitive Awareness Instrument (SMAI), which may be used by future researchers. When accounting for individual metacognitive scores, students in the face-to-face version of the course had significantly higher social metacognitive awareness scores than students in the online version of the course. This study also found that students in groups had some similarities in social metacognitive scores. A student’s metacognitive score was a significant predictor of their social metacognitive score. The results of this study indicate that more intervention may be needed for effective group work online. This also supports the research that social metacognition is an independent construct, and so social metacognition or socially-shared regulation should continue to be studied as an important factor in group work. The study also supports the research that indicates that individual metacognition can predict social metacognition. Thus, it is possible that improving individual metacognitive abilities will improve social metacognitive abilities

The impact of 'computer-supported collaborative inquiry' for science learning in secondary education

Social Sciences, Ghent Universitynrpages: 264status: publishe