Multilevel analysis in CSCL Research

Janssen, J., Erkens, G., Kirschner, P. A., & Kanselaar, G. (2011). Multilevel analysis in CSCL research. In S. Puntambekar, G. Erkens, & C. Hmelo-Silver (Eds.), Analyzing interactions in CSCL: Methods, approaches and issues (pp. 187-205). New York: Springer. doi:10.1007/978-1-4419-7710-6_9CSCL researchers are often interested in the processes that unfold between learners in online learning environments and the outcomes that stem from these interactions. However, studying collaborative learning processes is not an easy task. Researchers have to make quite a few methodological decisions such as how to study the collaborative process itself (e.g., develop a coding scheme or a questionnaire), on the appropriate unit of analysis (e.g., the individual or the group), and which statistical technique to use (e.g., descriptive statistics, analysis of variance, correlation analysis). Recently, several researchers have turned to multilevel analysis (MLA) to answer their research questions (e.g., Cress, 2008; De Wever, Van Keer, Schellens, & Valcke, 2007; Dewiyanti, Brand-Gruwel, Jochems, & Broers, 2007; Schellens, Van Keer, & Valcke, 2005; Strijbos, Martens, Jochems, & Broers, 2004; Stylianou-Georgiou, Papanastasiou, & Puntambekar, chapter #). However, CSCL studies that apply MLA analysis still remain relatively scarce. Instead, many CSCL researchers continue to use ‘traditional’ statistical techniques (e.g., analysis of variance, regression analysis), although these techniques may not be appropriate for what is being studied. An important aim of this chapter is therefore to explain why MLA is often necessary to correctly answer the questions CSCL researchers address. Furthermore, we wish to highlight the consequences of failing to use MLA when this is called for, using data from our own studies

Collaborative trails in e-learning environments

This deliverable focuses on collaboration within groups of learners, and hence collaborative trails. We begin by reviewing the theoretical background to collaborative learning and looking at the kinds of support that computers can give to groups of learners working collaboratively, and then look more deeply at some of the issues in designing environments to support collaborative learning trails and at tools and techniques, including collaborative filtering, that can be used for analysing collaborative trails. We then review the state-of-the-art in supporting collaborative learning in three different areas – experimental academic systems, systems using mobile technology (which are also generally academic), and commercially available systems. The final part of the deliverable presents three scenarios that show where technology that supports groups working collaboratively and producing collaborative trails may be heading in the near future

Accessible collaborative learning environments for mobile devices

Mención Internacional en el título de doctorNew technologies and devices are being used in learning environments by teachers and students. Some of these tools are computer supported collaborative learning tools that help them collaborate with each other and share knowledge. Chat applications are one of these tools. These tools allow sharing materials and knowledge or solve doubts in real time without the necessity of being in the same room at the same time. Especially, these tools are being used in mobile devices which make collaboration more ubiquitous because people can use them everywhere. However, existing chat applications are not fully accessible and present accessibility barriers that users need to face every day. People with disabilities encounter these barriers every day despite of they have the same rights as people without disabilities according to multiple regulations in many countries around the World. These barriers might not be faced by people with disabilities only, people with disabilities who use mobile devices in different environments e.g. on the move or in bright environments can suffer similar problems as people with disabilities. This thesis aims to identify the accessibility barriers that m-learning chat applications have. Besides, considering these problems, this research aims, as far as possible, to improve the accessibility of chat applications. As a result, people with and without disabilities could collaborate with each other without facing accessibility barriers that will mermaid their learning. The main objectives of this thesis are: firstly, identify accessibility barriers that people with and without disabilities face when they use chat applications; secondly, specify the requirements that accessible m-learning chat applications should include for being accessible; and finally, provide an accessible interaction improvement for these applications. All these objectives have been achieved following a user centred design approach. As a result, more than 200 people with and without disabilities have participated in this thesis.Las tecnologías de la información se utilizan en entornos educativos para ayudar a los estudiantes y profesores a compartir y mejorar el aprendizaje. Algunas de estas herramientas permiten a los estudiantes compartir conocimiento y aprender colaborando entre sí, y se suelen denominar herramientas de aprendizaje colaborativas. Un ejemplo de herramienta colaborativa es la aplicación Chat. A través de estas aplicaciones, los profesores y estudiantes pueden compartir recursos y conocimiento o resolver dudas en tiempo real, sin la necesidad de encontrarse en la misma aula al mismo tiempo. Estas herramientas se utilizan hoy en día en dispositivos móviles que permiten realizar colaboraciones de forma ubicua, ya que se pueden utilizar desde cualquier lugar. Sin embargo, hoy en día las aplicaciones chats que existen en el mercado no son completamente accesibles, presentando barreras de accesibilidad que los usuarios tienen que sortear cada día. Las personas con discapacidad sufren estas barreras, a pesar de que están amparados por leyes de todo el mundo que especifican que tienen los mismos derechos que las personas sin discapacidad. Estas barreras de accesibilidad no son barreras que sólo personas con discapacidad pueden percibir, personas sin discapacidad pueden sufrir los mismos problemas cuando utilizan estas herramientas en dispositivos móviles, cuando se están desplazando o cuando utilizan los dispositivos en espacios abiertos con mucha luz. En esta tesis doctoral se pretende estudiar las barreras de accesibilidad que presentan las aplicaciones chat en entornos educativos con dispositivos móviles. De esta forma, se trata, en la medida de lo posible, de mejorar la accesibilidad de este tipo de aplicaciones. Como resultado, personas con discapacidad y sin discapacidad podrán colaborar entre sí, sin encontrar problemas de accesibilidad. Los tres objetivos principales de esta tesis son: primero, identificar los problemas que las personas con y sin discapacidad tienen cuando utilizan los chats; segundo, especificar los requisitos de accesibilidad que los chats deben incluir en entornos de aprendizaje utilizando dispositivos móviles; y finalmente, realizar una propuesta de mejora de accesibilidad de este tipo de aplicaciones. Todos estos objetivos se han alcanzado siguiendo para ello un diseño centrado en el usuario en el que se ha contado con la participación de más de 200 personas con y sin discapacidad para obtener cada una de las aportaciones resultado de los objetivos propuestos.Programa Oficial de Doctorado en Ciencia y Tecnología InformáticaPresidente: Covadonga Rodrigo San Juan.- Secretario: María Belén Ruiz Mezcua.- Vocal: Leonel Caseiro Morgad

A Computational Linguistic Analysis of Learners Discourse in Computer-Mediated Group Learning Environments

Communication, collaboration and the social co-construction of knowledge are now considered critical 21st century skills and have taken a principal role in recent theoretical and technological developments in education research. The overall objective of this dissertation was to investigate collaborative learning to gain insight on why some groups are more successful than others. In such discussions, group members naturally assume different roles. These roles emerge through participants’ interactions without any prior instruction or assignment. Different combinations of these roles can produce characteristically different group outcomes, being either less or more productive towards collective goals. However, there has been little research on how to automatically identify these roles and fuse the quality of the process of collaborative interactions with the learning outcome. A major goal of this dissertation is to develop a group communication analysis (GCA) framework, a novel methodology that applies automated computational linguistic techniques to the sequential interactions of online group communication. The GCA involves computing six distinct measures of participant discourse interaction and behavioral patterns and then clustering participants based on their profiles across these measures. The GCA was applied to several large collaborative learning datasets, and identified roles that exhibit distinct patterns in behavioral engagement style (i.e., active or passive, leading or following), contribution characteristics (i.e., providing new information or echoing given material), and social orientation. Through bootstrapping and replication analysis, the roles were found to generalize both within and across different collaborative interaction datasets, indicating that these roles are robust constructs. A multilevel analysis shows that the social roles are predictive of success, both for individual team members and for the overall group. Furthermore, the presence of specific roles within a team produce characteristically different outcomes; leading to specific hypotheses as to optimal group composition. Ideally, the developed analytical tools and findings of this dissertation will contribute to our understanding of how individuals learn together as a group and thereby advance the learning and discourse sciences. More broadly, GCA provides a framework to explore the intra- and inter-personal patterns indicative of the participants’ roles and the sociocognitive processes related to successful collaboration

‘We’re all in this toGather’ – A Virtual World for Improving Knowledge Exchange and Social Interaction for Digital Work

One drastic change that has been established in many organizations is the possibility of location-independent work. However, working remotely also creates distinct challenges that organizations must face. Thus, remote work could lead to a decrease in social interactions and therefore less implicit knowledge exchange in teams. However, informal conversations are crucial for building and maintaining team cohesion as well as experience transfer among employees. To address this problem, we apply a design science research approach to examine how a virtual world as a work environment could help to overcome those challenges within our research group. We designed a prototype of a virtual world that is based on knowledge gained from three design thinking workshops and tested it over four weeks in a real-world work case. Furthermore, we conducted 16 interviews with employees and present our initial findings of the effects on group awareness, social identity, IT identity, trust, and acceptance

BEHAVIORAL INTERDEPENDENCE IN PROJECT TEAM COLLABORATION: STUDY OF ENGINEERING STUDENTS’ COLLABORATIVE BEHAVIORS IN HIGH LEVELS OF INTERDEPENDENT TASK SETTINGS

In teamwork learning settings, tasks are often designed at varying levels of interdependence that requires students to complete the tasks by relying only on their team members sharing resources, knowledge, and skills. However, well-structured tasks do not always guarantee task-related collaborative behaviors will occur and are simply not adequate for us to understand the collaboration process and participants’ actual collaborative behaviors. To deepen our understanding of collaboration and explore how increased collaboration may be promoted in high-level interdependent task settings, this study uses behavioral interdependence as an analytical concept to describe and examine individual students’ actual behaviors as they worked collaboratively on an interdependently-structured engineering design project. Behavioral interdependence is “the amount of task-related interaction actually engaged in by group members in completing their work” (Wageman, 2001, p. 207). The concept of behavioral interdependence helps us to understand students’ task-related collaborative behaviors. However, this concept has received scarce attention in collaboration literature. This study was set in a context of college engineering students collaborating on an authentic design project. A descriptive, instrumental two-case study methodology was employed to respond to two main research questions: (1) what individual behaviors are observed in project teams when students were working under the high task interdependence condition and (2) what patterns of team behaviors are observed in such a condition. After examining and comparing two newly-formed college student project teams’ collaborative behaviors in solving an interdependently-structured engineering design project, answers to the research questions help explore how team behavioral patterns formed out of, or were affected by, students’ individual behaviors and how behaviors affected team collaboration and performance. This study resulted in rich descriptions of individual student behaviors and behavior changes, team behaviors and behavior changes, and how individual behaviors were related to team behaviors and overall team collaboration and performance. Results suggested that (1) individual behaviors were closely associated with team behaviors, collaboration, and performance, (2) students’ early behavioral patterns largely predicted their continuous behaviors, (3) urgent deadlines were likely to change behaviors of students who had poor performance in task management and temporal planning, (4) individuals performing better in disciplinary, technical areas tended to have more contribution to and better participation in teamwork, and (5) teams with high levels of behavioral interdependence tended to have better performance in teamwork. Several recommendations are provided for designing instruction in high interdependent task settings such as careful estimation of task completion time considering students’ varying collaboration skills and time management ability levels (task / activity design recommendation), providing suitable scaffolding strategies to support students who are not adequate in technical fields or in skills in areas of self-management, effective communication, and temporal planning (activity preparation recommendation), and paying attention to students’ behaviors at the early stage of their collaboration and providing timely corrective feedback (formative evaluation recommendations)