The Big Five:Addressing Recurrent Multimodal Learning Data Challenges

The analysis of multimodal data in learning is a growing field of research, which has led to the development of different analytics solutions. However, there is no standardised approach to handle multimodal data. In this paper, we describe and outline a solution for five recurrent challenges in the analysis of multimodal data: the data collection, storing, annotation, processing and exploitation. For each of these challenges, we envision possible solutions. The prototypes for some of the proposed solutions will be discussed during the Multimodal Challenge of the fourth Learning Analytics & Knowledge Hackathon, a two-day hands-on workshop in which the authors will open up the prototypes for trials, validation and feedback

Multimodal Challenge: Analytics Beyond User-computer Interaction Data

This contribution describes one the challenges explored in the Fourth LAK Hackathon. This challenge aims at shifting the focus from learning situations which can be easily traced through user-computer interactions data and concentrate more on user-world interactions events, typical of co-located and practice-based learning experiences. This mission, pursued by the multimodal learning analytics (MMLA) community, seeks to bridge the gap between digital and physical learning spaces. The “multimodal” approach consists in combining learners’ motoric actions with physiological responses and data about the learning contexts. These data can be collected through multiple wearable sensors and Internet of Things (IoT) devices. This Hackathon table will confront with three main challenges arising from the analysis and valorisation of multimodal datasets: 1) the data collection and storing, 2) the data annotation, 3) the data processing and exploitation. Some research questions which will be considered in this Hackathon challenge are the following: how to process the raw sensor data streams and extract relevant features? which data mining and machine learning techniques can be applied? how can we compare two action recordings? How to combine sensor data with Experience API (xAPI)? what are meaningful visualisations for these data

eCollaboration in der Hochschullehre: Bewertung mittels Learning Analytics

Wissen bekommt in unserer Gesellschaft eine immer stärkere Bedeutung und stellt Individuen und Organisationen vor verschiedenste Herausforderungen. Der gesamtwirtschaftliche Anteil materieller Güter wird gegenüber wissensintensiven Dienstleistungen zurückgehen, d.h. es bedarf zukünftig immer stärker gut aus- und weitergebildeter Experten, die miteinander die Innovationskraft von Organisationen steigern und kundenspezifische Lösungen entwickeln. Ausgehend von den für Experten benötigten Kompetenzen sogenannter Wissensarbeiter fokussiert diese Arbeit die Begleitung und Bewertung von kollaborativer Teamarbeit. Als Forschungsobjekt dienen Virtual Collaborative Learning-Veranstaltungen, in denen die zukünftigen Experten als Lernende teilnehmen. Sie werden in virtuellen Klassenräumen von Lernbegleitern beobachtet und bewertet, um sie bei der Entwicklung neuer Kompetenzen zu unterstützen und so die Erreichung der Lernziele zu gewährleisten. Da die Lernbegleitung bislang manuell durchgeführt wird, ist eine zeitnahe Beurteilung und Intervention nicht möglich. Mit Learning Analytics sollen Lösungen vorgeschlagen werden, die die Lernbegleiter in ihrer Arbeit unterstützen und den Aufwand reduzieren. Hierzu thematisiert diese Dissertation in sechs kumulativen Beiträgen, wie die zugrundeliegenden Daten verfügbar gemacht werden können und welche Beobachtungen anhand welcher Kriterien mittels ausgewählter Methoden der Learning Analytics durchgeführt werden sollten. Im Rahmen des Design Science Paradigmas werden verschiedene qualitative und quantitative Forschungsmethoden zur Datenerhebung und -auswertung angewendet. Im Ergebnis entsteht ein beispielhafter Katalog von Learning Analytics Methoden, die sich auf spezifische Erkenntnisziele der Beobachtungen von kollaborativer Gruppenarbeit beziehen. Ebenso wird der Einsatz einer ausgewählten Methode evaluiert. Die gewonnenen Erkenntnisse können zukünftig auf vergleichbare kollaborative Lehrangebote in der Ausbildung von Wissensarbeitern, aber auch zur Analyse und Unterstützung der virtuellen Zusammenarbeit im geschäftlichen Umfeld transferiert werden

Analytics-based approach to the study of learning networks in digital education settings

Investigating howgroups communicate, build knowledge and expertise, reach consensus or collaboratively solve complex problems, became one of the main foci of contemporary research in learning and social sciences. Emerging models of communication and empowerment of networks as a form of social organization further reshaped practice and pedagogy of online education, bringing research on learning networks into the mainstream of educational and social science research. In such conditions, massive open online courses (MOOCs) emerged as one of the promising approaches to facilitating learning in networked settings and shifting education towards more open and lifelong learning. Nevertheless, this most recent educational turn highlights the importance of understanding social and technological (i.e., material) factors as mutually interdependent, challenging the existing forms of pedagogy and practice of assessment for learning in online environments. On the other hand, the main focus of the contemporary research on networked learning is primarily oriented towards retrospective analysis of learning networks and informing design of future tasks and recommendations for learning. Although providing invaluable insights for understanding learning in networked settings, the nature of commonly applied approaches does not necessarily allow for providing means for understanding learning as it unfolds. In that sense, learning analytics, as a multidisciplinary research field, presents a complementary research strand to the contemporary research on learning networks. Providing theory-driven and analytics-based methods that would allow for comprehensive assessment of complex learning skills, learning analytics positions itself either as the end point or a part of the pedagogy of learning in networked settings. The thesis contributes to the development of learning analytics-based research in studying learning networks that emerge fromthe context of learning with MOOCs. Being rooted in the well-established evidence-centered design assessment framework, the thesis develops a conceptual analytics-based model that provides means for understanding learning networks from both individual and network levels. The proposed model provides a theory-driven conceptualization of the main constructs, along with their mutual relationships, necessary for studying learning networks. Specifically, to provide comprehensive understanding of learning networks, it is necessary to account for structure of learner interactions, discourse generated in the learning process, and dynamics of structural and discourse properties. These three elements – structure, discourse, and dynamics – should be observed as mutually dependent, taking into account learners’ personal interests, motivation, behavior, and contextual factors that determine the environment in which a specific learning network develops. The thesis also offers an operationalization of the constructs identified in the model with the aim at providing learning analytics-methods for the implementation of assessment for learning. In so doing, I offered a redefinition of the existing educational framework that defines learner engagement in order to account for specific aspects of learning networks emerging from learning with MOOCs. Finally, throughout the empirical work presented in five peer-reviewed studies, the thesis provides an evaluation of the proposed model and introduces novel learning analytics methods that provide different perspectives for understanding learning networks. The empirical work also provides significant theoretical and methodological contributions for research and practice in the context of learning networks emerging from learning with MOOCs

Eye on Collaborative Creativity : Insights From Multiple-Person Mobile Gaze Tracking in the Context of Collaborative Design

Early Career WorkshopNon peer reviewe

The student-produced electronic portfolio in craft education

The authors studied primary school students’ experiences of using an electronic portfolio in their craft education over four years. A stimulated recall interview was applied to collect user experiences and qualitative content analysis to analyse the collected data. The results indicate that the electronic portfolio was experienced as a multipurpose tool to support learning. It makes the learning process visible and in that way helps focus on and improves the quality of learning. © ISLS.Peer reviewe

Designing Tools for Reflection: a concept-driven approach

We are surrounded by objects. We often use and interact with them to do our daily activities. They do not only support us and augment our abilities, but also, can be considered as companions of our thoughts. We think with objects, because they contain information about us: about our memories, experiences, emotions, and activities as Sherry Turkle highlights (2011). Furthermore, our everyday objects are increasingly computed, smart and connected to the Internet. They are able to collect data, elaborate and provide real-time feedbacks. These feedbacks cannot only support us to improve our activities, but also enables critical thinking and reflection on our actions. This resonates very well with what Donald Schön meant by having reflective conversation with materials at hand (1983;1996). He highlighted that materials –artifacts– of a situation talk back to designer, so they enable and support reflection in action of designing. So, how about if we consider that our daily objects can talk back and make us think on our actions in order to consider alternatives? This dissertation, is an attempt to consider this opportunity. The nature of this dissertation is mostly conceptual and its scope is defining the physical and behavioral characteristics of smart artifacts able to provoke thoughts and reflection in user leading to a conscious behavior change. I sought to use existing theories about reflective thinking in HCI and beyond, as valuable sources for developing design concept. I have been inspired by the Concept-Driven interaction design research (Stolterman and Wiberg 2011) and created and defined the whole structure of this dissertation based on this methodology, from the definition of the concept – Tool for Reflection – to the construction of a theoretical model from the design outcome –Make Me Think model. During this process, I used different methods such as conducting literature analysis, context analysis, survey, participatory session and prototyping. The sustainable urban mobility behaviors in the city of Turin (Italy) as the target behavior and home as the place for using Tool for Reflection have been chosen for this research. In particular, informed by architectural studies, I conceptualized In-Between Places as a category of places that connect home places to city places. I suggested to consider such areas as suitable places for evoking thoughts on urban mobility behaviors, in home. This dissertation provides a theoretical perspective with which to guide the design of smart objects that evoke reflection. It first provides a set of characteristics of a Tool for Reflection as a physical artifact. Then it provides a theoretical model, considering the relationship between a Tool for Reflection and a user. The key contributions include the design of the Sóle, a smart lamp, not only as an example of a Tool for Reflection with its theoretically pre-defined characteristics, but also as an instrument for iterating from design to the theory. The overall approach, the methodology and the findings should be of interest in particular to researchers working on design for reflection in the HCI. More broadly this dissertation can be of interest of researchers in the HCI, whose research is around designing artifacts, both as an ‘outcome’ and as an ‘instrument’ of the research process

COGNITIVE PRESENCE IN PEER FACILITATED ASYNCHRONOUS ONLINE DISCUSSION: THE PATTERNS AND HOW TO FACILITATE

This study, in the context of peer-facilitated asynchronous online discussion, explored the characteristics and patterns of students’ cognitive presence, and examined the practices that aim to enhance cognitive presence development. Participants were 53 students from a graduate-level online course that focused on the integration of educational technologies. Data were collected from discussion transcripts, student survey, student artifacts, and researcher’s observations. Results demonstrated four phases of students’ cognitive presence: Triggering event, Exploration, Integration, and Resolution. Among the four phases, students’ cognitive presence tended to aggregate at the middle phases: Integration and Exploration. Percentage of the Resolution was very low. The distribution of students’ discussion behaviors further revealed: a) the hierarchical relationship between the four phases: Integration and Resolution involved a higher-level of cognitive engagement, and Triggering event and Exploration involved a lower-level of cognitive engagement; b) the phase of Resolution heavily relied on experiment, while the other three phases heavily relied on making use of personal experience; c) creating of cognitive presence occurred in both the private space of individual activities and the shared space of having dialogues. The conversation analysis of threads and episodes explored the temporal evolvement of cognitive presence. The results showed that, in an ongoing discussion, students’ cognitive presence evolved in a non-linear way, rather than strictly phase by phase as suggested by the PI model. Experiments were designed and conducted to determine the effects of two pedagogical interventions – 1) providing guidance on peer facilitation techniques; 2) asking students to label their posts. The results showed that the Intervention 1 and the combination of two interventions credibly improved students’ cognitive presence. They were especially effective in improving Integration, a higher level of cognitive presence. After having added Intervention 2, cognitive presence increased from the first-half to the second-half semester, although the improvement was not found to be statistically credible. This study confirmed the close association between and among cognitive presence, social interaction, and peer facilitation. The results clearly showed that Intervention 1 – providing guidance on peer facilitation credibly improved students’ social interaction and peer facilitation. However, Mixed findings were obtained for Intervention 2 – asking students to label their posts. It was found that Intervention 2 positively increased students’ social interaction. However, it did not show any impact on students’ peer facilitation behaviors. It is also worth noting that the effect of the combination of two interventions was much larger than any single one of them. Conversation analysis was conducted to zoom in on the dynamic process of discussion. The cases revealed that when students were provided with the guidance on peer facilitation techniques, they tended to use a variety of facilitation techniques in a strategic way to help peers to achieve a sustained and deeper-level conversation. Compared to the control group, the students in the treatment group showed more peer facilitation behaviors, which led to more conversations and more higher-level cognitive presence. This study has unpacked the complexity of students’ cognitive presence in a peer-facilitated discussion environment, especially when students are coached in performing teaching presence. The results shed light on the pedagogical practices and strategies of creating an online learning community that incubates rich cognitive presence. Finally, implications are discussed for the research and practices in online instruction and discussion analytics