Layered evaluation of interactive adaptive systems : framework and formative methods

Peer reviewedPostprin

Enhancing knowledge management in online collaborative learning

This study aims to explore two crucial aspects of collaborative work and learning: on the one hand, the importance of enabling collaborative learning applications to capture and structure the information generated by group activity and, on the other hand, to extract the relevant knowledge in order to provide learners and tutors with efficient awareness, feedback and support as regards group performance and collaboration. To this end, in this paper we first propose a conceptual model for data analysis and management that identifies and classifies the many kinds of indicators that describe collaboration and learning into high-level aspects of collaboration. Then, we provide a computational platform that, at a first step, collects and classifies both the event information generated asynchronously from the users' actions and the labeled dialogues from the synchronous collaboration according to these indicators. This information is then analyzed in next steps to eventually extract and present to participants the relevant knowledge about the collaboration. The ultimate aim of this platform is to efficiently embed information and knowledge into collaborative learning applications. We eventually suggest a generalization of our approach to be used in diverse collaborative learning situations and domains

Textual Mediation in Simulated Nursing Handoffs: Examining How Student Writing Coordinates Action

In clinical nursing simulations, a group of students provide care for a robotic patient during a structured scenario. As care is transferred from one group to another, they participate in a patient handoff, with outgoing students passing key information onto incoming students. In healthcare, the nursing handoff is a critical and perilous communication moment that is mediated by a range of participants and texts. Drawing on observations and video recordings of 52 simulation handoffs in the United States, this article examines how two student-designed texts – a collaborative patient chart and individual notes – are leveraged during the handoff. I also consider how handoff talk and writing changes as student nursing knowledge increases over the course of a year. By focusing on textual mediation of the simulated nursing handoff, this article contributes to existing research on professional writing pedagogy and to nursing scholarship on the handoff. Ultimately, it argues that a textual mediation framework can help bridge class room and professional contexts by evaluating student writing not for how successfully it meets a set of imposed criteria but for how effectively it supports classroom activities

Systemic intervention for computer-supported collaborative learning

This paper presents a systemic intervention approach as a means to overcome the methodological challenges involved in research into computer-supported collaborative learning applied to the promotion of mathematical problem-solving (CSCL-MPS) skills in schools. These challenges include how to develop an integrated analysis of several aspects of the learning process; and how to reflect on learning purposes, the context of application and participants' identities. The focus of systemic intervention is on processes for thinking through whose views and what issues and values should be considered pertinent in an analysis. Systemic intervention also advocates mixing methods from different traditions to address the purposes of multiple stakeholders. Consequently, a design for CSCL-MPS research is presented that includes several methods. This methodological design is used to analyse and reflect upon both a CSCL-MPS project with Colombian schools, and the identities of the participants in that project

Gaming techniques and the product development process : commonalities and cross-applications

The use of computer-based tools is now firmly embedded within the product development process, providing a wide range of uses from visualisation to analysis. However, the specialisation required to make effective use of these tools has led to the compartmentalisation of expertise in design teams, resulting in communication problems between individual members. This paper therefore considers how computer gaming techniques and strategies could be used to enhance communication and group design activities throughout the product design process

Involving Users to Improve the Collaborative Logical Framework

In order to support collaboration in web-based learning, there is a need for an intelligent support that facilitates its management during the design, development, and analysis of the collaborative learning experience and supports both students and instructors. At aDeNu research group we have proposed the Collaborative Logical Framework (CLF) to create effective scenarios that support learning through interaction, exploration, discussion, and collaborative knowledge construction. This approach draws on artificial intelligence techniques to support and foster an effective involvement of students to collaborate. At the same time, the instructors’ workload is reduced as some of their tasks—especially those related to the monitoring of the students behavior—are automated. After introducing the CLF approach, in this paper, we present two formative evaluations with users carried out to improve the design of this collaborative tool and thus enrich the personalized support provided. In the first one, we analyze, following the layered evaluation approach, the results of an observational study with 56 participants. In the second one, we tested the infrastructure to gather emotional data when carrying out another observational study with 17 participants

Responsible research and innovation in science education: insights from evaluating the impact of using digital media and arts-based methods on RRI values

The European Commission policy approach of Responsible Research and Innovation (RRI) is gaining momentum in European research planning and development as a strategy to align scientific and technological progress with socially desirable and acceptable ends. One of the RRI agendas is science education, aiming to foster future generations' acquisition of skills and values needed to engage in society responsibly. To this end, it is argued that RRI-based science education can benefit from more interdisciplinary methods such as those based on arts and digital technologies. However, the evidence existing on the impact of science education activities using digital media and arts-based methods on RRI values remains underexplored. This article comparatively reviews previous evidence on the evaluation of these activities, from primary to higher education, to examine whether and how RRI-related learning outcomes are evaluated and how these activities impact on students' learning. Forty academic publications were selected and its content analysed according to five RRI values: creative and critical thinking, engagement, inclusiveness, gender equality and integration of ethical issues. When evaluating the impact of digital and arts-based methods in science education activities, creative and critical thinking, engagement and partly inclusiveness are the RRI values mainly addressed. In contrast, gender equality and ethics integration are neglected. Digital-based methods seem to be more focused on students' questioning and inquiry skills, whereas those using arts often examine imagination, curiosity and autonomy. Differences in the evaluation focus between studies on digital media and those on arts partly explain differences in their impact on RRI values, but also result in non-documented outcomes and undermine their potential. Further developments in interdisciplinary approaches to science education following the RRI policy agenda should reinforce the design of the activities as well as procedural aspects of the evaluation research

Neural Graph Collaborative Filtering

Learning vector representations (aka. embeddings) of users and items lies at the core of modern recommender systems. Ranging from early matrix factorization to recently emerged deep learning based methods, existing efforts typically obtain a user's (or an item's) embedding by mapping from pre-existing features that describe the user (or the item), such as ID and attributes. We argue that an inherent drawback of such methods is that, the collaborative signal, which is latent in user-item interactions, is not encoded in the embedding process. As such, the resultant embeddings may not be sufficient to capture the collaborative filtering effect. In this work, we propose to integrate the user-item interactions -- more specifically the bipartite graph structure -- into the embedding process. We develop a new recommendation framework Neural Graph Collaborative Filtering (NGCF), which exploits the user-item graph structure by propagating embeddings on it. This leads to the expressive modeling of high-order connectivity in user-item graph, effectively injecting the collaborative signal into the embedding process in an explicit manner. We conduct extensive experiments on three public benchmarks, demonstrating significant improvements over several state-of-the-art models like HOP-Rec and Collaborative Memory Network. Further analysis verifies the importance of embedding propagation for learning better user and item representations, justifying the rationality and effectiveness of NGCF. Codes are available at https://github.com/xiangwang1223/neural_graph_collaborative_filtering.Comment: SIGIR 2019; the latest version of NGCF paper, which is distinct from the version published in ACM Digital Librar

Exploring participatory design for SNS-based AEH systems

The rapidly emerging and growing social networking sites (SNS) offer an opportunity to improve adaptive e-learning experience by introducing a social dimension, connecting users within the system. Making connections and providing communication tools can engage students in creating effective learning environment and enriching learning experiences. Researchers have been working on introducing SNS features into adaptive educational hypermedia systems. The next stage research is centered on how to enhance SNS facilities of AEH systems, in order to engage students’ participation in collaborative learning and generating and enriching learning materials. Students are the core participants in the adaptive e-learning process, so it is essential for the system designers to consider students’ opinions. This paper aims at exploring how to apply participatory design methodology in the early stage of the SNS-based AEH system design process