How Design Science Research Helps Improve Learning Efficiency in Online Conversations

In this design science research paper, we report on our constructing and evaluating an attention-guidance system that we integrated into a computer-supported collaborative learning system. Drawing on social constructivist literature, our proposed design focuses on attracting, retaining, and, if necessary, reacquiring users’ attention on task-relevant information in online collaborative literature processing. The investigation involved an experiment across two sections of students in a human-computer interaction course. Results show that the new design allowed users to consistently reflect and evaluate the content of a text as they capitalized on one another’s reasoning to resolve misconceptions. Moreover, we found that the new system increased users’ perceptions of learning. However, the difference in knowledge gain scores was marginally significant and represented a medium effect size. Interestingly, we found that the attention-guidance system supported more efficient learning. Finally, we discovered that task-oriented reading of text, revisions of incomplete or incorrect ideas, and perceptions of learning mediated the relationship between software system and learning efficiency. We discuss the theoretical and practical implications

Integrating Learning Analytics to Measure Message Quality in Large Online Conversations

Research on computer-supported collaborative learning (CSCL) often employs content analysis as an approach to investigate message quality in asynchronous online discussions using systematic message-coding schemas. Although this approach helps researchers count the frequencies by which students engage in different socio-cognitive actions, it does not explain how students articulate their ideas in categorized messages. This study investigates the effects of a recommender system on the quality of students’ messages from voluminous discussions. We employ learning analytics to produce a quasi-quality index score for each message. Moreover, we examine the relationship between this score and the phases of a popular message-coding schema. Empirical findings show that a custom CSCL environment extended by a recommender system supports students to explore different viewpoints and modify interpretations with higher quasi-quality index scores than students assigned to the control software. Theoretical and practical implications are also discussed

The Virtual Tutor: Combining Conversational Agents with Learning Analytics to support Formative Assessment in Online Collaborative Learning

The objective of this design science research project is to combine Learning Analytics data with a conversational agent communication interface, the Virtual Tutor, which is able to support formative assessment for educators and learners in online collaborative learning (OCL) environments. The main benefit for educators is providing user-adaptable Learning Analytics data requests to fit the information needs for formative assessment. Learners receive semi-automated feedback on their platform activity in form of reports, which shall trigger self-reflection processes. By extracting requirements from the potential users and deriving design principles, a conversational agent is implemented and evaluated in an online collaborative learning course. The results indicate that the Virtual Tutor reduces the task load of educators, supports formative assessment and gives scaffolded guidance to the learners by reflecting their performance, thus triggering self-reflection processes. This research provides a first step towards data supported (semi-)automated feedback systems for formative assessment in OCL courses

Getting Around to It: How Design Science Researchers Set Future Work Agendas

Background: There is a long tradition of writing about future work in research papers, and information systems design science research (IS DSR) is no exception. However, there is a lack of studies on (1) how IS DSR authors currently envision the next steps for their work and (2) guidelines to improve the communication of opportunities to accumulate knowledge. Method: This paper contributes to this topic, building on a systematic literature review of 123 IS DSR papers published between 2018 and 2022. Results: Design-oriented research requires the research team to decide which tasks to carry out immediately in building the future and which to postpone as research debt. The paper\u27s contribution is threefold. First, we propose a research debt lifecycle to support (1) project stakeholders, (2) IS DSR community, and (3) societies looking for better futures. Second, we discuss the anatomy of future work in recent IS DSR. Finally, we suggest guidelines to manage and report the next research steps. Conclusion: This paper presents a pioneering assessment of future work suggestions in the IS field, focusing on the design science research paradigm. Future work directions emerge from researchers\u27 choices during the IS DSR process that must be continuously managed

Design science research in PhD education: designing for assistance tools

170 p.scholars. Major conferences have tracks dedicated to DSR, and even leading journals have publishedspecial issues on it. In line with this momentum, DSR has also gained acceptance among PhD students.Indeed, DSR is well regarded for its ability to bring together theoretical and practical knowledge,addressing both rigor and relevance. But in exchange, DSR calls for high levels of commitment andmaturity. PhD students, as they are transitioning towards becoming independent researchers, usually lacksuch maturity. On top of that, the lack of widely accepted software tools for conducting DSR does nothelp.This Thesis is aimed at providing PhD students with tool support for carrying out DSR. To thatend, we focus on problematic situations related to four basic activities conducted throughout thedoctorate: inquiry, reading, writing and peer review. For each of these problems, a purposeful artifact isdesigned, developed and evaluated with real stakeholders. The outcome: DScaffolding and Review&Go,two browser extensions for Google Chrome currently in use by practitioners

Collaboration in remote access laboratories

Collaboration in Remote Access Laboratories (RALs) is becoming increasingly important in both engineering and science education institutions, and with RALs service providers as an enabler to improving accessibility, reducing costs, and improving time-efficiency and student support. Yet research on the use of collaboration in RALs, in general, is limited. There is a lack of exploratory and empirical studies that provide an in-depth and holistic investigation of the design process and factors that influence the adoption of collaboration in RALs. Therefore, this study makes a significant and original contribution to current theoretical and practice knowledge with regards to pedagogical change in engineering education through the use of technology and remote access laboratories, where social constructivist practices are applied, in particular, engineering students undertaking LAB work in a different mode or approach to the traditional learning environment. This research employed a case study qualitative method with triangulation of data. Data were collected through observation of students working collaboratively in the trial of collaborative learning in RALs using the Voltage Divider Experiment task, and follow-up, in-depth interviews, with inductive analysis and activity recoding. The research explored Kagan’s PIES that relate to outcomes of the collaborative approach, Dillenbourg’s four elements of collaborative learning and Doolittle’s eleven principles of learning experience design as the theoretical bases of the collaborative pedagogical design of the RALs learning experience. While confirming their continued relevance to this context for learning three new principles were shown to be essential to facilitate and enhance contemporary learning in RALs. These included the need to build in the leadership of the collaborative learning experience, ensure task authenticity and participants acquisition of the soft skills, including interpersonal skills and teamwork, and their relevance to the workplace (employability). Additionally, this research highlighted how learning in RALs facilitates formative assessment that feeds forward to better support students’ learning where they need to communicate with each other during the LAB work collaborative learning experiences, thus drawing attention to the need for careful academic planning. The study also addressed the limitations of collaboration in RALs. It investigated the extent to which engineering students accepted collaborative learning in RALs as a workable alternative to traditional in-LAB work. It identified the key factors that are likely to influence the adoption of such pedagogical change, including factors to be considered when planning to adopt collaboration in RALs. This resulted in the development of an instructional framework for collaboration in RALs. It was concluded that collaboration in RALs has the potential to improve LAB learning through the availability of remote access, the facilitation of a sense of reality (comparable to traditional hands-on experience) and the opportunity for group work, and the need for skills that more closely related to those needed in students’ future workplaces