Using Learning Analytics to Devise Interactive Personalised Nudges for Active Video Watching

Videos can be a powerful medium for acquiring soft skills, where learning requires contextualisation in personal experience and ability to see different perspectives. However, to learn effectively while watching videos, students need to actively engage with video content. We implemented interactive notetaking during video watching in an active video watching system (AVW) as a means to encourage engagement. This paper proposes a systematic approach to utilise learning analytics for the introduction of adaptive intervention - a choice architecture for personalised nudges in the AVW to extend learning. A user study was conducted and used as an illustration. By characterising clusters derived from user profiles, we identify different styles of engagement, such as parochial learning, habitual video watching, and self-regulated learning (which is the target ideal behaviour). To find opportunities for interventions, interaction traces in the AVW were used to identify video intervals with high user interest and relevant behaviour patterns that indicate when nudges may be triggered. A prediction model was developed to identify comments that are likely to have high social value, and can be used as examples in nudges. A framework for interactive personalised nudges was then conceptualised for the case study

Investigating Engagement and Learning Differences between Native and EFL students in Active Video Watching

Video-based learning (VBL) requires good listening and reading comprehension skills, which could be challenging for English as a foreign language (EFL) students. In this paper, we investigate the differences between EFL and Native English speakers in a VBL platform called AVW-Space, in order to identify potential interventions that would be helpful for EFL students. AVW-Space provides note-taking, peer-reviewing, visualisations and personalised nudges to support engagement in VBL. Although previous studies on AVW-Space showed these supports were effective for increasing engagement, we discovered significant differences in learning outcomes and engagement between EFL/Native students, which stem from different learning strategies, background knowledge and language barriers. This research contributes to using learning analytics to understand better the differences between EFL and Native students, and providing more specialised support for EFL students in VBL

Using Network-Text Analysis to Characterise Learner Engagement in Active Video Watching

Video is becoming more and more popular as a learning medium in a variety of educational settings, ranging from flipped classrooms to MOOCs to informal learning. The prevailing educational usage of videos is based on watching prepared videos, which calls for accompanying video usage with activities to promote constructive learning. In the Active Video Watching (AVW) approach, learner engagement during video watching is induced via interactive notetaking, similar to video commenting in social video-sharing platforms. This coincides with the JuxtaLearn practice, in which student-created videos were shared on a social networking platform and commented by other students. Drawing on the experience of both AVW and JuxtaLearn, we combine and refine analysis techniques to characterise learner engagement. The approach draws on network-text analysis of learner-generated comments as a basis. This allows for capturing pedagogically relevant aspects of divergence, convergence and (dis-) continuity in textual commenting behaviour related to different learner types. The lexical-semantic analytics approach using learner-generated artefacts provides deep insights into learner engagement. This has broader application in video-based learning environments

Supporting engagement in active video watching using quality nudges and visualisations

Learning by watching videos has been a popular method in e-learning. However, developing and maintaining constructive engagement is a crucial challenge in video-based learning (VBL). AVW-Space is an online VBL platform that enhances student engagement by providing note- taking and peer-reviewing. Previous studies with AVW-Space showed higher learning outcomes for students who write high-quality comments. Furthermore, an earlier study on AVW-Space suggested that visualising the student progress could help learners monitor and regulate their learning. Thus, this research aimed to increase engagement in AVW-Space by offering 1) personalised prompts, named Quality nudges, to encourage writing better comments and 2) visualisations of the student model to facilitate monitoring and controlling learning. I conducted a series of studies to investigate the effectiveness of Quality nudges and visualisations on the students’ engagement and learning. Firstly, I automated the assessment of comments quality using machine learning approaches. Then, I developed Quality nudges which encourage students to write better comments by triggering critical thinking and self-reflection. Next, I conducted a study in the context of presentation skills to analyse the effectiveness of the Quality nudges. The results showed that Quality nudges improved the quality of comments and increased learning consequently. After adding new visual learning analytics to AVW- Space, I investigated the effectiveness of the visualisations by conducting another study in the context of presentation skills. The results showed that the visualisations enhanced constructive engagement and learning even further. I also investigated the generalisability of nudges and visualisation for another transferable skill by making Quality nudges and visualisations customisable and conducting a study in the context of communication skills. Although the results showed that students used visualisations and nudges for communication skills differently from the participants in the study on presentation skills, findings indicated these interventions were still effective in increasing the quality of comments and enhancing constructive behaviour and learning. This research contributes to the development of intelligent learning environments which provide personalised interventions to encourage constructive commenting behaviours during video-based learning. The interventions proposed in this research can be applied to other domains which involve critical thinking and self-reflection. Another contribution of this research is providing visual learning analytics for students in VBL platforms to increase learning awareness and engagement. The nudges and visualisations proposed in this research could be applied to any other video-based learning platform that allows commenting

Ontology-based Domain Diversity Profiling of User Comments

Diversity has been the subject of study in various disciplines from biology to social science and computing. Respecting and utilising the diversity of the population is increasingly important to broadening knowledge. This paper describes a pipeline for diversity profiling of a pool of text in order to understand its coverage of an underpinning domain. The application is illustrated by using a domain ontology on presentation skills in a case study with 38 postgraduates who made comments while learning pitch presentations with the Active Video Watching system (AVW-Space). The outcome shows different patterns of coverage on the domain by the comments in each of the eight videos

Investigating the Effect of Adding Nudges to Increase Engagement in Active Video Watching

In order for videos to be a powerful medium for learning, it is crucial that learners engage in constructive learning. Historic interactions of previous learners can provide a rich resource to enhance interaction and promote engagement fostering constructive learning. This paper proposes such a novel approach of adding nudges to AVW-Space, a platform for video-based learning. We present the enhancements implemented in AVW-Space in the form of interactive visualizations and personalized prompts. A study focusing on presentation skills was conducted in a large first-year engineering course, in which AVW-Space provided an online resource for the students to use as they wish. The students were randomly divided into the control and experimental groups, which had access to the original and enhanced version of AVW-Space respectively. Our findings show that nudging is effective in fostering constructive learning: there was a significant difference in the percentage of constructive students in the two groups. The experimental group students wrote more comments, found AVW-Space easier to use, reported less frustration when commenting, and had higher confidence in their performance on commenting

Characterizing Comment Types and Levels of Engagement in Video-Based Learning as a Basis for Adaptive Nudging

Video is frequently used as a learning medium in a variety of educational settings, including large online courses as well as informal learning scenarios. To foster learner engagement around instructional videos, our learning scenario facilitates interactive note taking and commenting similar to popular social video-sharing platforms. This approach has recently been enriched by introducing nudging mechanisms, which raises questions about ensuing learning effects. To better understand the nature of these effects, we take a closer look at the content of the comments. Our study is based on an ex post analysis of a larger data set from a recent study. As a first step of analysis, video comments are clustered based on a feature set that captures the temporal and semantic alignment of comments with the videos. Based on the ensuing typology of comments, learners are characterized through the types of comments that they have contributed. The results will allow for a better targeting of nudges to improve video-based learning

Self-Regulation, Knowledge, Experience: Which User Characteristics Are Useful for Predicting Video Engagement?

The use of videos in education has attracted considerable research attention. However, in order to gain the most benefits, learners need to actively engage with videos. It is an important, yet challenging, task to disentangle the relation between engagement with videos and learning, and at the same time to take into account relevant individual differences in order to offer personalised support. In this paper we investigate the question: `Can user characteristics relating to self-regulation, knowledge, and experience be leveraged for predicting user engagement with videos?'. Our results show that users' domain knowledge and self-regulation abilities can inform overall engagement prediction (inactive, passive and constructive learners), which makes them useful for adaptation and personalisation

Quality in MOOCs: Surveying the Terrain

The purpose of this review is to identify quality measures and to highlight some of the tensions surrounding notions of quality, as well as the need for new ways of thinking about and approaching quality in MOOCs. It draws on the literature on both MOOCs and quality in education more generally in order to provide a framework for thinking about quality and the different variables and questions that must be considered when conceptualising quality in MOOCs. The review adopts a relativist approach, positioning quality as a measure for a specific purpose. The review draws upon Biggs’s (1993) 3P model to explore notions and dimensions of quality in relation to MOOCs — presage, process and product variables — which correspond to an input–environment–output model. The review brings together literature examining how quality should be interpreted and assessed in MOOCs at a more general and theoretical level, as well as empirical research studies that explore how these ideas about quality can be operationalised, including the measures and instruments that can be employed. What emerges from the literature are the complexities involved in interpreting and measuring quality in MOOCs and the importance of both context and perspective to discussions of quality

Reflective experiential learning : towards improving the communication skills of software engineering students using Active Video Watching

Communication skills are a key competency for software engineers, as they spend a significant amount of time communicating with various stakeholders. Teaching communication skills in a conventional educational context is resource intensive. The university institutions and instructors may lack the capability and resources to provide adequate guidance, feedback and engagement to each student. Today, video-based learning is widely utilised in both formal education and informal learning in a range of settings, and it is regarded as one of the most important tools for creating engaging learning environments. Numerous studies indicate, however, that for students to learn effectively while watching videos, they must engage actively with video content. Therefore, this study employs Active Video Watching (AVW) via AVWSpace to facilitate engagement and teach software engineering students face-to-face meeting communication skills. With reference to the online training, I assess the students' experiences with AVW-Space and the effectiveness of AVW as a reflective strategy for teaching the communication skills. The findings of this study show that only students who commented on videos and rated other students' comments (constructive learners) increased their conceptual knowledge of meeting communication skills, whereas those who were less engaged with the learning materials (passive and active learners) did not. Students noted some difficulties with commenting on videos and rating comments in AVW-Space, which led to recommendations on further improvements to the platform. Overall, the students were enthusiastic about the opportunity to engage in the online training, which allowed them to reflect on prior experiences and learn from their peers, confirming the effectiveness of AVW for learning of soft skills