Unravelling the dynamics of learning design within and between disciplines in higher education using learning analytics

Designing effective learning experience in virtual learning environment (VLE) can be supported by learning analytics (LA) through explicit feedback on how learning design (LD) influences students’ engagement, satisfaction and performance. Marrying LA with LD not only puts existing pedagogical theories in instructional design to the test with actual learning data, but also provides the context of learning which helps educators translate established LA findings to direct interventions. My dissertation aims at unpacking the complexity of LD and its impact on students’ engagement, satisfaction and performance on VLE using LA. The context of this study is 400+ online and blended learning modules at the Open University (OU) UK. This research combines multiple sources of data from the OU Learning Design Initiative (OULDI), system log data, self-reported surveys, and performance data. Given the scope of this study, a wide range of visualization techniques, social network analysis, multi-level modelling, and machine learning will be used

A review of ten years of implementation and research in aligning learning design with learning analytics at the Open University UK

There is an increased recognition that learning design drives both student learning experience and quality enhancements of teaching and learning. The Open University UK (OU) has been one of few institutions that have explicitly and systematically captured the designs for learning at a large scale. By applying advanced analytical techniques on large and fine-grained datasets, the OU has been unpacking the complexity of instructional practices, as well as providing conceptual and empirical evidence of how learning design influences student behaviour, satisfaction, and performance. This study discusses the implementation of learning design at the OU in the last ten years, and critically reviews empirical evidence from eight recent large-scale studies that have linked learning design with learning analytics. Four future research themes are identified to support future adoptions of learning design approaches

Mixing and Matching Learning Design and Learning Analytics

In the last five years, learning analytics has proved its potential in predicting academic performance based on trace data of learning activities. However, the role of pedagogical context in learning analytics has not been fully understood. To date, it has been difficult to quantify learning in a way that can be measured and compared. By coding the design of e-learning courses, this study demonstrates how learning design is being implemented on a large scale at the Open University UK, and how learning analytics could support as well as benefit from learning design. Building on our previous work, our analysis was conducted longitudinally on 23 undergraduate distance learning modules and their 40,083 students. The innovative aspect of this study is the availability of fine-grained learning design data at individual task level, which allows us to consider the connections between learning activities, and the media used to produce the activities. Using a combination of visualizations and social network analysis, our findings revealed a diversity in how learning activities were designed within and between disciplines as well as individual learning activities. By reflecting on the learning design in an explicit manner, educators are empowered to compare and contrast their design using their own institutional data

Linking students' timing of engagement to learning design and academic performance

In recent years, the connection between Learning Design (LD) and Learning Analytics (LA) has been emphasized by many scholars as it could enhance our interpretation of LA findings and translate them to meaningful interventions. Together with numerous conceptual studies, a gradual accumulation of empirical evidence has indicated a strong connection between how instructors design for learning and student behaviour. Nonetheless, students' timing of engagement and its relation to LD and academic performance have received limited attention. Therefore, this study investigates to what extent students' timing of engagement aligned with instructor learning design, and how engagement varied across different levels of performance. The analysis was conducted over 28 weeks using trace data, on 387 students, and replicated over two semesters in 2015 and 2016. Our findings revealed a mismatch between how instructors designed for learning and how students studied in reality. In most weeks, students spent less time studying the assigned materials on the VLE compared to the number of hours recommended by instructors. The timing of engagement also varied, from in advance to catching up patterns. High-performing students spent more time studying in advance, while low-performing students spent a higher proportion of their time on catching-up activities. This study reinforced the importance of pedagogical context to transform analytics into actionable insights

Methods and models of next generation technology enhanced learning - White Paper

Our understanding of learning with technology is increasingly lagging behind technological advancements, such that it is no longer possible to fully understand learning with technologies without bringing together evidence from practice-based experiences and theoretical insight to inform research, design, policy and practice. Furthermore, whilst practical experiences and theoretical insights make significant contributions towards understanding learning with new technologies, the dynamic nature of learner practices and study contexts make it difficult to predict future requirements in terms of methods and models for next generation technology enhanced learning. We therefore require formal and comprehensive methods and models of learning with technology that accommodate theory and practice whilst allowing us to anticipate methodological innovations that capture future transitions and changes in learner practices and study contexts, in order to inform research, design, policy and practice. Workshop participants represented different communities of interest including research, design, evaluation and assessment. The overall objective was to anticipate methodological innovations in technology enhanced learning research and design over the next 5/10 years

Learning design – making practice explicit

New technologies have immense potential for learning, but the sheer variety possible also creates challenges for learners in terms of navigating through an increasingly complex digital landscape and for teachers in terms of how to design and support learning interventions. How can learners and teachers make informed decisions about what technologies to use in the design and support of learning activities? This presentation will consider this question and present a new methodology for design – 'learning design', which aims to shift the creation and support of learning from what has traditionally been an implicit, belief-based practice to one that is explicit and design based. Learning design research at the Open University, UK has included the development of a set of conceptual design views, a tool for visualising designs (CompendiumLD) and a social networking site, for sharing and discussing learning and teaching ideas and designs (Cloudworks). An overview of this work will be provided, along with a discussion of the perceived benefits of this new approach to educational design

Student profiling in a dispositional learning analytics application using formative assessment

How learning disposition data can help us translating learning feedback from a learning analytics application into actionable learning interventions, is the main focus of this empirical study. It extends previous work where the focus was on deriving timely prediction models in a data rich context, encompassing trace data from learning management systems, formative assessment data, e-tutorial trace data as well as learning dispositions. In this same educational context, the current study investigates how the application of cluster analysis based on e-tutorial trace data allows student profiling into different at-risk groups, and how these at-risk groups can be characterized with the help of learning disposition data. It is our conjecture that establishing a chain of antecedent-consequence relationships starting from learning disposition, through student activity in e-tutorials and formative assessment performance, to course performance, adds a crucial dimension to current learning analytics studies: that of profiling students with descriptors that easily lend themselves to the design of educational interventions

What would learning in an open world look like? A vision for the future

The pace of current technological advancement is phenomenal. In the last few years we have seen the emergence of ever more sophisticated gaming technologies, rich, immersive virtual worlds and new social networking services that enable learners and teachers to connect and communicate in new ways. The pace of change looks set to continue as annual Horizon reports testify (http://www.nmc.org/horizon). Clearly new technologies offer much in an educational context, with the promise of flexible, personalised and student-centred learning. Indeed research over the past few years, looking at learners' use of technologies, has given us a rich picture of how learners of all ages are appropriating new tools within their own context, mixing different applications for finding/managing information and for communicating with others (Sharpe and Beetham, forthcoming)

FORGE: An eLearning Framework for Remote Laboratory Experimentation on FIRE Testbed Infrastructure

The Forging Online Education through FIRE (FORGE) initiative provides educators and learners in higher education with access to world-class FIRE testbed infrastructure. FORGE supports experimentally driven research in an eLearning environment by complementing traditional classroom and online courses with interactive remote laboratory experiments. The project has achieved its objectives by defining and implementing a framework called FORGEBox. This framework offers the methodology, environment, tools and resources to support the creation of HTML-based online educational material capable accessing virtualized and physical FIRE testbed infrastruc- ture easily. FORGEBox also captures valuable quantitative and qualitative learning analytic information using questionnaires and Learning Analytics that can help optimise and support student learning. To date, FORGE has produced courses covering a wide range of networking and communication domains. These are freely available from FORGEBox.eu and have resulted in over 24,000 experiments undertaken by more than 1,800 students across 10 countries worldwide. This work has shown that the use of remote high- performance testbed facilities for hands-on remote experimentation can have a valuable impact on the learning experience for both educators and learners. Additionally, certain challenges in developing FIRE-based courseware have been identified, which has led to a set of recommendations in order to support the use of FIRE facilities for teaching and learning purposes