New measurement paradigms

This collection of New Measurement Paradigms papers represents a snapshot of the variety of measurement methods in use at the time of writing across several projects funded by the National Science Foundation (US) through its REESE and DR K–12 programs. All of the projects are developing and testing intelligent learning environments that seek to carefully measure and promote student learning, and the purpose of this collection of papers is to describe and illustrate the use of several measurement methods employed to achieve this. The papers are deliberately short because they are designed to introduce the methods in use and not to be a textbook chapter on each method. The New Measurement Paradigms collection is designed to serve as a reference point for researchers who are working in projects that are creating e-learning environments in which there is a need to make judgments about students’ levels of knowledge and skills, or for those interested in this but who have not yet delved into these methods

AI in Education needs interpretable machine learning: Lessons from Open Learner Modelling

Interpretability of the underlying AI representations is a key raison d'\^{e}tre for Open Learner Modelling (OLM) -- a branch of Intelligent Tutoring Systems (ITS) research. OLMs provide tools for 'opening' up the AI models of learners' cognition and emotions for the purpose of supporting human learning and teaching. Over thirty years of research in ITS (also known as AI in Education) produced important work, which informs about how AI can be used in Education to best effects and, through the OLM research, what are the necessary considerations to make it interpretable and explainable for the benefit of learning. We argue that this work can provide a valuable starting point for a framework of interpretable AI, and as such is of relevance to the application of both knowledge-based and machine learning systems in other high-stakes contexts, beyond education.Comment: presented at 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018), Stockholm, Swede

Design and evaluation of a case-based system for modelling exploratory learning behaviour of math generalisation

Exploratory learning environments (ELEs) promote a view of learning that encourages students to construct and/or explore models and observe the effects of modifying their parameters. The freedom given to learners in this exploration context leads to a variety of learner approaches for constructing models and makes modelling of learner behaviour a challenging task. To address this issue, we propose a learner modelling mechanism for monitoring learners’ actions when constructing/exploring models by modelling sequences of actions reflecting different strategies in solving a task. This is based on a modified version of case-based reasoning for problems with multiple solutions. In our formulation, approaches to explore the task are represented as sequences of simple cases linked by temporal and dependency relations, which are mapped to the learners’ behaviour in the system by means of appropriate similarity metrics. This paper presents the development and validation of the modelling mechanism. The model was validated in the context of an ELE for mathematical generalisation using data from classroom sessions and pedagogically-driven learning scenarios

Innovating Pedagogy 2015: Open University Innovation Report 4

This series of reports explores new forms of teaching, learning and assessment for an interactive world, to guide teachers and policy makers in productive innovation. This fourth report proposes ten innovations that are already in currency but have not yet had a profound influence on education. To produce it, a group of academics at the Institute of Educational Technology in The Open University collaborated with researchers from the Center for Technology in Learning at SRI International. We proposed a long list of new educational terms, theories, and practices. We then pared these down to ten that have the potential to provoke major shifts in educational practice, particularly in post-school education. Lastly, we drew on published and unpublished writings to compile the ten sketches of new pedagogies that might transform education. These are summarised below in an approximate order of immediacy and timescale to widespread implementation

Combining scaffolding for content and scaffolding for dialogue to support conceptual break throughs in understanding probability

The final publication is available at Springer via http://dx.doi.org/10.1007/s11858-015-0720-5In this paper, we explore the relationship between scaffolding, dialogue and conceptual breakthroughs, using data from a design-based research study into the development of understanding of probability in 10-12 year old students. Our aim in the study was to gain insight into how the combination of the scaffolding of content using technology and scaffolding for dialogue in the expectation that this would facilitate conceptual breakthroughs. We analyse video-recordings and transcripts of pairs and triads talking together around TinkerPlots software with worksheets and teacher interventions, focusing on moments of conceptual breakthrough. The dialogue scaffolding promoted both dialogue moves specific to the context of probability and dialogue in itself. This paper focuses on an episode of learning that occurred within dialogues (framed and supported by the scaffolding. We present this as support for our claim that combining scaffolding for content with scaffolding for dialogue can be effective. This finding contributes to our understanding of both scaffolding and dialogic teaching in mathematics education by suggesting that scaffolding can be used effectively to prepare for conceptual development through dialogue.7th European Community Framework Programme - Marie Curie Intra European Fellowshi

Reinventing learning: a design-research odyssey

Design research is a broad, practice-based approach to investigating problems of education. This approach can catalyze the development of learning theory by fostering opportunities for transformational change in scholars’ interpretation of instructional interactions. Surveying a succession of design-research projects, I explain how challenges in understanding students’ behaviors promoted my own recapitulation of a historical evolution in educators’ conceptualizations of learning—Romantic, Progressivist, and Synthetic (Schön, Intuitive thinking? A metaphor underlying some ideas of educational reform (working paper 8). Division for Study and Research in Education, MIT, Cambridge, 1981)—and beyond to a proposed Systemic view. In reflection, I consider methodological adaptations to design-research practice that may enhance its contributions in accord with its objectives

Towards the Situated Engagement Evaluation Model (SEEM) : making the invisible visible

This thesis explores the multifaceted concept of engagement within online learning environments. Key research aims are to suggest approaches and an extendable model for evaluating, monitoring and developing understanding of online learner engagement. The overall intention is to offer educators insight, practical guidance and tools for supporting timely intervention in fostering learner engagement. This thesis reviews the major theoretical perspectives on learning and highlights the role of student engagement in relation to the research literature. It discusses the limitations of the methods applied in current research and attempts to address this problem by crossing the disciplinary boundaries to draw together a range of perspectives and methodologies. A review of the literature provides a foundation for a learner engagement evaluation model that employs a variety of evaluation methods and accommodates the possible diversity of learning experiences. The proposed ‘Situated Engagement Evaluation Model’ (SEEM) is positioned to reflect the wide theoretical perspective of social learning. It constitutes a comprehensive system of intertwined components (Learning Content; Pedagogical Design Elements; Learning Profiles; and Dialogue and Communication) that learners may interact with, and integrates dynamically changing preferences and predispositions (e.g. cultural, emotional, cognitive) potentially informative in engagement studies. Prior to (and independently of) the development of SEEM, four empirical studies were conducted and reported here. These explored patterns of online engagement with respect to learning content, learning profiles, patterns of communication and elements of pedagogical design. Studies were then revisited to evaluate the usefulness of SEEM for monitoring and evaluating student engagement, and to discuss its potential for guiding intervention to improve learning experiences. The practical relevance for integrated and automated implementation of SEEM in online learning is considered further

Construals as a complement to intelligent tutoring systems in medical education

This is a preliminary version of a report prepared by Meurig and Will Beynon in conjunction with a poster paper "Mediating Intelligence through Observation, Dependency and Agency in Making Construals of Malaria" at the 11th International Conference on Intelligent Tutoring Systems (ITS 2012) and a paper "Construals to Support Exploratory and Collaborative Learning in Medicine" at the associated workshop on Intelligent Support for Exploratory Environments (ISEE 2012). A final version of the report will be published at a later stage after feedback from presentations at these events has been taken into account, and the experimental versions of the JS-EDEN interpreter used in making construals have been developed to a more mature and stable form

A LEARNER INTERACTION STUDY OF DIFFERENT ACHIEVEMENT GROUPS IN MPOCS WITH LEARNING ANALYTICS TECHNIQUES

The purpose of this study was to conduct data-driven research by employing learning analytics methodology and Big Data in learning management systems (LMSs), and then to identify and compare learners’ interaction patterns in different achievement groups through different course processes in Massive Private Online Courses (MPOCs). Learner interaction is the foundation of a successful online learning experience. However, the uncertainties about the temporal and sequential patterns of online interaction and the lack of knowledge about using dynamic interaction traces in LMSs have prevented research on ways to improve interactive qualities and learning effectiveness in online learning. Also, most research focuses on the most popular online learning organization form, Massive Open Online Courses (MOOCs), and little online learning research has been conducted to investigate learners’ interaction behaviors in another important online learning organization form: MPOCs. To fill these needs, the study pays attention to investigate the frequent and effective interaction patterns in different achievement groups as well as in different course processes, and attaches importance to LMS trace data (log data) in better serving learners and instructors in online learning. Further, the learning analytics methodology and techniques are introduced here into online interaction research. I assume that learners with different achievements express different interaction characteristics. Therefore, the hypotheses in this study are: 1) the interaction activity patterns of the high-achievement group and the low-achievement group are different; 2) in both groups, interaction activity patterns evolve through different course processes (such as the learning process and the exam process). The final purpose is to find interaction activity patterns that characterize the different achievement groups in specific MPOCs courses. Some learning analytics approaches, including Hidden Markov models (HMMs) and other related measures, are taken into account to identify frequently occurring interaction activity sequence patterns of High/Low achievement groups in the Learning/Exam processes under MPOCs settings. The results demonstrate that High-achievement learners especially focused on content learning, assignments, and quizzes to consolidate their knowledge construction in both Learning and Exam processes, while Low-achievement learners significantly did not perform the same. Further, High-achievement learners adjusted their learning strategies based on the goals of different course processes; Low-achievement learners were inactive in the learning process and opportunistic in the exam process. In addition, despite achievements or course processes, all learners were most interested in checking their performance statements, but they engaged little in forum discussion and group learning. In sum, the comparative analysis implies that certain interaction patterns may distinguish the High-achievement learners from the Low-achievement ones, and learners change their patterns more or less based on different course processes. This study provides an attempt to conduct learner interaction research by employing learning analytics techniques. In the short term, the results will give in-depth knowledge of the dynamic interaction patterns of MPOCs learners. In the long term, the results will help learners to gain insight into and evaluate their learning, help instructors identify at-risk learners and adjust instructional strategies, help developers and administrators to build recommendation systems based on objective and comprehensive information, all of which in turn will help to improve the achievements of all learner groups in specific MPOC courses