Adaptive Analytics: It’s About Time

This article describes a cooperative research partnership among a large public university, a for-profit private institution and their common adaptive learning platform provider. The focus of this work explored adaptive analytics that uses data the investigators describe as metaphorical “digital learning dust” produced by the platform as a matter of course. The information configured itself into acquired knowledge, growth, baseline status and engagement. Two complimentary models evolved. The first, in the public university, captured end-of-course data for predicting success. The second approach, in the private university, formed the basis of a dynamic real-time data analytic algorithm. In both cases the variables that best predicted students at risk (effective use of time and revision attempts) were deemed teachable skills that can improve with intervention

How Simulation can Illuminate Pedagogical and System Design Issues in Dynamic Open Ended Learning Environments

A Dynamic Open-Ended Learning Environment (DOELE) is a collection of learners and learning objects (LOs) that could be constantly changing. In DOELEs, learners need the support of Advanced Learning Technology (ALT), but most ALT is not designed to run in such environments. An architecture for designing advanced learning technology that is compatible with DOELEs is the ecological approach (EA). This thesis looks at how to test and develop ALT based on the EA, and argues that this process would benefit from the use of simulation. The essential components of an EA-based simulation are: simulated learners, simulated LOs, and their simulated interactions. In this thesis the value of simulation is demonstrated with two experiments. The first experiment focuses on the pedagogical issue of peer impact, how learning is impacted by the performance of peers. By systematically varying the number and type of learners and LOs in a DOELE, the simulation uncovers behaviours that would otherwise go unseen. The second experiment shows how to validate and tune a new instructional planner built on the EA, the Collaborative Filtering based on Learning Sequences planner (CFLS). When the CFLS planner is configured appropriately, simulated learners achieve higher performance measurements that those learners using the baseline planners. Simulation results lead to predictions that ultimately need to be proven in the real world, but even without real world validation such predictions can be useful to researchers to inform the ALT system design process. This thesis work shows that it is not necessary to model all the details of the real world to come to a better understanding of a pedagogical issue such as peer impact. And, simulation allowed for the design of the first known instructional planner to be based on usage data, the CFLS planner. The use of simulation for the design of EA-based systems opens new possibilities for instructional planning without knowledge engineering. Such systems can find niche learning paths that may have never been thought of by a human designer. By exploring pedagogical and ALT system design issues for DOELEs, this thesis shows that simulation is a valuable addition to the toolkit for ALT researchers

Applications of Discourse Structure for Spoken Dialogue Systems

Language exhibits structure beyond the sentence level (e.g. the syntactic structure of a sentence). In particular, dialogues, either human-human or human-computer, have an inherent structure called the discourse structure. Models of discourse structure attempt to explain why a sequence of random utterances combines to form a dialogue or no dialogue at all. Due to the relatively simple structure of the dialogues that occur in the information-access domains of typical spoken dialogue systems (e.g. travel planning), discourse structure has often seen limited application in such systems. In this research, we investigate the utility of discourse structure for spoken dialogue systems in more complex domains, e.g. tutoring. This work was driven by two intuitions.First, we believed that the "position in the dialogue" is a critical information source for two tasks: performance analysis and characterization of dialogue phenomena. We define this concept using transitions in the discourse structure. For performance analysis, these transitions are used to create a number of novel factors which we show to be predictive of system performance. One of these factors informs a promising modification of our system which is implemented and compared with the original version of the system through a user study. Results show that the modification leads to objective improvements. For characterization of dialogue phenomena, we find statistical dependencies between discourse structure transitions and two dialogue phenomena which allow us to speculate where and why these dialogue phenomena occur and to better understand system behavior.Second, we believed that users will benefit from direct access to discourse structure information. We enable this through a graphical representation of discourse structure called the Navigation Map. We demonstrate the subjective and objective utility of the Navigation Map through two user studies.Overall, our work demonstrates that discourse structure is an important information source for designers of spoken dialogue systems

Big data for monitoring educational systems

This report considers “how advances in big data are likely to transform the context and methodology of monitoring educational systems within a long-term perspective (10-30 years) and impact the evidence based policy development in the sector”, big data are “large amounts of different types of data produced with high velocity from a high number of various types of sources.” Five independent experts were commissioned by Ecorys, responding to themes of: students' privacy, educational equity and efficiency, student tracking, assessment and skills. The experts were asked to consider the “macro perspective on governance on educational systems at all levels from primary, secondary education and tertiary – the latter covering all aspects of tertiary from further, to higher, and to VET”, prioritising primary and secondary levels of education

Interpreting Early Career Trajectories

Career decisions of four teachers are explored through the concept of figured worlds in this qualitative, longitudinal case study. Participants were purposefully chosen for similarity at entry, with a range of career trajectories over time. Teacher career paths included remaining in one school, repeated changes in schools, attrition after relocation, and nonrenewal of contract. Data included interviews, observations, participants’ assessments, and pupils’ work. Cross-case analysis suggests that no single teacher attribute or workplace condition determined teachers’ career decisions; rather, teachers’ ability to refigure their identity within the figured world of teaching shaped career trajectory. Key factors such as ability to address disequilibrium, teacher identity, agency, and collaborative capacity are examined. Implications call for pre-service preparation and professional development to navigate cultures of schools, amended administrative involvement in teacher retention, and policy reform acknowledging the complexity of teachers’ figured worlds

Technology-Based Personalization: Instructional Reform in Five Public Schools

This dissertation addresses the question: How does an attempt to redesign instructional delivery using technology-based personalization affect the technical core of teaching, learning, and student outcomes? In recent years, many prominent educators, business leaders, and philanthropists have suggested that schools be redesigned to personalize students’ learning experiences using technology. However, the justification for these reforms remains largely theoretical. Empirical research on technology-based personalization is sparse, and what little research does exist focuses predominantly on macro effects rather than the specific school-level, class-level, student-level, and lesson-level mechanisms that contribute to overall student achievement. The absence of research that pushes inside the “black box” of implementation is particularly problematic given a century of failed attempts to reform the technical core of instructional delivery, with symbolic reforms typically withering in the face of institutional resistance. This study attempts to address that gap by examining the implementation of an innovative model for using technology-based personalization to deliver middle school math instruction. I draw upon theoretical tools from institutional theory, instructional improvement, and the history of educational reform to deepen our understanding of how technology-based personalization affects the role of students and teachers, the logistics of content delivery, and students’ learning outcomes. Unlike previous studies in K-12 settings, which typically use summative assessments and virtual control groups to estimate aggregate effects on student learning, this study examines the relationships among a diverse set of lesson-level variables, including instructional modality, instructional content, group size and composition, teacher characteristics, student characteristics, and learning outcomes. In doing so, this study contributes to our understanding of the on-the-ground processes and mechanisms by which technology- based personalization affects (or does not affect) student learning. Although the instructional model documented in this case study will remain anonymous, it is well known and respected among educators and philanthropists, and regarded as one of the most prominent and archetypical examples of technology-based personalization currently active in American schools. Using multiple methods, including novel applications of hierarchical linear modeling, cluster analysis, and heatmap data visualization, I explore: (a) the degree to which ground-level implementation of technology-based personalization represents an authentic departure from the traditional technology of schooling, and (b) the relationships among various elements of the model and student learning outcomes. I draw on longitudinal data from a full year of implementation in five schools, including the daily lesson assignments and assessment scores of 1,238 unique students supervised by 48 teachers. This study supports four main findings: (a) the program succeeds in altering the technical core of instruction in several fundamental ways; (b) policy and logistical constraints limit the program’s ability to reform the technical core of instruction to the degree that it aspires; (c) students who enter the program as already higher-performing are more successful on daily exit slips than students who enter the program with lower performance; and (d) the quantitative methods used in this paper represent useful and replicable tools for exploring the data produced by technology-based and personalized models

Exploring the role of experience API in supporting new trends in Educational Technology: A literature review

Despite its growth in the area of educational technology, Experience API (xAPI) continues to be under-used as a solution across the different platforms in institutions and organizations. There is a lack of any detailed summary in the literature about the potential and the limitation of using xAPI in conjunction with learning platforms and technologies. This thesis examines the role of xAPI in promoting, shaping and supporting learning in organizational contexts. This discussion is developed by using cases reported in the literature and new cases from contemporary educational technologies. The thesis illustrates the role the standard plays within current major trends in digital learning and within the context of a broader ecosystem of learning platforms and technologies. It provides a useful and thorough account of xAPI and its potential to an audience of individuals responsible for implementing xAPI within organizations. xAPI provides to some extent a promise of improved impact to Performance Evaluation and Evaluating training Effectiveness. However, xAPI lacks concrete cases and examples to support its utilization in the fields of Learning Analytics, Performance Management, Predictive Learning and Workforce Planning. Keywords: Experience API (xAPI), Learning Management Systems, Learning Record Store, Learning Analytics, Microlearning, Evaluation Effectiveness, Predictive Learning, Adaptive Learning, Workforce Planning

Current Issues in Emerging eLearning, Volume 7, Issue 1: APLU Special Issue on Implementing Adaptive Learning At Scale

The second of two Specials Issues of the CIEE journal to have been produced and guest edited by the Personalized Learning Consortium (PLC) of the Association of Public and Land-grant Universities (APLU), featuring important research resulting from university initiatives to launch, implement and scale up the use of adaptive courseware and the strategies of adaptive learning