Stability and sensitivity of Learning Analytics based prediction models

Learning analytics seek to enhance the learning processes through systematic measurements of learning related data and to provide informative feedback to learners and educators. Track data from Learning Management Systems (LMS) constitute a main data source for learning analytics. This empirical contribution provides an application of Buckingham Shum and Deakin Crick’s theoretical framework of dispositional learning analytics: an infrastructure that combines learning dispositions data with data extracted from computer-assisted, formative assessments and LMSs. In two cohorts of a large introductory quantitative methods module, 2049 students were enrolled in a module based on principles of blended learning, combining face-to-face Problem-Based Learning sessions with e-tutorials. We investigated the predictive power of learning dispositions, outcomes of continuous formative assessments and other system generated data in modelling student performance and their potential to generate informative feedback. Using a dynamic, longitudinal perspective, computer-assisted formative assessments seem to be the best predictor for detecting underperforming students and academic performance, while basic LMS data did not substantially predict learning. If timely feedback is crucial, both use-intensity related track data from e-tutorial systems, and learning dispositions, are valuable sources for feedback generation

Student profiling in a dispositional learning analytics application using formative assessment

How learning disposition data can help us translating learning feedback from a learning analytics application into actionable learning interventions, is the main focus of this empirical study. It extends previous work where the focus was on deriving timely prediction models in a data rich context, encompassing trace data from learning management systems, formative assessment data, e-tutorial trace data as well as learning dispositions. In this same educational context, the current study investigates how the application of cluster analysis based on e-tutorial trace data allows student profiling into different at-risk groups, and how these at-risk groups can be characterized with the help of learning disposition data. It is our conjecture that establishing a chain of antecedent-consequence relationships starting from learning disposition, through student activity in e-tutorials and formative assessment performance, to course performance, adds a crucial dimension to current learning analytics studies: that of profiling students with descriptors that easily lend themselves to the design of educational interventions

Cracking the Code: Synchronizing Policy and Practice for Performance-Based Learning

Proposes a policy framework for integrating performance-based learning into the education system, synchronizing policy and practice, and ensuring collaborative state leadership and flexible federal leadership. Lists state policy issues and exemplars

Responsible research and innovation in science education: insights from evaluating the impact of using digital media and arts-based methods on RRI values

The European Commission policy approach of Responsible Research and Innovation (RRI) is gaining momentum in European research planning and development as a strategy to align scientific and technological progress with socially desirable and acceptable ends. One of the RRI agendas is science education, aiming to foster future generations' acquisition of skills and values needed to engage in society responsibly. To this end, it is argued that RRI-based science education can benefit from more interdisciplinary methods such as those based on arts and digital technologies. However, the evidence existing on the impact of science education activities using digital media and arts-based methods on RRI values remains underexplored. This article comparatively reviews previous evidence on the evaluation of these activities, from primary to higher education, to examine whether and how RRI-related learning outcomes are evaluated and how these activities impact on students' learning. Forty academic publications were selected and its content analysed according to five RRI values: creative and critical thinking, engagement, inclusiveness, gender equality and integration of ethical issues. When evaluating the impact of digital and arts-based methods in science education activities, creative and critical thinking, engagement and partly inclusiveness are the RRI values mainly addressed. In contrast, gender equality and ethics integration are neglected. Digital-based methods seem to be more focused on students' questioning and inquiry skills, whereas those using arts often examine imagination, curiosity and autonomy. Differences in the evaluation focus between studies on digital media and those on arts partly explain differences in their impact on RRI values, but also result in non-documented outcomes and undermine their potential. Further developments in interdisciplinary approaches to science education following the RRI policy agenda should reinforce the design of the activities as well as procedural aspects of the evaluation research

Layered evaluation of interactive adaptive systems : framework and formative methods

Peer reviewedPostprin

When Failure Is Not an Option: Designing Competency-Based Pathways for Next Generation Learning

Proposes an online learning-assisted model in which students advance by demonstrating mastery of subjects based on clear, measurable objectives and meaningful assessments. Examines innovation drivers, challenges, and philanthropic opportunities

Using evaluation to inform the development of a user-focused assessment engine

This paper reports on the evaluation of a new assessment system, Technologies for Online Interoperability (TOIA). TOIA was built from a user-focussed specification of an assessment system. The formative evaluation of the project complemented this initial specification by ensuring that user feedback on the development and use of the system was iteratively fed back into the development process. The paper begins by summarising some of the key barriers and enablers to the use of assessment systems and the uptake of Computer-Assisted Assessment (CAA). It goes on to provide a critique of the impact of technology on assessment and considers whether innovative uses of information and communication technology (ICT) might result in new e-pedagogies and practices in assessment. The paper then reports on the findings of the TOIA evaluation and discusses how these were used to inform the development of the system

Embedded formative assessment and classroom process quality. How do they interact in promoting students\u27 science understanding

In this study we examine the interplay between curriculum-embedded formative assessment-a well-known teaching practice-and general features of classroom process quality (i.e., cognitive activation, supportive climate, classroom management) and their combined effect on elementary school students\u27 understanding of the scientific concepts of floating and sinking. We used data from a cluster-randomized controlled trial and compared curriculum-embedded formative assessment (17 classes) with a control group (11 classes). Curriculum-embedded formative assessment and classroom process quality promoted students\u27 learning. Moreover, classroom process quality and embedded formative assessment interacted in promoting student learning. To ensure effective instruction and consequently satisfactory learning outcomes, teachers need to combine specific teaching practices with high classroom process quality. (DIPF/Orig.