Curricular Concept Maps as Structured Learning Diaries : Collecting Data on Self-Regulated Learning and Conceptual Thinking for Learning Analytics Applications

The collection and selection of the data used in learning analytics applications deserve more attention. Optimally, selection of data should be guided by pedagogical purposes instead of data availability. Using design science research methodology, we designed an artifact to collect time-series data on students’ self-regulated learning and conceptual thinking. Our artifact combines curriculum data, concept mapping, and structured learning diaries. We evaluated the artifact in a case study, verifying that it provides relevant data, requires a limited amount of effort from students, and works in different educational contexts. Combined with learning analytics applications and interventions, our artifact provides possibilities to add value for students, teachers, and academic leaders.The collection and selection of the data used in learning analytics applications deserve more attention. Optimally, selection of data should be guided by pedagogical purposes instead of data availability. Using design science research methodology, we designed an artifact to collect time-series data on students’ self-regulated learning and conceptual thinking. Our artifact combines curriculum data, concept mapping, and structured learning diaries. We evaluated the artifact in a case study, verifying that it provides relevant data, requires a limited amount of effort from students, and works in different educational contexts. Combined with learning analytics applications and interventions, our artifact provides possibilities to add value for students, teachers, and academic leaders.Peer reviewe

Unpacking the intertemporal impact of self-regulation in a blended mathematics environment

With the arrival of fine-grained log-data and the emergence of learning analytics, there may be new avenues to explore how Self-Regulated Learning (SRL) can provide a lens to how students learn in blended and online environments. In particular, recent research has found that the notion of time may be an essential but complex concept through which students make (un)conscious and self-regulated decisions as to when, what, and how to study. This study explored distinct clusters of behavioural engagement in an online e-tutorial called Sowiso at different time points (before tutorials, before quizzes, before exams), and their associations with self-regulated learning strategies, epistemic learning emotions, activity learning emotions, and academic performance. Using a cluster analysis on trace data of 1035 students practicing 429 online exercises in Sowiso, we identified four distinct cluster of students (e.g. early mastery, strategic, exam-driven, and inactive). Further analyses revealed significant differences between these four clusters in their academic performance, step-wise cognitive processing strategies, external self-regulation strategies, epistemic learning emotions and activity learning emotions. Our findings took a step forward towards personalised and actionable feedback in learning analytics by recognizing the complexity of how and when students engage in learning activities over time, and supporting educators to design early and theoretically informed interventions based on learning dispositions

Framing Professional Learning Analytics as Reframing Oneself

Central to imagining the future of technology-enhanced professional learning is the question of how data are gathered, analyzed, and fed back to stakeholders. The field of learning analytics (LA) has emerged over the last decade at the intersection of data science, learning sciences, human-centered and instructional design, and organizational change, and so could in principle inform how data can be gathered and analyzed in ways that support professional learning. However, in contrast to formal education where most research in LA has been conducted, much work-integrated learning is experiential, social, situated, and practice-bound. Supporting such learning exposes a significant weakness in LA research, and to make sense of this gap, this article proposes an adaptation of the Knowledge-Agency Window framework. It draws attention to how different forms of professional learning locate on the dimensions of learner agency and knowledge creation. Specifically, we argue that the concept of “reframing oneself” holds particular relevance for informal, work-integrated learning. To illustrate how this insight translates into LA design for professionals, three examples are provided: first, analyzing personal and team skills profiles (skills analytics); second, making sense of challenging workplace experiences (reflective writing analytics); and third, reflecting on orientation to learning (dispositional analytics). We foreground professional agency as a key requirement for such techniques to be used effectively and ethically

Evaluating an Automated Analysis using Machine Learning and Natural Language Processing Approaches to Classify Computer Science Students' Reflective Writing

Reflection writing is a common practice in higher education. However, manual analysis of written reflections is time-consuming. This study presents an automated analysis of reflective writing to analyze reflective writing in CS education based on conceptual Reflective Writing Framework (RWF) and application of natural language processing and machine learning algorithm. This paper investigates two groups of features extraction (n-grams and PoS n-grams) and random forest (RF) algorithm that utilize such features to detect the presence or absence of the seven indicators (description of an experience, understandings, feelings, reasoning, perspective, new learning, and future action). The automated analysis of reflective writing is evaluated based on 74 CS student essays (1113 sentences) that are from the final year project reports in CS’s students. Results showed the seven indicators can be reliably distinguished by their features and these indicators can be used in an automated reflective writing analysis for determining the level of students’ reflective writing. Finally, we consider the implications of how the conceptualization of refection quality and providing individualized learning support to students in order to help them develop reflective skills

Connecting Learning Analytics and Problem-Based Learning – Potentials and Challenges

Learning analytics (LA) are a young but fast-growing field, which, according to some authors, holds big promises for education. Some claim that LA solutions can help measure and support constructivist classrooms and 21st century skills, thus creating a potential of making an alignment between LA and PBL principles and practices. Despite this argument, LA have not yet gained much interest among the Problem-Based Learning (PBL) practitioners and researchers and the possible connections between PBL and LA have not yet been properly explored. The purpose of this paper is, therefore, to investigate how LA can potentially be used to support and inform PBL practice. We do this by identifying central themes that remain constant across various orchestrations of PBL (collaboration, self-directed learning, and reflection) and present examples of LA tools and concepts that have been developed within LA and neighbouring fields (e.g. CSCL) in connection to those themes. This selection of LA solutions is later used as a basis for discussing wider potentials, challenges and recommendations for making connections between PBL and LA. &nbsp

TEXT MINING DATA FROM STUDENTS TO REVEAL MEANINGFUL INFORMATION FOR EDUCATORS

Academic institutions adopt different advising tools for various objectives. Past research used both numeric and text data to predict students’ performance. Moreover, numerous research projects have been conducted to find different learning strategies and profiles of students. Those strategies of learning together with academic profiles assisted in the advising process. This research proposes an approach to supplement these activities by text mining students’ essays to better understand different students’ profiles across different courses (subjects). Text analysis was performed on 99 essays written by undergraduate students in three different courses. The essays and terms were projected in a 20-dimensional vector space. The 20 dimensions were used as independent variables in a regression analysis to predict a student’s final grade in a course. Further analyses were performed on the dimensions found statistically significant. This study is a preliminary analysis to demonstrate a novel approach of extracting meaningful information by text mining essays written by students to develop an advising tool that can be used by educators

Mobile-assisted language learning through learning analytics for self-regulated learning (MALLAS): A conceptual framework

Many adult second and foreign language learners have insufficient opportunities to engage in language learning. However, their successful acquisition of a target language is critical for various reasons, including their fast integration in a host country and their smooth adaptation to new work or educational settings. This suggests that they need additional support to succeed in their second language acquisition. We argue that such support would benefit from recent advances in the fields of mobile-assisted language learning, self-regulated language learning, and learning analytics. In particular, this paper offers a conceptual framework, mobile-assisted language learning through learning analytics for self-regulated learning (MALLAS), to help learning designers support second language learners through the use of learning analytics to enable self-regulated learning. Although the MALLAS framework is presented here as an analytical tool that can be used to operationalise the support of mobile-assisted language learning in a specific exemplary learning context, it would be of interest to researchers who wish to better understand and support self-regulated language learning in mobile contexts

Automated Analysis of Reflection in Writing: Validating Machine Learning Approaches

Reflective writing is an important educational practice to train reflective thinking. Currently, researchers must manually analyze these writings, limiting practice and research because the analysis is time and resource consuming. This study evaluates whether machine learning can be used to automate this manual analysis. The study investigates eight categories that are often used in models to assess reflective writing, and the evaluation is based on 76 student essays (5,080 sentences) that are largely from third- and second-year health, business, and engineering students. To test the automated analysis of reflection in writings, machine learning models were built based on a random sample of 80% of the sentences. These models were then tested on the remaining 20% of the sentences. Overall, the standardized evaluation shows that five out of eight categories can be detected automatically with substantial or almost perfect reliability, while the other three categories can be detected with moderate reliability (Cohen's κ ranges between .53 and .85). The accuracies of the automated analysis were on average 10% lower than the accuracies of the manual analysis. These findings enable reflection analytics that is immediate and scalable