Enhancing writing analytics in science education research with machine learning and natural language processing—Formative assessment of science and non-science preservice teachers’ written reflections

IntroductionScience educators use writing assignments to assess competencies and facilitate learning processes such as conceptual understanding or reflective thinking. Writing assignments are typically scored with holistic, summative coding rubrics. This, however, is not very responsive to the more fine-grained features of text composition and represented knowledge in texts, which might be more relevant for adaptive guidance and writing-to-learn interventions. In this study we examine potentials of machine learning (ML) in combination with natural language processing (NLP) to provide means for analytic, formative assessment of written reflections in science teacher education.MethodsML and NLP are used to filter higher-level reasoning sentences in physics and non-physics teachers’ written reflections on a standardized teaching vignette. We particularly probe to what extent a previously trained ML model can facilitate the filtering, and to what extent further fine-tuning of the previously trained ML model can enhance performance. The filtered sentences are then clustered with ML and NLP to identify themes and represented knowledge in the teachers’ written reflections.ResultsResults indicate that ML and NLP can be used to filter higher-level reasoning elements in physics and non-physics preservice teachers’ written reflections. Furthermore, the applied clustering approach yields specific topics in the written reflections that indicate quality differences in physics and non-physics preservice teachers’ texts.DiscussionOverall, we argue that ML and NLP can enhance writing analytics in science education. For example, previously trained ML models can be utilized in further research to filter higher-level reasoning sentences, and thus provide science education researchers efficient mean to answer derived research questions

Leveraging Multimodal Learning Analytics to Understand How Humans Learn with Emerging Technologies

Major education and training challenges are plaguing the United States in preparing the next generation of the future workforce to meet the demands of the 21st Century. Several calls have been released to improve education programs to ensure learners are acquiring 21st century knowledge, skills, and abilities (KSAs). As we embark on the digital and automation ages of the 21st century, it is essential that we move away from traditional education programs that define and measure KSAs as static constructs (e.g., standardized assessments) with little consideration of the actual real-time deployment of these processes, missing critical information on the degree to which learners are acquiring and applying 21st century KSAs. The objective of this dissertation is to use 1 book chapter and 2 journal articles to illustrate the value in leveraging emerging technologies and multimodal trace data to define and measure scientific thinking, reflection, and self-regulated learning--core 21st century skills, across contexts, domains, tasks, and populations (e.g., medical versus undergraduates versus middle-school students). Chapters 2-4 of this dissertation provide evidence of ways to leverage multimodal trace data guided by theoretical perspectives in cognitive and learning sciences, with a special focus in self-regulated learning, to assess the extent to which learners engaged in scientific thinking, reflection, and self-regulated learning during learning activities with emerging technologies. Overall, results from these chapters illustrate that it is necessary to utilize methods that capture learning processes as they unfold during learning activities that are guided by theoretical perspectives in self-regulated learning. Findings from this research hold significant broader impacts for addressing the education and training challenges in the United States by collecting multimodal trace data over the course of learning to not only detect and identify how learners are developing KSAs such as scientific thinking, reflection, and self-regulated learning, but where these data could be fed into an intelligent and adaptive system to repurpose it back to trainers, teachers, instructors, and learners for just-in-time interventions and individualized feedback. The intellectual merit of this dissertation focuses predominantly on the importance of utilizing rich streams of multimodal trace data that are mapped onto different theoretical perspectives on how humans self-regulate across tasks like clinical reasoning, scientific thinking, and reflection with emerging technologies such as a game-based learning environment called Crystal Island. Discussion is incorporated around ways to leverage multimodal trace data on undergraduate, middle-school, and medical student populations across a range of tasks including learning about microbiology to problem solving with a game-based learning environment called Crystal Island and clinically reasoning about diagnoses across emerging technologies

Capturing and Scaffolding the Complexities of Self-Regulation During Game-Based Learning

Game-based learning environments (GBLEs) can offer students with engaging interactive instructional materials while also providing a research platform to investigate the dynamics and intricacies of effective self-regulated learning (SRL). Past research has indicated learners are often unable to monitor and regulate their cognitive and metacognitive processes within GBLEs accurately and effectively on their own due mostly to the open-ended nature of these environments. The future design and development of GBLEs and embedded scaffolds, therefore, require a better understanding of the discrepancies between the affordances of GBLEs and the required use of SRL. Specifically, how to incorporate interdisciplinary theories and concepts outside of traditional educational, learning, and psychological sciences literature, how to utilize process data to measure SRL processes during interactions with instructional materials accounting for the dynamics of leaners\u27 SRL, and how to improve SRL-driven scaffolds to be individualized and adaptive based on the level of agency GBLEs provide. Across four studies, this dissertation investigates learners\u27 SRL while they learn about microbiology using CRYSTAL ISLAND, a GBLE, building upon each other by enhancing the type of data collected, analytical methodologies used, and applied theoretical models and theories. Specifically, this dissertation utilizes a combination of traditional statistical approaches (i.e., linear regression models), non-linear statistical approaches (i.e., growth modeling), and non-linear dynamical theory (NDST) approaches (aRQA) with process trace data to contribute to the field\u27s current understanding of the dynamics and complexities of SRL. Furthermore, this dissertation examines how limited agency can act as an implicit scaffold during game-based learning to promote the use of SRL processes and increase learning outcomes

Supporting learning in intelligent tutoring systems with motivational strategies.

Motivation and affect detection are prominent yet challenging areas of research in the field of Intelligent Tutoring Systems (ITSs). Devising strategies to engage learners and motivate them to practice regularly are of great interest to researchers. In the learning and education domain, where students use ITSs regularly, motivating them to engage with the system effectively may lead to higher learning outcomes. Therefore, developing an ITS which provides a complete learning experience to students by catering to their cognitive, affective, metacognitive, and motivational needs is an ambitious yet promising area of research. This dissertation is the first step towards this goal in the context of SQL-Tutor, a mature ITS for tutoring SQL. In this research project, I have conducted a series of studies to detect and evaluate learners' affective states and employed various strategies for increasing motivation and engagement to improve learning from SQL-Tutor. Firstly, I established the reliability of iMotions to correctly identify learners' emotions and found that worked examples alleviated learners' frustration while solving problems with SQL-Tutor. Gamification is introduced as a motivational strategy to persuade learners to practice with the system. Gamification has emerged as a strong engagement and motivation strategy in learning environments for young learners. I evaluated the effects of gamified SQL-Tutor on undergraduate students and found that gamification indirectly improved learning by influencing learners’ time on task. It helped students by increasing their motivation which produce similar effects as intrinsically motivated students. Additionally, prior knowledge, gamification experience, and interest in the topic moderated the effects of gamification. Lastly, self-regulated learning support is presented as another strategy to affect learners’ internal motivation and skills. The support provided in the form of interventions improved students’ learning outcomes. Additionally, the learners' challenge-accepting behaviour, problem selection, goal setting, and self-reflection have improved with support without experiencing any negative emotions. This research project contributes to the latest trends of motivation and learning research in ITS

Lernen, Lehren und Forschen in einer digital geprägten Welt. Gesellschaft für Didaktik der Chemie und Physik. Jahrestagung in Aachen 2022

Die Tagung der Gesellschaft für Didaktik der Chemie und Physik (GDCP) fand vom 12. bis zum 15. September 2022 an der RWTH Aachen statt. Der vorliegende Band umfasst die ausgearbeiteten Beiträge der Teilnehmenden zum Thema: "Lernen, Lehren und Forschen in der digital geprägten Welt"