Lost in transition – Learning analytics on the transfer from knowledge acquisition to knowledge application in complex problem solving

Since Complex Problem Solving (CPS) skills represent a key competence for educational success, they are of great relevance for learning analytics. More specifically, CPS serves as a pertinent showcase for addressing a crucial existing gap contemporary education is facing, the gap between students' ability to acquire and subsequently apply knowledge in uncertain situations, which are increasingly important in the 21st century. While the CPS process incorporates both the acquisition and application of knowledge, many earlier studies have focused on identifying the factors relevant for success in knowledge acquisition. Given the dearth of existing research on factors influencing a successful transition between both CPS phases, we investigated the rates of successful and unsuccessful knowledge transition over the course of nine CPS items in a sample of N = 1151 students in 9th grade. Results showed that many participants were unable to transition their knowledge from the acquisition to the application phase, which was presumably due to an inefficient mental model transfer. Furthermore, the likelihood of students being ‘lost in transition’ was higher in more complex items. Implications are discussed in light of learning analytics, and particularly with regard to the factors to be taken into account by future CPS training programs

Using Log-File Data to Uncover Strategy Use in Complex Problem Solving

The aim of the current project is to utilize log-file data to enhance the understanding of strategy use in Complex Problem Solving (CPS). CPS can be defined as the ability to manipulate the existing variables of a particular novel, complex, intransparent, and dynamic environment successfully in order to reach a predefined goal. In order to successfully solve a CPS task, the goal-directed application and variation of certain strategies is necessary. To uncover the systematic use of these strategies, log-files have been deemed a fruitful resource as they contain not only the final results of a computer-based CPS item, but also the individual steps undertaken while solving such a task. Recent studies using log-files have highlighted the importance of some strategies for CPS, like the varying one variable at a time (VOTAT) strategy, or of engaging in noninterfering observations (i.e., idle rounds). In addition, other strategies have also been shown to be relevant in neighboring fields of CPS. However, comprehensive studies investigating a broad repertoire of strategies applied in CPS are scarce. Hence, on the basis of existing large-scale assessment data sets, the present project set out to investigate which strategies hidden in log-files are relevant for solving a CPS task in order to gain a more thorough understanding of how CPS tasks are being approached. Preliminary findings indicate that, in addition to VOTAT and idle rounds, particularly the flexibility of switching between strategies should be taken into account