Student engagement with resources as observable signifiers of success in practice based learning

Practice-based learning activities with a focus on Science, Technology, Art, Math and Engineering (STEAM) are providing new opportunities for teaching these subjects. However, we lack widely accepted ways of assessing and monitoring these practices to inform educators and learners and enable the provision of effective support. Here, we report the results from a study with 15 teenage students taking part in a 2-day Hack. We present results from analysis of video data recording collaborative working between groups of students. The analysis of the video data is completed using the ERICAP analytical framework (Luckin et al., 2017) based on ecology of resources and interactive, constructive, active and passive engagement concepts. The results illustrate the differences between students' engagement with resources which might be utilized as signifiers of student success in similar learning environments.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

Machine and Human Observable Differences in Groups’ Collaborative Problem-Solving Behaviours

This paper contributes to our understanding of how to design learning analytics to capture and analyse collaborative problem-solving (CPS) in practice-based learning activities. Most research in learning analytics focuses on student interaction in digital learning environments, yet still most learning and teaching in schools occurs in physical environments. Investigation of student interaction in physical environments can be used to generate observable differences among students, which can then be used in the design and implementation of Learning Analytics. Here, we present several original methods for identifying such differences in groups CPS behaviours. Our data set is based on human observation, hand position ( fiducial marker) and heads direction (face recognition) data from eighteen students working in six groups of three. The results show that the high competent CPS groups spend an equal distribution of time on their problem-solving and collaboration stages. Whereas, the low competent CPS groups spend most of their time in identifying knowledge and skill defi ciencies only. Moreover, as machine observable data shows, high competent CPS groups present symmetrical contributions to the physical tasks and present high synchrony and individual accountability values. The findings have signifi cant implications on the design and implementation of future learning analytics systems.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech. Agencia Estatal de Investigación (AEI) y el Fondo Europeo de Desarrollo Regional (FEDER), TIN2016-80774-R

Learning Bayesian Networks for Student Modeling

In the last decade, there has been a growing interest in using Bayesian Networks (BN) in the student modelling problem. This increased interest is probably due to the fact that BNs provide a sound methodology for this difficult task. In order to develop a Bayesian student model, it is necessary to define the structure (nodes and links) and the parameters. Usually the structure can be elicited with the help of human experts (teachers), but the difficulty of the problem of parameter specification is widely recognized in this and other domains. In the work presented here we have performed a set of experiments to compare the performance of two Bayesian Student Models, whose parameters have been specified by experts and learnt from data respectively. Results show that both models are able to provide reasonable estimations for knowledge variables in the student model, in spite of the small size of the dataset available for learning the parametersUniversidad de Málaga. Campus de Excelencia Internacional Andalucía Tec

The NISPI framework: Analysing collaborative problem-solving from students' physical interactions

Collaborative problem-solving (CPS) is a fundamental skill for success in modern societies, and part of many common constructivist teaching approaches. However, its effective implementation and evaluation in both digital and physical learning environments are challenging for educators. This paper presents an original method for identifying differences in students' CPS behaviours when they are taking part in face-to-face practice-based learning (PBL). The dataset is based on high school and university students' hand position and head direction data, which can be automated deploying existing multimodal learning analytics systems. The framework uses Nonverbal Indexes of Students' Physical Interactivity (NISPI) to interpret the key parameters of students' CPS competence. The results show that the NISPI framework can be used to judge students' CPS competence levels accurately based on their non-verbal behaviour data. The findings have significant implications for design, research and development of educational technology.Agencia Estatal de Investigaci on (AEI) y el Fondo Europeo de Desarrollo Regional (FEDER), TIN2016-80774-R

Using machine learning techniques for architectural design tracking: an experimental study of the design of a shelter

In this paper, we present a study aimed at tracking and analysing the design process. More concretely, we intend to explore whether some elements of the conceptual design stage in architecture might have an influence on the quality of the final project and to find and assess common solution pathways in problem-solving behaviour. In this sense, we propose a new methodology for design tracking, based on the application of data analysis and machine learning techniques to data obtained in snapshots of selected design instants. This methodology has been applied in an experimental study, in which fifty-two novice designers were required to design a shelter with the help of a specifically developed computer tool that allowed collecting snapshots of the project at six selected design instants. The snapshots were described according to nine variables. Data analysis and machine learning techniques were then used to extract the knowledge contained in the data. More concretely, supervised learning techniques (decision trees) were used to find strategies employed in higher-quality designs, while unsupervised learning techniques (clustering) were used to find common solution pathways. Results provide evidence that supervised learning techniques allow elucidating the class of the best projects by considering the order of some of the decisions taken. Also, unsupervised learning techniques can find several common problem-solving pathways by grouping projects into clusters that use similar strategies. In this way, our work suggests a novel approach to design tracking, using quantitative analysis methods that can complement and enrich the traditional qualitative approach.This work has been partially funded by the Spanish Government, Agencia Estatal de Investigaci ́on (AEI), and the European Union, Fondo Europeo de Desarrollo Regional (FEDER), grant TIN2016-80774-R (AEI/FEDER, UE). Funding for open access charge: Universidad de Málaga/CBUA

Using Bayesian networks to improve knowledge assessment

In this paper, we describe the integration and evaluation of an existing generic Bayesian student model (GBSM) into an existing computerized testing system within the Mathematics Education Project (PmatE - Projecto Matemática Ensino) of the University of Aveiro. This generic Bayesian student model had been previously evaluated with simulated students, but a real application was still missing. In the work presented here, we have used the GBSM to define Bayesian Student Models (BSMs) for a concrete domain: first degree equations. In order to test the diagnosis capabilities of such BSMs, an evaluation with 152 students has been performed. Each of the 152 students took both a computerized test within PMatE and a written exam, both of them designed to measure students’ knowledge in 12 concepts related to first degree equations. The written exam was graded by three experts. Then two BSMs were developed, one for the computer test and another one for the written exam. These BSMs were used to obtain estimations of student’s knowledge on the same 12 concepts, and the inter-rater agreement among the different measures was computed. Results show a high degree of agreement among the scores given by the experts and also among the diagnosis provided by the BSM in the written exam and expert’s average, but a low degree of agreement among the diagnosis provided by the BSM in the computer test and expert’s averagePlan Nacional de I+D+i, Gobierno de España. TIN2009-1417