Modelling for understanding AND for prediction/classification - the power of neural networks in research

Two articles, Edelsbrunner and, Schneider (2013), and Nokelainen and Silander (2014) comment on Musso, Kyndt, Cascallar, and Dochy (2013). Several relevant issues are raised and some important clarifications are made in response to both commentaries. Predictive systems based on artificial neural networks continue to be the focus of current research and several advances have improved the model building and the interpretation of the resulting neural network models. What is needed is the courage and open-mindedness to actually explore new paths and rigorously apply new methodologies which can perhaps, sometimes unexpectedly, provide new conceptualisations and tools for theoretical advancement and practical applied research. This is particularly true in the fields of educational science and social sciences, where the complexity of the problems to be solved requires the exploration of proven methods and new methods, the latter usually not among the common arsenal of tools of neither practitioners nor researchers in these fields. This response will enrich the understanding of the predictive systems methodology proposed by the authors and clarify the application of the procedure, as well as give a perspective on its place among other predictive approaches

Predicting general academic performance and identifying the differential contribution of participating variables using artificial neural networks

oai:flr.journals.publicknowledgeproject.org:article/13Many studies have explored the contribution of different factors from diverse theoretical perspectives to the explanation of academic performance. These factors have been identified as having important implications not only for the study of learning processes, but also as tools for improving curriculum designs, tutorial systems, and students’ outcomes. Some authors have suggested that traditional statistical methods do not always yield accurate predictions and/or classifications (Everson, 1995; Garson, 1998). This paper explores a relatively new methodological approach for the field of learning and education, but which is widely used in other areas, such as computational sciences, engineering and economics. This study uses cognitive and non-cognitive measures of students, together with background information, in order to design predictive models of student performance using artificial neural networks (ANN). These predictions of performance constitute a true predictive classification of academic performance over time, a year in advance of the actual observed measure of academic performance. A total sample of 864 university students of both genders, ages ranging between 18 and 25 was used. Three neural network models were developed. Two of the models (identifying the top 33% and the lowest 33% groups, respectively) were able to reach 100% correct identification of all students in each of the two groups. The third model (identifying low, mid and high performance levels) reached precisions from 87% to 100% for the three groups. Analyses also explored the predicted outcomes at an individual level, and their correlations with the observed results, as a continuous variable for the whole group of students. Results demonstrate the greater accuracy of the ANN compared to traditional methods such as discriminant analyses.  In addition, the ANN provided information on those predictors that best explained the different levels of expected performance. Thus, results have allowed the identification of the specific influence of each pattern of variables on different levels of academic performance, providing a better understanding of the variables with the greatest impact on individual learning processes, and of those factors that best explain these processes for different academic levels

The well-fit for the FET model : understanding training transfer factors in Spain

Learning transfer evaluation is a necessary process for practitioners to assess the effectiveness of training, and the outcomes of training produces in workers' behaviors. In this paper, we explore an alternative way to evaluate transfer: through the study of transfer facilitators and barriers. Our aim is to validate the Factors to Evaluate Transfer (FET) model in a large sample of Spanish employees using confirmatory factor analysis. We applied the Spanish version of the FET scale to a sample of 2,745 Spanish workers of public service institutions and private companies. The results show a seven-factor model as the best choice according to the adjustment indices presented in the paper. We obtained a shorter version of the instrument, with adequate construct validity as well as good reliability and internal consistency. This model is a step forward in the measurement of indirect transfer and allows keeping working on the FET model to diagnosis transfer factors and increase the probabilities of higher learning transfer levels.L'avaluació de la transferència de l'aprenentatge és un procés necessari perquè els professionals avaluïn l'eficàcia de la formació i els seus resultats en els treballadors. Aquest article explora una forma alternativa d'avaluar la transferència: a través de l'estudi de facilitadors i obstacles de la transferència. L'objectiu és validar el model FET (factors per avaluar la transferència), en una mostra de treballadors espanyols amb una anàlisi factorial confirmatòria. Es va aplicar l'escala FET en espanyol a una mostra de 2.745 treballadors espanyols de l'Administració pública i l'empresa privada. Els resultats mostren un model de set factors com la millor opció sobre la base dels índexs d'ajust presentats en l'article. Vam obtenir una versió més reduïda de l'instrument, amb una validació de constructe adequada, així com una bona fiabilitat i consistència interna. Aquest model és un pas endavant en la mesura de la transferència indirecta i permet seguir treballant en el model FET per utilitzar-lo com a diagnosi de factors de transferència i augmentar la probabilitat de nivells més alts de transferència de l'aprenentatge.La evaluación de la transferencia del aprendizaje es un proceso necesario para que los profesionales evalúen la eficacia de la formación y sus resultados en los trabajadores. Este artículo explora una forma alternativa de evaluar la transferencia: a través del estudio de facilitadores y obstáculos de la transferencia. Su objetivo es validar el modelo FET (factores para evaluar la transferencia), en una muestra de empleados españoles con un análisis factorial confirmatorio. Se aplicó la escala FET en español a una muestra de 2.745 trabajadores españoles de la Administración pública y la empresa privada. Los resultados muestran un modelo de siete factores como la mejor opción sobre la base de los índices de ajuste presentados en el artículo. Obtuvimos una versión más reducida del instrumento, con una validación de constructo adecuada, así como una buena fiabilidad y consistencia interna. Este modelo es un paso adelante en la medición de transferencia indirecta y permite seguir trabajando en el modelo FET para usarlo como diagnóstico de factores de transferencia y aumentar la probabilidad de mayores niveles de transferencia del aprendizaje

Predicting Effortful Control at 3 Years of Age from Measures of Attention and Home Environment in Infancy: A Machine Learning Approach

Effortful control (EC) is a dimension of temperament that encompass individual differences in self-regulation and the control of reactivity. Much research suggests that EC has a strong foundation on the development of executive attention, but increasing evidence also shows a significant contribution of the rearing environment to individual differences in EC. The aim of the current study was to predict the development of EC at 36 months of age from early attentional and environmental measures taken in infancy using a machine learning approach. A sample of 78 infants participated in a longitudinal study running three waves of data collection at 6, 9, and 36 months of age. Attentional tasks were administered at 6 months of age, with two additional measures (i.e., one attentional measure and another self-restraint measure) being collected at 9 months of age. Parents reported household environment variables during wave 1, and their child’s EC at 36 months. A machinelearning algorithm was implemented to identify children with low EC scores at 36 months of age. An “attention only” model showed greater predictive sensitivity than the “environmental only” model. However, a model including both attentional and environmental variables was able to classify the groups (Low-EC vs. Average-to-High EC) with 100% accuracy. Sensitivity analyses indicate that socioeconomic variables together with attention control processes at 6 months, and self-restraint capacity at 9 months, are the most important predictors of EC. Results suggest a foundational role of executive attention processes in the development of EC in complex interactions with household environments and provide a new tool to identify early markers of socio-emotional regulation development.Spanish State Research Agency (Ref: PSI2017-82670-PPID2020-113996GB-I00)PRE2018-083592Maria ZambranoThe Spanish Government through the European Union NextGeneration EU fund

Individual differences in basic cognitive processes and self-regulated learning:Their interaction effects on math performance

The study analyzes the relationships between working memory capacity, executive attention, and self-regulated learning (srl) on math performance (mp), and more specifically on items with different levels of complexity and difficulty. Sample: 575 university students (female: 47.5%; 18–25 years old), first academic year. Instruments: attention network test; automated operation span; mathematics test; on-line motivation questionnaire, and learning strategies questionnaire. Results confirm the crucial role of individual differences in wmc that impact directly on mp, mediated by subjective competence. Affective srl contribute significantly as mediating variables but their positive effect depends on the availability of cognitive resources. Findings partially confirmed the differential contribution of cognitive processes in the prediction of performance in complex vs difficult items. We found support for a complex pattern of interactions between cognitive processes and components of srl model at the strategy level, in their effect on mp, and given specific item characteristics

Validación de la versión en español de la Escala de remoralización

The Remoralization Scale (RS) was developed in order to measure an individual's state in terms of how the person perceives him or herself in relation to his/her self-concept, self-value, hope, empowerment and positive anticipation. However, there is no data on the psychometric properties of the instrument in a non-clinical population. The aim of this study was to validate a Spanish version of the Remoralization Scale (RS) in a non-clinical sample with a prevention objective and/or the promotion of mental health. The original version of the RS was translated into Spanish and it was applied to a non-clinical sample of 1443 university students in Argentina (18–25 years old). Exploratory and confirmatory factor analyses were performed to study the factorial structure and the validity of the construct. Results suggest that a two factor-model (self-satisfaction and self-concept) results in the best fit for this non-clinical population. The reliability of the total scale and for both sub-scales was moderate to high. Discriminant analysis and contrasting groups analysis showed significant results: the clinical sample and the depression-symptoms sample showed less “remoralization” than the non-clinical sample and a group without depression symptoms, respectively. Results are discussed taking into account previous conceptualizations and studies. In conclusion, the RS was validated for its use in a non-clinical Argentinean population. Regarding its construct validity, a two-factor model with high reliability was obtained.La Escala de Remoralización (RS) fue desarrollada para medir el estado de un individuo en términos de cómo se percibe a sí mismo con relación a su autoconcepto, autovaloración, esperanza, empoderamiento y anticipación positiva. Sin embargo, no hay datos de propiedades psicométricas en la población no clínica. El objetivo de este estudio fue validar una versión española de la RS en una muestra no clínica con un objetivo de prevención y/o promoción de la salud mental. La versión original de la RS fue traducida al español y se aplicó a una muestra no clínica de 1.443 estudiantes universitarios en Argentina (de 18 a 25 años). Se realizaron estudios de análisis factorial exploratorio y confirmatorio para estudiar la estructura factorial y la validez del constructo. Los resultados indican que un modelo de 2 factores (autosatisfacción y autoconcepto) presenta un mejor ajuste para esta población no clínica. La consistencia interna de la escala total y de ambas subescalas fue de moderada a alta. Los análisis discriminantes y análisis de grupos contrastados fueron significativos: la muestra clínica y sujetos con síntomas de depresión mostraron menos «remoralización» que la muestra no clínica y sujetos sin síntomas depresivos. Los resultados se discutieron teniendo en cuenta conceptualizaciones y estudios previos. En conclusión, la RS fue validada para su uso en una población no clínica de Argentina. Respecto a su validez de constructo, se obtuvo un modelo de 2 factores con una alta confiabilidad

Predicting Effortful Control at 3 Years of Age from Measures of Attention and Home Environment in Infancy: A Machine Learning Approach

Effortful control (EC) is a dimension of temperament that encompass individual differences in self-regulation and the control of reactivity. Much research suggests that EC has a strong foundation on the development of executive attention, but increasing evidence also shows a significant contribution of the rearing environment to individual differences in EC. The aim of the current study was to predict the development of EC at 36 months of age from early attentional and environmental measures taken in infancy using a machine learning approach. A sample of 78 infants participated in a longitudinal study running three waves of data collection at 6, 9, and 36 months of age. Attentional tasks were administered at 6 months of age, with two additional measures (i.e., one attentional measure and another self-restraint measure) being collected at 9 months of age. Parents reported household environment variables during wave 1, and their child’s EC at 36 months. A machine-learning algorithm was implemented to identify children with low EC scores at 36 months of age. An “attention only” model showed greater predictive sensitivity than the “environmental only” model. However, a model including both attentional and environmental variables was able to classify the groups (Low-EC vs. Average-to-High EC) with 100% accuracy. Sensitivity analyses indicate that socio-economic variables together with attention control processes at 6 months, and self-restraint capacity at 9 months, are the most important predictors of EC. Results suggest a foundational role of executive attention processes in the development of EC in complex interactions with household environments and provide a new tool to identify early markers of socio-emotional regulation development