A meta-analysis of the effect of peer feedback on academic achievement in the STEM fields of higher education

Recently, there has been a significant increase in the use of peer feedback in higher education. However, the evidence of the effect of peer feedback on students’ academic achievement does not seem conclusive and, to our knowledge, there has not yet been a meta-analysis of the effect of peer feedback on general academic achievement in the STEM fields of higher education. Therefore, we conducted a meta-analysis to determine whether peer feedback is beneficial to STEM higher education students’ academic achievement. The final data set for the meta-analysis consisted of 286 effect sizes from 90 independent samples in 75 studies, with a total of over 14,000 participants. All effect sizes were calculated as Cohen’s d values. A random-effects model used to synthesise the effect sizes indicated a significant positive summary effect size (d = .421, SE = .037, 95% CI = .350, .493, p = .000). The variance of the true effect sizes (T2) was .069. The Qw value of 644.167 was significant (p = .000) and the I2 value of 88.512 was high. Therefore, in order to identify the source of the between-study heterogeneity, moderator analyses were conducted to evaluate the influence of various methodological quality characteristics and peer feedback intervention characteristics on the effect of a peer feedback intervention. The results of this study will provide researchers, policy makers and practitioners with the information they need to decide whether or not to use peer feedback and to be able to design peer feedback interventions for maximal effectiveness

Equidad en los sistemas educativos de la Unión Europea: Un estudio basado en PISA 2015

Equity in education has recently become a hot topic for international debate and it has gained much interest in the Netherlands as well the last years. In this study, we evaluate and compare equity across the educational systems of European Union member states with a focus on the Dutch context by using PISA 2015 data. PISA 2015 considers inclusive education and fairness as important aspects of equity. Inclusive education is reflected in the segment of students that are 15 years of age and are still in school as well as those students who obtain a basic level to function well in society. Fairness relates to how well countries manage to achieve education outcomes independent of the background characteristics of students. EU countries are compared with one another on these categories using effect sizes derived from differences in PISA scores in science, reading and mathematics. Particular attention is paid to equity results in the Netherlands. Although there is still room for improvement, for many aspects of equity, the Dutch education system scores well when compared to other EU countriesLa equidad en educación se ha convertido recientemente en un tema central para los debates internacionales y en los Países Bajos también ha adquirido mucho interés en los últimos años. En este estudio evaluamos y comparamos la equidad en los sistemas educativos de los Estados miembros de la Unión Europea en la base de resultados de PISA 2015 haciendo hincapié en el contexto neerlandés. PISA 2015 considera la educación inclusiva y la justicia como unos aspectos importantes de la equidad. La educación inclusiva está reflejada en un segmento de estudiantes de 15 años que siguen estudiando en colegio o ya han obtenido un nivel básico de educación para funcionar en la sociedad. La justicia se relaciona con lo como los países consiguen lograr los resultados buenos de enseñanza independientemente de las características del entorno de los estudiantes. Se comparan los países de la UE entre sí, en estas categorías utilizando los tamaños de efecto derivados de las diferencias en los resultados obtenidos en las evaluaciones de PISA en ciencia, lectura y matemáticas. Se presta una atención especial a los resultados de equidad en los Países Bajos. Aunque todavía queda margen de mejora, el sistema educativo neerlandés tiene una buena puntuación comparando con otros países de la UE

Personalized and Automated Feedback in Summative Assessment Using Recommender Systems

In this study we explore the use of recommender systems as a means of providing automated and personalized feedback to students following summative assessment. The intended feedback is a personalized set of test questions (items) for each student that they could benefit from practicing with. Recommended items can be beneficial for students as they can support their learning process by targeting specific gaps in their knowledge, especially when there is little time to get feedback from instructors. The items are recommended using several commonly used recommender system algorithms, and are based on the students' scores in a summative assessment. The results show that in the context of the Dutch secondary education final examinations, item recommendations can be made to students with an acceptable level of model performance. Furthermore, it does not take a computationally complex model to do so: a simple baseline model which takes into account global, student-specific, and item-specific averages obtained similar performance to more complex models. Overall, we conclude that recommender systems are a promising tool for helping students in their learning process by combining multiple data sources and new methodologies, without putting additional strain on their instructors

A best-evidence meta-analysis of the effects of digital monitoring tools for teachers on student achievement

In this study the effects of the use of digital student monitoring tools for teachers (DMTs) on student achievement (primary and secondary schools, mathematics, reading, and language) were investigated through a meta-analysis (n = 14). The studies were also coded for feedback and intervention features, which resulted in three groups of combinations of DMTs and interventions. The meta-analytic findings indicate that the use of a DMT overall has a moderate effect (ES =.12) on student achievement for studies in which student achievement is measured by means of researcher-independent tests. Positive effects were also found for the use of DMTs in primary education (ES =.14), reading (ES =.17), mathematics (ES =.10), and for two groups of DMT-intervention combinations (ES =.25 and.13). Our results are encouraging but should be interpreted with caution, given the small number of studies that met our stringent inclusion criteria

Bayesian covariance structure modelling for measurement invariance testing

In a Bayesian Covariance Structure Model (BCSM) the dependence structure implied by random item parameters is modelled directly through the covariance structure. The corresponding measurement invariance assumption for an item is represented by an additional correlation in the item responses in a group. The BCSM for measurement invariance testing is defined for mixed response types, where the additional correlation is tested with the Bayes factor. It is shown that measurement invariance can be tested simultaneously across items and thresholds for multiple groups. This avoids the risk of capitalization on chance that occurs in multiple-step procedures and avoids cumbersome procedures where items are examined sequentially. The proposed measurement invariance procedure is applied to PISA data, where the advantages of the method are illustrated

dexter: An R Package to Manage and Analyze Test Data

In this study, we present a package for R that is intended as a professional tool for the management and analysis of data from educational tests and useful both in high-stakes assessment programs and survey research. Focused on psychometric models based on the sum score as the scoring rule and having sufficient statistics for their parameters, dexter fully exploits the many theoretical and practical advantages of this choice: lack of unnecessary assumptions, stable and fast estimation, and powerful and sensible diagnostic techniques. It includes an easy to use data management system tailored to the structure of test data and compatible with the current paradigm of tidy data. Companion packages currently include a graphical user interface and support for multi-stage testing

Comparability of Feedback in PISA 2015 across Culturally Diverse Countries

The aim of this study was to investigate the comparability of feedback across culturally diverse countries by assessing the measurement invariance in PISA 2015 data. A multi-group confirmatory factor analysis showed that the feedback scale implemented in PISA 2015 was not invariant across countries. The intercepts and residuals of the factor model were clearly not the same, and the factor loadings also differed. Model fit slightly improved when the more individualism-oriented countries were separated from the more collectivism-oriented countries, but not to an acceptable level. This implies that the feedback results from PISA 2015 have a different meaning across countries, and it is necessary to be careful when making cross-cultural comparisons. However, the absence of measurement invariance did not affect the relationship between feedback and science achievement scores. This means that feedback, as measured by PISA 2015, can be compared across culturally different countries, although the current form of this scale lacks important, culturally specific element

Effects of feedback in a computer-based learning environment on students’ learning outcomes: a meta-analysis

In this meta-analysis, we investigated the effects of methods for providing item-based feedback in a computer-based environment on students’ learning outcomes. From 40 studies, 70 effect sizes were computed, which ranged from −0.78 to 2.29. A mixed model was used for the data analysis. The results show that elaborated feedback (EF; e.g., providing an explanation) produced larger effect sizes (0.49) than feedback regarding the correctness of the answer (KR; 0.05) or providing the correct answer (KCR; 0.32). EF was particularly more effective than KR and KCR for higher order learning outcomes. Effect sizes were positively affected by EF feedback, and larger effect sizes were found for mathematics compared with social sciences, science, and languages. Effect sizes were negatively affected by delayed feedback timing and by primary and high school. Although the results suggested that immediate feedback was more effective for lower order learning than delayed feedback and vice versa, no significant interaction was found

The Holistic and fragmented assessment of oral reading fluency in children attending early primary education: A scoping review protocol

The current scoping review aims to improve the scientific knowledge on oral reading fluency assessment for Latin-based languages, while facilitating the creation of an objective, valid and affordable in-class reading fluency assessment tool that provides sufficient diagnostic insights to optimize personalized learning to read trajectories in the Netherlands