Disentangling general achievement levels and subject-specific strengths and weaknesses in mathematics, reading, and science

The paper uses data from the combined TIMSS (Trends in International Mathematics and Science Study) and PIRLS (Progress in International Reading Literacy Study) assessment in 2011 to explore the subject-specific strengths and weaknesses among fourth grade students worldwide. Previous research came to the conclusion that students only differed in overall achievement levels and did not exhibit subject-specific strengths and weaknesses. This research did, however, not control for differences in overall performance levels when searching for profile differences. Therefore, the present study uses factor mixture analysis to study qualitatively different performance profiles in mathematics, reading, and science while controlling for differences in performance levels. Our findings suggest that the majority of students do not show pronounced strengths and weaknesses and differ mainly in performance levels across mathematics, reading, and science. At the same time, a smaller share of students does indeed show pronounced subject-specific strengths and weaknesses. This result does not represent an artefact, but we find clear and theory-conforming associations between the identified profiles and covariates. We find evidence for cross-country differences in the frequency of subject-specific strengths and weaknesses and gender differences, as well as differences between students who do not or only sometimes speak the language of test at home

Early tracking and different types of inequalities in achievement

Research to date on the effects of between-school tracking on inequalities in achievement and on performance has been inconclusive. A possible explanation is that different studies used different data, focused on different domains, and employed different measures of inequality. To address this issue, we used all accumulated data collected in the three largest international assessments—PISA (Programme for International Student Assessment), PIRLS (Progress in International Reading Literacy Study), and TIMSS (Trends in International Mathematics and Science Study)—in the past 20 years in 75 countries and regions. Following the seminal paper by Hanushek and Wößmann (2006), we combined data from a total of 21 cycles of primary and secondary school assessments to estimate difference-in-differences models for different outcome measures. We synthesized the effects using a meta-analytical approach and found strong evidence that tracking increased social achievement gaps, that it had smaller but still significant effects on dispersion inequalities, and that it had rather weak effects on educational inadequacies. In contrast, we did not find evidence that tracking increased performance levels. Besides these substantive findings, our study illustrated that the effect estimates varied considerably across the datasets used because the low number of countries as the units of analysis was a natural limitation. This finding casts doubt on the reproducibility of findings based on single international datasets and suggests that researchers should use different data sources to replicate analyses

Kooperativ gefördert? Ausgewählte Ergebnisse einer Evaluation des Promotionskollegs der Hans-Böckler-Stiftung

Es fehlt eine aktuelle Studie, die die strukturellen Rahmenbedingungen, Prozessqualität sowie die Effekte und Ergebnisse der von der Hans-Böckler-Stiftung geförderten Promotionskollegs nach einer inzwischen über zehnjährigen Laufzeit in den Blick nimmt. An diesem Forschungsdefizit setzt die … durchgeführte quantitativ und qualitativ angelegte Evaluationsstudie an. Die Autoren stellen im vorliegenden Beitrag einige solcher ausgewählten Ergebnisse vor, von denen sie annehmen, dass sie über den Kontext der Böckler-Kollegs hinaus für die Entwicklung von Graduiertenkollegs im Allgemeinen relevant sein dürften. (DIPF/Orig.

Sambandet mellan läsförståelseresultat i PISA och nationella prov: En fråga om motivation i prov där konsekvenserna är små respektive stora

Denna studie undersöker om det finns belägg för hypotesen att vissa elever inte gör sitt bästa på prov som inte har några konsekvenser för dem. Ett sådant prov är PISA som 15-åringar runt om i världen genomför utan att få några svar på hur väl de har presterat. Trots att PISA-resultaten kan ha en betydande inverkan på utbildningsreformer både i Sverige och globalt, saknas kunskap om hur motivationen påverkar elevernas prestationer. Urvalet bestod av 15-åriga svenska elever som deltog i PISA-provet 2018. Eleverna rapporterade även sin ansträngning på provet. PISA-data kopplades samman med registerdata, vilket möjliggjorde analyser av sambandet mellan PISA-resultat och nationella provbetyg samt hur detta varierade beroende på elevers ansträngningsnivå. Huvudsaklig analysmetod var regressionsanalys. Resultaten visar att svenska elever i stor utsträckning ansträngde sig på PISA-provet, men att de skulle ha ansträngt sig ännu mer om betyget hade påverkats. Högt skattad ansträngning var positivt relaterad till bättre PISA-resultat, även efter kontroll av det nationella provbetyget i läsförståelse. Dessutom framkom att mer motiverade elever uppvisade ett starkare samband mellan sina PISA-resultat och nationella provbetyg. Resultaten tyder på att en mindre grupp elever underpresterade relativt sina nationella provresultat. Studien betonar vikten av att undersöka och förstå faktorer som påverkar sambandet mellan prov med små konsekvenser och prov med stora konsekvenser. English abstract The Relationship Between Reading Achievement in PISA and National Tests: A Question of Test Motivation in Low-Stakes Versus High-Stakes Assessments The following study examines whether there is evidence for the hypothesis that students do not do their best on low-stakes tests. One such test is PISA, which 15-year-olds around the world take without receiving any feedback about how they performed. Despite the significance of PISA results for educational reforms in Sweden and globally, there is limited knowledge about the impact of motivation on student performance. The sample comprised 15-year-old Swedish students who participated in the 2018 PISA assessment. The students also reported their effort on the PISA test. PISA data was linked with registry data, enabling analyses of the relationship between PISA results and national test grades, and how this varied depending on the students’ effort levels. Regression analysis was the main method of analysis. While most students exerted significant effort on the PISA assessment, they indicated a greater willingness to exert even more effort if the results had consequences for their grades. Higher effort was positively associated with better PISA performance, even after controlling for national test scores in reading comprehension. Additionally, it was found that more motivated students showed a stronger correlation between their PISA results and national test grades. The results suggest that a smaller group of students underperformed relative to their national test scores. These results emphasize the need to understand factors contributing to differential performance between low- and high-stakes tests

Can summer camps reduce educational inequality?

Die Autoren stellen die Ergebnisse der Evaluierung eines Bochumer Sommercamps vor. Bildungsungleichheit entsteht nicht zuletzt durch ein fehlendes Anregungspotenzial benachteiligter Kinder während der Schulferien. Somit bieten Sommercamps eine Interventionsmöglichkeit. Die Evaluationsstudie zeigt, dass es gelingt, förderbedürftige Kinder für das Camp zu rekrutieren. Nach dem Camp beobachten wir eine höhere Lesekompetenz und die Kinder berichten eine hohe Zufriedenheit mit dem Camp. (DIPF/Orig.)The authors present findings from the evaluation study on a summer camp in Bochum. One source of educational inequality is that students from disadvantaged families suffer from missing learning opportunities during the summer holidays. Therefore, summer camps might be an opportunity to support these children at risk. The evaluation study reveals that the camp succeeds in recruiting disadvantaged students. After the camp the students achieve higher scores in reading literacy, and they report high satisfaction with the camp. (DIPF/Orig.

Knut Schwippert (Ed.). (2007). Progress in Reading Literacy. The Impact of PIRLS 2001 in 13 Countries. Münster: Waxmann. 280 pages. [Rezension]

Rezension zu: Knut Schwippert (Hrsg.) (2007): Progress in Reading Literacy. The Impact of PIRLS 2001 in 13 Countrie

Student achievement and educational inequality in half- and all-day schools. Evidence from Germany

Several countries have expanded extended education in recent years. In Germany, the most substantial educational reform is the ongoing transformation of the traditional half-day school system into an all-day school system. Among politicians, expectations are high that all-day schools will promote student achievement and reduce social achievement inequalities. To test these assumptions, we used representative data from the National Educational Panel Study (NEPS) to estimate two-level latent growth models for achievement in grades 5, 7, and 9. The analyses revealed initial achievement differences but no differences in achievement growth or changes in inequality throughout secondary school. This suggests that selection mechanisms are at work but that half- and all-day schools are not differentially effective. We discuss these findings in light of the international debate on the quality of extended education. (DIPF/Orig.

Referendare beurteilen. Eine explorative Analyse von Beurteilungsinstrumenten in der Zweiten Phase der Lehrerbildung

Die Frage nach den Möglichkeiten einer zuverlässigen Beurteilung der beruflichen Kompetenzen von Lehrerinnen und Lehrern wird innerhalb der wissenschaftlichen Fachdiskussion, aber auch in den Lehrerverbänden und in der Öffentlichkeit kontrovers diskutiert. Dabei wird häufig übersehen, dass die Beurteilung von (angehenden) Lehrkräften seit Jahrzehnten stattfindet: im Rahmen der Zweiten Ausbildungsphase, dem Referendariat oder Vorbereitungsdienst. Nach einer kurzen Übersicht über neuere Forschungsergebnisse zur Situation der Zweiten Phase wird über ein Forschungsprojekt berichtet, in dem die in der Zweiten Phase von Ausbildern (Fachleitern, Hauptseminarleitern) verwendeten Beurteilungsschemata und -manuale analysiert worden sind. Auf der Basis von 201 Beurteilungsschemata und 87 zusätzlicher Fragebögen zur Entstehung und zur Verwendung der Schemata wird eine Übersicht über die Praxis des Beurteilens im Referendariat gegeben. Die Ergebnisse machen u.a. deutlich, dass dem Großteil der Instrumente ein auf Unterrichten konzentriertes Bild der Lehrertätigkeit zugrunde liegt, die Breite der Standards für die Lehrerbildung also nicht abgebildet wird. In jedem zweiten Instrument bleibt unklar, wie der Beurteilungsakt im Einzelnen vollzogen wird. Die testdiagnostische Qualität der Instrumente ist in keinem Fall geprüft worden. Insgesamt wird deutlich, dass hinsichtlich der Qualität der Beurteilungsprozesse und -instrumente, die in der Zweiten Phase der Lehrerbildung eingesetzt werden, ein dringender Entwicklungsbedarf besteht. (DIPF/Orig.)The issue of the possibilities of a reliable assessment of the professional competences of teachers is being discussed controversially not only in the scientific debate but also within teacher associations and among the public. This debate often neglects the fact that an assessment of (prospective) teachers has been taking place for decades: namely within the framework of the second phase of training, the "Referendariat" (teacher training) or teaching practice. Following a short survey of recent research results regarding the present situation in the "second phase", the authors report on a research project analyzing the assessment patterns and manuals used by instructors (heads of departments, heads of seminars for advanced students) during the "second phase". On the basis of 201 assessment patterns and 87 additional questionnaires on the development and application of these patterns, a survey is given on the practice of assessment during the "Referendariat" (teacher training). The results show, among other things, that a large part of these instruments is based on a concept of the teaching profession focussing on instruction, meaning that the breadth of the standards for teacher training is not depicted. With every second instrument it remains unclear how the individual act of assessment is actually carried out. In none of the cases was the test-diagnostic quality of the instruments actually verified. All in all, it becomes clear that with regard to the quality of the processes and instruments of assessment applied during the second phase of teacher training there exists an urgent need for further development. (DIPF/Orig.

Effects of afternoon education programs on language achievement developments

Der flächendeckende Ganztagsschulausbau wurde Anfang der 2000er Jahre initiiert, um Missständen im deutschen Schulsystem entgegenzuwirken. Zu diesem Zeitpunkt gab es jedoch keine empirische Evidenz zu Effekten auf Leistungsniveaus oder Bildungsungleichheiten. Daher untersucht die vorliegende Studie, ob damals bestehende, lernförderlich ausgerichtete Nachmittagsangebote einen Einfluss auf Leistungsmaße hatten. Hierzu nutzen wir Daten der repräsentativen Längsschnittstudie DESI, in der die Entwicklung sprachlicher Kompetenzen in Deutsch und Englisch im Verlauf der neunten Jahrgangsstufe untersucht wurden. Unsere Ergebnisse bieten keine Evidenz dafür, dass zu Beginn des Angebotsausbaus davon ausgegangen werden konnte, dass Nachmittagsangebote das Leistungsniveau oder soziale Bildungsungleichheiten beeinflusst hätten. (DIPF/Orig.)The massive expansion of all-day schooling was initiated in the early 2000s to improve educational outcomes in the German school system. Empirical evidence on the effects of all-day schooling on achievement levels or educational inequalities did not exist at this time. Against this background, the present study uses data from the early 2000s to investigate the effects of all-day schooling on educational outcomes. We use data from the representative longitudinal DESI study that assessed student performance in German and English as a second language in the course of the ninth grade. Our results do not support the assertion that afternoon programs affected achievement levels or educational inequalities at the beginning of the expansion of all-day schooling in Germany. (DIPF/Orig.