Learning from null results: an analysis of the unexpected findings of a mathematical intervention study conducted in inclusive classrooms

Abstract

This paper presents an additional, more detailed, analysis of the null results and unexpected outcome of a published intervention study. Pfister, Stöckli et al. (2015) report the results of an intervention study in mathematics in inclusive classrooms. In that study the general education teacher implemented an intervention aimed at supporting low achievers in mathematics lessons in 58 inclusive classrooms with 888 third grade students. Two experimental groups (n = 37 teachers) worked with the program, the third group (groupCONTROL; n = 21 teachers) continued to teach ‘as usual’. The experimental groupMAT (n = 16 teachers) was given the program material and the experimental groupMEET (n = 21 teachers) received the material and had two in-service training sessions. Contrary to theory-based predictions, groupMAT outperformed groupMEET. To try to understand this finding, in the present study we investigated the effects of the three treatment conditions on students with different levels of math achievement at t1 by using the data to estimate random slope models with cross-level interactions. Contrary to our expectations, high achieving students in groupMAT had significantly greater learning gains than those in groupMEET. Control variables at the individual and class level could not explain this outcome. The analysis highlighted the methodological challenges of studies conducted in complex inclusive education settings and raised the question of whether targeted support in an inclusive setting, without individual support outside of the classroom, can meet the needs of low achieving students in mathematics