Oketch, M

Rolleston, C

Rossiter, J

English

UCL Discovery

1  Teacher value-added with Ethiopia school survey data RISE Blog. August, 2019 By Moses Oketch (UCL), Caine Rolleston (UCL), Jack Rossiter (CGD) In Ethiopia, as elsewhere, teachers are expected to deliver the grade-specific curriculum, set out in textbooks and teachers guides. The curriculum is designed to be cumulative in grades, with higher-order skills building on their respective foundations. However, due to slow learning progress in early primary grades, language-based transitions in upper primary grades and automatic promotion policies, many children reach grade seven or eight lacking prerequisite skills. Learning progress among these students is arguably limited by the mis-match between curriculum content and their skills base.  Some teachers will be able to make adaptations in the classroom, to account for the gap between student skills and curriculum expectation. This can include providing extra support to early-low-achievers, so that they catch up and go on to learn the grade’s content. Other teachers will be less able to make such adaptations. This can be on account of limited pedagogical skills, limited awareness of students’ skills owing to large classes, especially where there is a very heterogeneous range of skills among the incoming cohort.  Teachers may feel it is futile trying to adapt in face of the overriding need to ‘complete the curriculum’. In such circumstances, teachers who do not or cannot adapt may be most effective for students with high early achievement, who may be high in ability and ‘school readiness’ linked to their backgrounds and whose learning levels are closer to the curriculum content; but far less effective for previously lower achieving students in the same classroom. Conversely, it may be that teachers who do make adaptations to accommodate a majority of students and ‘teach to the right level’ in that respect, may do so at the cost of limiting progress among higher achievers.  Data from Young Lives school surveys in Ethiopia may be employed to examine these possible scenarios. The surveys collect achievement data in mathematics at the beginning and end of the school year for pupils in grades 7 and 8 (the last two grade of upper primary schooling), measuring attainment on a common scale. There is some indication in Ethiopia of school-level differences in the relationships between beginning-of-year and end-of-year scores, consistent with the scenarios considered. Figure 1 below present a scatter-plot of mathematics test scores at the beginning (x-axis1) and end (y-axis) of the school year and includes a linear regression line of best fit.  School A illustrates a situation where substantial progress is made by students with above average prior achievement and limited progress for students with below average prior achievement. By contrast, School B illustrates a situation where the group of students is more homogenous (in terms of prior achievement). In this school greater progress is made by students with below average prior achievement – perhaps at the cost of high-early-achievers’ progress.   Figure 1: Two schools, showing scatter of prior (X) and post (Y) scores, lines of fit for school and for sample. School A                                                           1 Beginning of the year test scores are centred on a mean of 500 and have a standard deviation fixed at 100.  They are estimated from raw scores using item response modelling (IRT).  End of the year test scores are reported on the same scale for direct comparability. 2   School B  Source:  Young Lives Ethiopia School Survey 2016-17  These relationships draw attention to the concepts of teacher ‘effectiveness’ and ‘value-added’, raising questions about (i) who teachers are effective for, as classrooms are not groups of homogenous average learners; and (ii) whether ‘apparent’ teacher effectiveness is influenced by the prior learning levels, or by heterogeneity in prior learning levels, among the students that a teacher faces.  While such questions are familiar in developed countries, few developing countries have data which allows their exploration and the situations faced by teachers in developing countries likely make the issues significantly more pressing.   School and teacher effectiveness studies in the literature for developing countries rarely attend to issues of differential effectiveness, not least owing to lack of data.  But apparent differential effectiveness as illustrated in the graphs above may be the result of (i) teachers’ pedagogical approaches, for example in adapting the curriculum or (ii) the composition of the student group (for example it may be ‘easier’ for teachers to be effective for students with advantaged backgrounds) or 3  some combination of (i) and (ii) which includes the teachers’ response to classroom composition.  Each of these possibilities seems likely to play a role in the Ethiopian context – where classes are large and often diverse and where teacher training and expertise also varies relatively widely.   Simple estimates of school and teacher effectiveness, generally based on the teacher-level residual from a regression equation - that is teacher ‘value-added’ - represent a ‘black box’ measure of progress made by students taught by a particular teacher. These are sometimes conditioned on students’ prior achievement and sometimes their backgrounds but the estimates conceal a potentially wide range of scenarios in terms of who is making progress and for what reason (pedagogically, for example).  This can raise important equity questions.  For example, high-early-achievement might indicate ‘preparedness’ to learn which is the result of a supportive home environment and attendance at pre-school. Progress among these students may be the result of this as well as of teacher strategies to ‘extend’ the learning of high achievers.  If progress among this group is large enough and especially if this group represents a large share of learners in a class, their progress may allow a teacher to present as ‘effective’, on average, even though a (possibly large) number of students makes little progress.  And what if progress is highest among low-early-achievers but modest among those with high prior achievement? Even if overall effectiveness is lower, a classroom in which the mean achievement at the end of the year depends little on a student’s beginning of year score may arguably be said to represent a more equitable situation because it has decreased differences in outcomes across its intake spectrum. Again, arguably, the students most in need of progress have benefitted more from educational effectiveness in this situation. Further, perhaps these classrooms better represent environments in which teacher effectiveness/value added is directed towards the achievement of universal basic skills. Nonetheless, in classrooms with extreme achievement heterogeneity – such as in Ethiopia Grade 7/8 where Young Lives data indicate that variation in prior achievement can be as much as five or more equivalent years of schooling in one classroom – it may be instructive to review teacher effectiveness for different sub-groups and in relation to classroom composition. This idea of differential effectiveness based on prior achievement – or on the distribution of prior achievement in a classroom is linked to ‘Teaching at the Right Level’ (TaRL) literature. TaRL research indicates large gains when students are grouped according to skills rather than based on age or number of grades completed. Typical TaRL approaches include some adjustment in the content that teachers deliver and the processes that they use but they rely on substantial adjustment in the matching between content and student capability. This suggests that there may be ‘untapped effectiveness’ among teachers insofar as a (simple) reallocation of students to classes produces large learning gains, without much else (i.e.  although there may be some orientation on how to deliver the new material, there is no teacher replacement, retraining or major resource addition). The TaRL approach may hold the greatest potential in a context of curriculum/skills mismatches, such as might exist in Ethiopia’s upper-primary grades. In addition to the issues of TaRL, differential value-added analysis may be able to inform related issues in Ethiopia as mentioned briefly here, namely (i) diagnosis of the ‘overambitious’ curricula problem and (ii) questions relating to how students should be grouped in schools for efficient and equitable learning progress.   Research along these lines may shed some light on whether Ethiopia’s teachers are ‘ineffective’ for many students because they lack skills or motivation to teach or whether they are ‘ineffective’ for many students because they are being asked to teach material that most children are under-prepared to learn. What if teachers are ‘ineffective’ simply because the students they are tasked with teaching are under-prepared? Is there evidence that teachers might be more effective where 4  the student group is homogenous and/or closer to the level of the curriculum? How then does variation in teacher effectiveness relate to variation in prior achievement? Using the Ethiopia school survey data it will be possible for us to investigate some of these relationships for classes and/or teachers and to advance our understanding about how the heterogeneity of students’ skills in a classroom affect observed teacher value added.  

Teacher value-added with Ethiopia school survey data

https://discovery.ucl.ac.uk/10093001/1/Teacher%20value%20added-%20RISE%20BLOG%20Revised-%20FINAL.pdf

Teacher value-added with Ethiopia school survey data

Abstract

Similar works

Full text

Available Versions

UCL Discovery