Education Can Compensate for Society - a Bit

In this paper I reflect on the findings of a number of loosely related research projects undertaken with colleagues over the last ten years. Their common theme is equity, in formal education and beyond, in wider family and social settings, and with inequity expressed as the stratification of a variety of educational outcomes. The projects are based on a standard mixture of pre-existing records, official documents, large-scale surveys, observations, interviews and focus groups. The numeric data were largely used to create biographical models of educational experiences, and the in-depth data were used to try to explain individual decisions and disparities at each stage of the model. Data have been collected for England and Wales, in five other countries of the European Union and for Japan. A meta-view of these various findings suggests that national school intakes tend to be at least moderately segregated by prior attainment and socio-economic factors, and that learning outcomes as assessed by formal means, such as examinations, are heavily stratified by these same factors. There is no convincing evidence that compulsory schooling does very much to overcome the initial disparity in the resources and attainment of school intakes. On the other hand, there are indications that the nature of a national school system and the social experiences of young people in schools can begin to equalise educational outcomes as more widely envisaged, including learning to trust and willingness to help others, aspirations, and attitudes to continuing in education and training. The cost-free implications of the argument in this paper, if accepted, are that everything possible should be done to make school intakes comprehensive, and that explicit consideration, by teachers and leaders, of the applied principles of equity could reduce potentially harmful misunderstandings in educational contexts

What to do instead of significance testing? Calculating the ‘number of counterfactual cases needed to disturb a finding’

This brief paper introduces a new approach to assessing the trustworthiness of research comparisons when expressed numerically. The ‘number needed to disturb’ a research finding would be the number of counterfactual values that can be added to the smallest arm of any comparison before the difference or ‘effect’ size disappears, minus the number of cases missing key values. This way of presenting the security of findings has several advantages over the use of significance tests, effect sizes and confidence intervals. It is not predicated on random sampling, full response or any specific distribution of data. It bundles together the sample size, magnitude of the finding and the level of attrition in a way that is standardised and therefore comparable between studies

The possible advantages of the mean absolute deviation 'effect' size

A range of ‘effect’ sizes already exists, for presenting a relatively easy to interpret estimate of a difference or change between two sets of observations. All are based on use of the standard deviation of the observations, involving squaring and then square-rooting, which makes results hard to interpret, hard to teach and may distort extreme scores. An effect size based on the simpler mean absolute deviation overcomes these issues to some extent, while being at least as efficient and leading to the same substantive results in almost all cases. This paper proposes the use of an easy to comprehend effect size based on the mean difference between treatment groups, divided by their mean absolute deviation. Using a simulation of 1,656 trials each of 100 cases using a before and after design, the paper shows that the substantive findings from any such trial would be the same whether a traditional effect size like Cohen’s d or the mean absolute deviation effect size is used. The mean absolute deviation effect size works. Among the advantages of using the mean absolute deviation effect size are its relative simplicity, efficiency, everyday meaning, and the lack of distortion of extreme scores caused by the squaring involved in computing the standard deviation. Given that working with absolute values is no longer the barrier to computation that it apparently was before the advent of digital calculators, there is a clear place for the mean absolute deviation effect size

What is the evidence on the impact of Pupil Premium funding on school intakes and attainment by age 16 in England?

The use of targeted additional funding for school-age education, intended to improve student attainment, is a widespread phenomenon internationally. It is slightly rarer that the funding is used to improve attainment specifically for the most disadvantaged students – often via trying to attract teachers to poorer areas, or encouraging families to send their children to school. It is even rarer that funding is used to try and reduce the attainment gap between economically disadvantaged students and their peers, and almost unheard for the funding to be intended to change the nature of school intakes by making disadvantaged students more attractive to schools. These last two were the objectives set for Pupil Premium funding to schools in England. The funding started in 2011, for all state-funded schools at the same time, so there is no easy counterfactual to help assess how effective it has been. The funding is a considerable investment every year and it is therefore important to know whether it works as intended. This paper presents a time series analysis of all students at secondary school in England from 2006, well before the funding started, until 2019, the most recent year for which there are attainment figures. It overcomes concerns that the official attainment gap between students labelled disadvantaged and the rest is sensitive to demographic, economic, legal and other concurrent policy changes. It does this by looking at a stable group of long-term disadvantaged students. It is argued that this group would have attracted Pupil Premium funding if it had existed in any year and under any economic conditions. After 2010, these long-term disadvantaged pupils became substantially less clustered in specific schools in their first year and throughout their remaining school life. This improvement cannot be explained by economic or other factors used in this paper, and so it looks as though the Pupil Premium has been effective here. The picture for the attainment gap at age 16 is more mixed. It is partly confused by changes in the grading of assessments in 2014 and again from 2016. The reasons why the improvements are less clear than at primary school are discussed, and they involve the nature of evidence available to secondary schools to help them improve the attainment of their most disadvantaged students

Ethnic proportionality of teachers and students, and the link to school-level outcomes

In England there are proportionately more White British teachers than White British pupils, and so a mismatch between the proportion of teachers and pupils of each ethnic minority group. This mismatch may reduce appropriate role models for some pupils, and has been linked to differences in school processes, and the behaviour and treatment of ethnic minority pupils. The evidence is weaker on any link between ethnic disproportionality and attainment. This paper uses school-level school workforce and pupil attainment data to assess this link. The results are presented as correlations between teacher/pupil characteristics and attainment scores at ages 11 and 16, and as regression models predicting attainment scores using teacher/pupil characteristics. There is no evidence here that ethnic (dis)proportionality is linked to discernible differences in pupil attainment, once relative poverty is taken into account. However, the data is linked at school rather than individual level, we cannot separate the attainment of pupils of different ethnic origins, and the ethnic classification for teachers is simply binary. We are working to overcome these data limitations, and hope to present future analyses based on individual data with more detailed ethnic groupings, to provide a more definitive result

What should an index of school segregation measure?

The article aims to make a methodological contribution to the education segregation literature, providing a critique of previous measures of segregation used in the literature, as well as suggesting an alternative approach to measuring segregation. Specifically, the paper examines Gorard, Fitz and Taylor's finding that social segregation between schools, as measured by free school meals (FSM) entitlement, fell significantly in the years following the 1988 Education Reform Act. Using Annual Schools Census data from 1989 to 2004, the paper challenges the magnitude of their findings, suggesting that the method used by Gorard et al. seriously overstates the size of the fall in segregation. We make the case for a segregation curve approach to measuring segregation, where comparisons of the level of segregation are possible regardless of the percentage FSM eligibility. Using this approach, we develop a new method for describing both the level and the location of school segregation

The value-added of primary schools: what is it really measuring?

This paper compares the official value-added scores in 2005 for all primary schools in three adjacent LEAs in England with the raw-score Key Stage 2 results for the same schools. The correlation coefficient for the raw- and value-added scores of these 457 schools is around +0.75. Scatterplots show that there are no low attaining schools with average or higher value-added, and no high attaining schools with below average value-added. At least some of the remaining scatter is explained by the small size of some schools. Although some relationship between these measures is to be expected – so that schools adding considerable value would tend to have high examination outcome scores – the relationship shown is too strong for this explanation to be considered sufficient. Value-added analysis is intended to remove the link between a schools’ intake scores and their raw-score outcomes at KS2. It should lead to an estimate of the differential progress made by pupils, assessed between schools. In fact, however, the relationship between value-added and raw scores is of the same size as the original relationship between intake scores and raw-scores that the value-added is intended to overcome. Therefore, however appealing the calculation of value-added figures is, their development is still at the stage where they are not ready to move from being a research tool to an instrument of judgement on schools. Such figures may mislead parents, governors and teachers and, even more importantly, they are being used in England by OFSTED to pre-determine the results of school inspections

Comparing government and private schools in Pakistan: the way forward for universal education

This paper presents an analysis of children’s proficiency in English, reading and maths on the basis of a citizen-led household survey run by the Annual Statistics of Education Report (ASER) in Pakistan in 2014. Our main analysis involves a sub-group of 26,070 children who were reported to be 8 years-old at the time of the survey. It was important for our purposes that this survey collected equivalent data on children in public, private and religious schools, as well as those not attending school at all. Unsurprisingly, the main difference in outcomes is between those children who attend school, and those who do not. Those missing out on school are more likely to be girls, and from poorer families in rural areas. For those who attend school, there are differences between state-funded and private school intakes, in terms of family background and test results. A binary logistic regression analysis is used to help assess the relationship between attending different types of schools and children’s attainment of a specific proficiency level. Once their different student intakes are taken into account, the difference in test outcomes between government and private schools largely disappears. The worst outcomes are associated with the small proportion of children educated only in Madrasahs. The paper ends by proposing that policy-makers press for enforcement of schooling for all, aiming for a universal state-funded system with equivalent opportunities for all, meaning that the stop gap of cheap private schools in poorer areas is no longer necessary

Students' use of Wikipedia as an academic resource — Patterns of use and perceptions of usefulness

Wikipedia is now an established information source in contemporary society. With initial fears over its detrimental influence on scholarship and study habits now subsiding, this paper investigates what part Wikipedia plays in the academic lives of undergraduate students. The paper draws upon survey data gathered from students across two universities in Australia (n = 1658), alongside follow-up group interview data from a subsample of 35 students. Analysis of this data suggests that Wikipedia is now an embedded feature of most students' study, although to a lesser extent than other online information sources such as YouTube and Facebook. For the most part, Wikipedia was described as an introductory and/or supplementary source of information — providing initial orientation and occasional clarification on study topics. While 87.5% of students reported using Wikipedia, it was seen to be of limited usefulness when compared with university-provided library resources, e-books, learning management systems, lecture recordings and academic literature databases. These findings were notably patterned in terms of students' gender, year of study, first language spoken and subject of study