Evaluation of Year 1 of the Academic Mentoring Programme: Impact Evaluation for Year 11. Evaluation Report: An exploration of impact in Year 11

The National Tutoring Programme (NTP) Academic Mentoring (AM) programme (2020/21) was designed to help disadvantaged pupils ‘catch up’ on missed learning by providing trained academic mentors to deliver one to one and small group tutoring in schools. This evaluation covers year 1 of the AM programme as delivered by Teach First from November 2020 to July 2021 (delivery was in three waves starting 26th October 2020, 15th January 2021 and 22nd February 2021). AM was one arm of the NTP. The NTP aimed to support teachers and schools in providing a sustained response to the Covid-19 pandemic and to provide a longer -term contribution to closing the attainment gap between disadvantaged pupils and their peers. The NTP was part of a wider government response to the pandemic, funded by the Department for Education (DfE) and was originally developed by the Education Endowment Foundation (EEF), Nesta, Impetus, The Sutton Trust, Teach First, and with the support of the KPMG Foundation. The DfE appointed Teach First to manage the provision of mentors (referred to as ‘academic mentors’) to schools; recruiting, training and placing them in schools. The mentor worked in the school setting as an employee of the school. It was expected that each academic mentor would work with at least 50 pupils between the date they started in school and the end of the academic year. Mentoring was provided online and/or face-to-face; and was one to one, or in groups of 2-4 pupils; and available in English/literacy, maths, science, humanities, and modern foreign languages. Mentoring was expected to be delivered in schools during normal teaching time, as well as before or after school. In certain circumstances, mentoring could be delivered online with pupil(s) at home. The AM programme was targeted at state-maintained primary and secondary schools serving disadvantaged populations. 89% of the schools met Teach First’s priority criteria, which is based on the proportion of children living in income deprived families (IDACI) and whether the school is in an area of chronic and persistent underperformance (AEA). The remaining 11% of schools had an above average proportion of pupils eligible for Pupil Premium (Teach First, 2021). Participating schools could decide which pupils received support from academic mentors. However, the programme encouraged them to select pupils from disadvantaged households or those whose education had been disproportionately impacted by Covid-19. Pupils in Years 1–11 were eligible (5–16 years old). The programme aimed to reach a minimum of 900 schools and 50,000 children, with 1,000 academic mentors. By the end of February 2021, it had surpassed targets having trained and placed 1,124 academic mentors in 946 schools and delivered mentoring sessions to 103,862 pupils, 49% of whom were identified by mentors as being eligible for Pupil Premium of Free School Meals (FSM), and 23% of whom were identified as having a special educational need or disability. The AM programme was initiated and delivered at a time of great pressure for schools when the education system had been disrupted by a series of school closures to most pupils and was contending with ongoing widespread pupil and staff absences. Covid-19 related issues disrupted the anticipated operation of academic mentoring during the year. The AM programme involved initial training and ongoing support from Teach First as intended but there was greater variation in schools’ deployment of mentors during the latter stages of the Autumn Term 2020/21, and during the January to March 2021 period of school closures to most pupils. This evaluation report presents the analysis of the impact of the AM programme on maths and English attainment outcomes for Year 11 pupils only—who represent a very small proportion of individuals targeted by the AM programme. Originally, it was planned to evaluate impact across all year groups (Years 1 – 11) at primary and secondary level using schools’ standardised assessment data from Renaissance Learning (RL) assessments and, in addition, to evaluate the impact for Year 6 pupils using Key Stage (KS) 2 data. However, these analyses could not go ahead as KS2 assessments were cancelled in summer 2021 (related to the ongoing Covid-19 pandemic) and because the number of schools providing agreement to use their RL data was insufficient to warrant impact analyses. Data was only available for pupils in Year 11. Since GCSEs could not go ahead as planned in 2021, the data was in the form of Teacher Assessed Grades (TAGs), which had not previously been used as an outcome measurement tool. Checks were therefore undertaken to explore if TAGs would be suitable as an outcome measure. The only analysis that could proceed was therefore exploratory. The evaluation uses a quasi-experimental design (QED), in which a group of secondary schools and Year 11 pupils who did not receive the AM programme were selected for comparison with schools and pupils who received the AM programme. Comparison schools were selected by matching schools that were similar in important, observable regards to the schools that participated in AM. The evaluation included analysis on the availability of AM for pupils who were eligible for Pupil Premium (a key focus of the overall NTP), and all pupils, as these groups could be identified for both the AM and non-AM schools. In addition, the evaluation aimed to analyse the impact on pupils who received AM by predicting their participation and identifying a comparison group of pupils with similar characteristics. Analysis was based on data about Year 11 pupils’ attainment and characteristics from the National Pupil Database (NPD) merged with data provided by Teach First about pupils’ participation in AM. In total, 159 AM schools (8,977 Year 11 pupils eligible for Pupil Premium) and an equal number of comparison schools (8,419 Year 11 pupils eligible for Pupil Premium) were included in the final analysis. The evaluation assessed impact in English and maths using Teacher Assessed Grades (TAGs) from 2021. Where appropriate, this impact evaluation refers to important implementation features from the implementation and process evaluation (IPE) conducted by Teach First themselves. However, there is no independent IPE data to draw on in the interpretation of the impact results. Of the Year 11 pupils selected for Academic Mentoring in this evaluation, 46% of them were eligible for Pupil Premium, however, despite this it is important to note that the number of Year 11 Pupil Premium-eligible pupils selected for AM in AM schools was small as a proportion of all Year 11 Pupil Premium-eligible pupils, and the number of these Year 11 Pupil Premium-eligible pupils receiving AM in maths and/or English (as opposed to other subjects), was smaller still. The same is the case when considering the whole year group of Year 11 pupils – the number receiving AM was small as a proportion of all Year 11 pupils. This means that in the analysis, the number of Year 11 pupils who actually received AM in maths and/or English was heavily ‘diluted’ by the number of pupils who did not. The primary impact findings must be therefore treated with a high degree of caution. The analysis was subject to very high dilution; a large proportion of the pupils eligible for Pupil Premium included in the analysis in AM schools were not selected for AM. This was due to limited programme reach and a tendency for teachers to allocate both non-Pupil Premium and Pupil Premium eligible pupils to the programme. This dilution means that, in order to detect an effect, either the effect would need to be very strong amongst the very small proportion of Year 11 pupils eligible for Pupil Premium who were selected for mentoring (and there was no indication that this was the case elsewhere in our analysis), and/or there would need to be strong spillover effects amongst the rest of the Year 11 pupils eligible for Pupil Premium. Although the programme Theory of Change includes such a mechanism, it is unlikely to be relevant at the dilution levels seen. With such high dilution, it is hard to detect whether AM had an effect on those who received mentoring in the analyses focusing on pupils eligible for Pupil Premium and on all pupils. It is not possible to conclude whether a lack of observed impact is due to the small proportion of disadvantaged pupils who received mentoring, or because AM did not work for those who received it. An additional challenge was that it was not possible to construct a comparison group of similar Year 11 pupils in nonAM to schools to those who received mentoring in AM schools, based on observable, pupil-level characteristics, and this impact analysis did not go ahead. Schools used information such as classroom assessments to select pupils into the programme that was not observable in the available datasets, suggesting that pupil-level selection was driven by unobserved dimensions. These constraints, both of very high dilution and not being able to identify a comparison group with similar pupil characteristics, mean that the evaluation is unable to conclude, with any certainty, whether or not AM had an impact on the English or mathematics attainment outcomes of those pupils who received it. The report must be considered in the light of these caveats. Year 11 pupils eligible for Pupil Premium in schools that received AM made, on average, similar progress in English compared to Year 11 pupils eligible for Pupil Premium in comparison schools (there was no evidence of an effect). In maths, Year 11 pupils eligible for Pupil Premium in schools that received AM made, on average, slightly more progress (equivalent to 1 months’ additional progress) compared to Year 11 pupils eligible for Pupil Premium in comparison schools. However, there is uncertainty around this result; it is also consistent with a null (0 months) effect or an effect of slightly larger than 1 month’s additional progress. A particular challenge in interpretation is that, on average, only 13% of Year 11 pupils eligible for Pupil Premium were selected for mentoring by schools, and only 4.2% of Year 11 pupils eligible for Pupil Premium were selected for mentoring in maths and 2.9% in English, meaning that the vast majority of pupils eligible for Pupil Premium included in the analysis did not receive mentoring. Therefore, this estimated impact of AM is severely diluted and it is unlikely any of these differences were due to AM. When looking at all Year 11 pupils, pupils in schools that received AM made, on average, similar progress in English and maths compared to all Year 11 pupils in comparison schools (there was no evidence of an effect). However, this finding was similarly subject to severe dilution: on average only 10% of Year 11 pupils in the analysed schools were selected for mentoring, with 3.4% in maths and 2.1% in English, and therefore it is hard to detect any effect that may (or may not) have been present. Within schools that offered AM to Year 11 pupils, there was no association between the number of completed mentoring sessions in maths and Year 11 outcomes in maths, or between the number of completed mentoring sessions in English and Year 11 outcomes in English. These results are associations and not necessarily causal

Tunneling and the Band Structure of Chaotic Systems

We compute the dispersion laws of chaotic periodic systems using the semiclassical periodic orbit theory to approximate the trace of the powers of the evolution operator. Aside from the usual real trajectories, we also include complex orbits. These turn out to be fundamental for a proper description of the band structure since they incorporate conduction processes through tunneling mechanisms. The results obtained, illustrated with the kicked-Harper model, are in excellent agreement with numerical simulations, even in the extreme quantum regime.Comment: 11 pages, Latex, figures on request to the author (to be sent by fax

Chaotic maps and flows: Exact Riemann-Siegel lookalike for spectral fluctuations

To treat the spectral statistics of quantum maps and flows that are fully chaotic classically, we use the rigorous Riemann-Siegel lookalike available for the spectral determinant of unitary time evolution operators $F$. Concentrating on dynamics without time reversal invariance we get the exact two-point correlator of the spectral density for finite dimension $N$ of the matrix representative of $F$, as phenomenologically given by random matrix theory. In the limit $N\to\infty$ the correlator of the Gaussian unitary ensemble is recovered. Previously conjectured cancellations of contributions of pseudo-orbits with periods beyond half the Heisenberg time are shown to be implied by the Riemann-Siegel lookalike

Spectral Analysis of the Supreme Court

The focus of this paper is the linear algebraic framework in which the spectral analysis of voting data like that above is carried out. As we will show, this framework can be used to pinpoint voting coalitions in small voting bodies like the United States Supreme Court. Our goal is to show how simple ideas from linear algebra can come together to say something interesting about voting. And what could be more simple than where our story begins— with counting

Quantum Chaotic Dynamics and Random Polynomials

We investigate the distribution of roots of polynomials of high degree with random coefficients which, among others, appear naturally in the context of "quantum chaotic dynamics". It is shown that under quite general conditions their roots tend to concentrate near the unit circle in the complex plane. In order to further increase this tendency, we study in detail the particular case of self-inversive random polynomials and show that for them a finite portion of all roots lies exactly on the unit circle. Correlation functions of these roots are also computed analytically, and compared to the correlations of eigenvalues of random matrices. The problem of ergodicity of chaotic wave-functions is also considered. For that purpose we introduce a family of random polynomials whose roots spread uniformly over phase space. While these results are consistent with random matrix theory predictions, they provide a new and different insight into the problem of quantum ergodicity. Special attention is devoted all over the paper to the role of symmetries in the distribution of roots of random polynomials.Comment: 33 pages, Latex, 6 Figures not included (a copy of them can be requested at [email protected]); to appear in Journal of Statistical Physic

Enhanced flight performance by genetic manipulation of wing shape in Drosophila

Insect wing shapes are remarkably diverse and the combination of shape and kinematics determines both aerial capabilities and power requirements. However, the contribution of any specific morphological feature to performance is not known. Using targeted RNA interference to modify wing shape far beyond the natural variation found within the population of a single species, we show a direct effect on flight performance that can be explained by physical modelling of the novel wing geometry. Our data show that altering the expression of a single gene can significantly enhance aerial agility and that the Drosophila wing shape is not, therefore, optimized for certain flight performance characteristics that are known to be important. Our technique points in a new direction for experiments on the evolution of performance specialities in animals

Wing shape patterns among urban, suburban, and rural populations of Ischnura elegans (Odonata: Coenagrionidae)

Dragonflies and damselflies (the Odonata) are among the most efficient flying insects. However, fragmentation of the landscape can increase distance between habitats and affect costs of dispersal, thus shaping phenotypic patterns of flight-related traits, such as wing shape, wing loading and wing size. Urban landscapes are highly fragmented, which limits dispersal among aquatic habitats. Hence, strong selective pressures can act upon urban populations in favour of individuals with increased flight performance or may lead to the reduction in dispersal traits. Here, we explore differentiation in morphological flight-related traits among urban, suburban, and rural populations of the damselfly Ischnura elegans, which is one of the most abundant species in both urban and rural ponds in Europe. We sampled 20 sites across Leeds and Bradford, UK, in an urban-to-rural gradient from June to August 2014 and 2015 (Nmales = 201, Nfemales = 119). We compared wing shape among different land use types using geometric morphometrics. Other wing properties analysed were wing aspect ratio, wing loading and wing centroid size. Unexpectedly, no significant effect of urban land use was found on wing shape. However, wing shape differed significantly between males and females. Additionally, females showed significantly larger wing centroid sizes (P < 0.001), increased wing loading (forewings: P = 0.007; hind wings: P = 0.002) and aspect ratio (P < 0.001) compared to males across all land use types. Possible mechanisms driving these results are further discussed

Geometry of Polynomials and Root-Finding via Path-Lifting

Using the interplay between topological, combinatorial, and geometric properties of polynomials and analytic results (primarily the covering structure and distortion estimates), we analyze a path-lifting method for finding approximate zeros, similar to those studied by Smale, Shub, Kim, and others. Given any polynomial, this simple algorithm always converges to a root, except on a finite set of initial points lying on a circle of a given radius. Specifically, the algorithm we analyze consists of iterating $$z - \frac{f(z)-t_kf(z_0)}{f'(z)}$$ where the $t_k$ form a decreasing sequence of real numbers and $z_0$ is chosen on a circle containing all the roots. We show that the number of iterates required to locate an approximate zero of a polynomial $f$ depends only on $\log|f(z_0)/\rho_\zeta|$ (where $\rho_\zeta$ is the radius of convergence of the branch of $f^{-1}$ taking $0$ to a root $\zeta$) and the logarithm of the angle between $f(z_0)$ and certain critical values. Previous complexity results for related algorithms depend linearly on the reciprocals of these angles. Note that the complexity of the algorithm does not depend directly on the degree of $f$, but only on the geometry of the critical values. Furthermore, for any polynomial $f$ with distinct roots, the average number of steps required over all starting points taken on a circle containing all the roots is bounded by a constant times the average of $\log(1/\rho_\zeta)$. The average of $\log(1/\rho_\zeta)$ over all polynomials $f$ with $d$ roots in the unit disk is ${\mathcal{O}}({d})$. This algorithm readily generalizes to finding all roots of a polynomial (without deflation); doing so increases the complexity by a factor of at most $d$.Comment: 44 pages, 12 figure

Causality in AdS/CFT and Lovelock theory

We explore the constraints imposed on higher curvature corrections of the Lovelock type due to causality restrictions in the boundary of asymptotically AdS space-time. In the framework of AdS/CFT, this is related to positivity of the energy constraints that arise in conformal collider physics. We present explicit analytic results that fully address these issues for cubic Lovelock gravity in arbitrary dimensions and give the formal analytic results that comprehend general Lovelock theory. The computations can be performed in two ways, both by considering a thermal setup in a black hole background and by studying the scattering of gravitons with a shock wave in AdS. We show that both computations coincide in Lovelock theory. The different helicities, as expected, provide the boundaries defining the region of allowed couplings. We generalize these results to arbitrary higher dimensions and discuss their consequences on the shear viscosity to energy density ratio of CFT plasmas, the possible existence of Boulware-Deser instabilities in Lovelock theory and the extent to which the AdS/CFT correspondence might be valid for arbitrary dimensions.Comment: 35 pages, 20 figures; v2: minor amendments and clarifications include

the effect of different selenium sources during the finishing phase on beef quality

n/