The impact of Feedback on student attainment: a systematic review

Abstract Meta-syntheses have reported positive impacts of feedback for student achievement at different stages of education and have been influential in establishing feedback as an effective strategy to support student learning. However, these syntheses combine studies of a variety of different feedback approaches, combine studies where feedback is one of a number of intervention components and have several methodological limitations. For example the lack of quality appraisal of the included studies. There is also still more research needed to investigate the impact of different types of feedback on different students in different settings. Objective This systematic review was conducted at the request of the Education Endowment Foundation to provide more precise estimates of the impact of different types of feedback in different contexts for different learners aged between 5 and 18. The review analysis sought to explore potential variations in the impact of feedback through subgroup analysis of the characteristics of the feedback, the educational setting, the learners and the subject. This review provides evidence that can be used to support the development of guidance for teachers and schools about feedback practices. Methods design A systematic review was undertaken in two stages. First, a systematic map identified and characterised a subset of studies that investigated the attainment impacts of feedback. Second, an in-depth review comprising of a meta-analysis was performed to answer the review questions about the impact of interventions that comprised of feedback only and to explore the variety of characteristics that may influence the impact of feedback. Methods search We used the Microsoft Academic Graph (MAG) dataset hosted in EPPI-Reviewer to conduct a semantic network analysis to identify records related to a set of pre-identified study references. The MAG search identified 23,725 potential studies for screening. Methods study selection Studies were selected using a set of pre-specified selection criterion. Semi-automated priority screening was used to screen the title and abstract of studies using bespoke systematic review software EPPI-Reviewer. The title and abstract screening was stopped after 3,028 studies and 745 were identified for full-text screening. Reviewers carried out a moderation exercise, all screening a selection of the same titles to develop consistency of screening. Thereafter, single reviewer screening was used with referral for a second reviewer opinion in cases of uncertainty. Methods data collection Studies were coded using a bespoke data extraction tool developed by the EEF Database Project. Study quality was assessed using a bespoke risk of bias assessment adapted from the ROBINS-I tool. The review team undertook a moderation exercise coding the same set of studies to develop consistency. Thereafter, single reviewer coding was used, based on the full text with referral for a second opinion in cases of uncertainty. Methods synthesis Data from the studies was used to calculate standardised effect sizes (Standardised Mean Difference- Hedge’s g). Effect sizes from each study were combined to produce a pooled estimate of effect using Random Effects Meta-analysis. Statistical Heterogeneity tests were carried out for each synthesis. Sensitivity analysis was carried out for assessed study quality. Subgroup analysis was completed using meta-analysis to explore outcomes according to the different characteristics of feedback, context and subjects. Main results The full text screening identified 304 studies to include in the initial systematic map, of which 171 studies investigated feedback only. After applying final selection criteria, 43 papers with 51 studies published in and after the year 2000 were included. The 51 studies had approximately 14,400 students. Forty studies were experiments with random allocation to groups and 11 were prospective quantitative experimental design studies. The overall ecological validity was assessed as moderate to high in 40 studies and the overall risk of bias assessed as low to moderate in 44 studies. The interventions took place in curriculum subjects including literacy, mathematics, science, social studies, and languages, and tested other cognitive outcomes. The source of feedback included teacher, researcher, digital, or automated means. Feedback to individual students is reported in 48 studies and feedback to group or class is reported in four studies. Feedback took the form of spoken verbal, non-verbal, written verbal, and written non-verbal. Different studies investigated feedback that took place immediately after the task, during the task and up to one week after the task (delayed feedback). Most of the feedback interventions gave the learner feedback about the outcome and the process/strategy. Some provided feedback on outcome only and two provided feedback about task/strategy only. On the main research question, the pooled estimate of effect of synthesis of all studies with a low or moderate risk of bias indicated that students who received feedback had better performance than students who did not receive feedback or experienced usual practice (g = 0.17, 95% C.I. 0.09 to 0.25). However, there is statistically significant heterogeneity between these studies (I2 = 44%, Test for Heterogeneity: Q(df = 37) = 65.92, p = 0.002), which suggests that this may not be a useful indicator of the general impact of feedback on attainment when compared to no feedback or usual practice. The heterogeneity analysis suggested considerable heterogeneity between studies in the main synthesis and all the subgroup synthesis, and in the majority of the cases the heterogeneity is statistically significant. This means caution is required when considering the results of the synthesis. The results of the subgroup synthesis suggest that a variety of student and context factors may have an effect on the impact of feedback. Conclusions The results of the review may be considered broadly consistent with claims made on the basis of previous synthesis and meta-synthesis, suggesting that feedback interventions, on average, have a positive impact on attainment when compared to no feedback or usual practice. The limitations in the study reports and the comparatively small number of studies within each subgroup synthesis meant that the review was not able to provide very much more certainty about the factors that affect variation in the impact of single component feedback interventions within different contexts and with different students. More research is needed in this area to consider what may moderate the impact of feedback. However, the findings further support the conclusion made by previous studies that feedback, on average, has a positive impact on attainment; moreover, this is based on a more precise and robust analysis than previous syntheses. This suggests that feedback may have a role to play in raising attainment alongside other effective interventions. Findings were further interpreted by a panel of expert practitioners and academics to produce the EEF’s Teacher feedback to improve pupil learning guidance report. 1. Background and review rationale Feedback can be defined as information communicated to the learner that is intended to modify the learner’s thinking or behaviour for the purpose of improving learning. Meta-syntheses have reported positive impacts of feedback, with effect sizes ranging from d = 0.70 to d = 0.79 for student achievement at different stages of education and have been influential in establishing feedback as highly effective with regards to student learning. For example, the EEF Teaching and Learning Toolkit meta-synthesis suggests that feedback may have ‘very high’ impact (equivalent to eight months’ additional progress) for relatively low cost. However, caution is necessary when interpreting the findings of these meta-syntheses for a number of reasons. Firstly, the average effect size reported in the EEF Toolkit is based on combining the estimates from existing meta-analyses of individual studies, which may contain limitations of various kinds (see the list below for examples) that may mean that average effect sizes identified are overestimates. Second, some studies included in syntheses (such as Kluger and DeNisi’s meta-analysis ) suggest that some feedback interventions may, in fact, negatively impact pupils. Third, previous meta-syntheses have not explored in detail the impact of potential moderating factors, such as different types of feedback. As Ekecrantz has argued, there is still a need to better understand how and under what circumstances teacher feedback on student performance promotes learning as well as, to question the generalised claim (that feedback improves attainment) itself. For example, a recent meta-analysis that re-analysed studies included in the original synthesis by Hattie and Timperley revised down the average effect size from the estimates of the effects of feedback from their originally published Standardised Mean Difference of d = 0.79 to d = 0.48. In the revised meta-analysis, 17% of the effect sizes from individual studies were negative. The confidence interval ranged from d = 0.48 to d = 0.62, and the authors found a wide range of effect sizes. Different moderators were also investigated to explore the impact of different characteristics of context and feedback. Whilst this meta-analysis offers improvements over previous meta-syntheses, it has a number of limitations, including: • It only included studies drawn from 36 existing meta-analyses, the most recent of which was published in 2015. Eligible studies published after 2015 or not included in these meta-analyses would not have been included. • All comparative study designs were included. Less robust study designs may have overestimated the positive effect of feedback. • There was no reported study quality assessment/moderation or sensitivity analysis, which may have led to an overestimation of the pooled effect sizes. • The meta-analyses included studies with high levels of heterogeneity, I2 = 80% or more (in the main and moderator analysis). This suggests that the synthesis may be combining studies/comparing feedback practices inappropriately. • The meta-analysis did not consider all potentially relevant moderating factors. It may also be the case that the impact of feedback depends on factors other than those analysed, including the ability of the learner, the learning context, and/or the frequency, duration, timing, and type of feedback. This systematic review was conducted at the request of the EEF to try and provide more accurate and precise estimates of the impact of different types of feedback in different schooling contexts. The review examines the impact of single component feedback, in different contexts, and for different learners with a greater degree of granularity and precision than is currently available via the EEF Teaching and Learning Toolkit strand on ‘Feedback’. For EEF, the purpose of the systematic review is to provide evidence that can be used to inform guidance for teachers and schools about effective feedback practices. The systematic review methods and processes were developed and carried out conterminously with the EEF Database project with a view to facilitating the future use of the produced resources and supporting the ongoing work of the Database project. 1.1 Domain being studied: Feedback approaches This review focuses on interventions that provide feedback from teachers to learners in mainstream educational settings. Feedback is defined in accordance with the EEF toolkit definition: ‘Feedback is information given to the learner and/or teacher about the learner’s performance relative to learning goals or outcomes. It should aim to produce (and be capable of) producing improvement in students’ learning. Feedback redirects or refocuses either the teacher’s or the learner’s actions to achieve a goal, by aligning effort and activity with an outcome. It can be about the output of the activity, the process of the activity, the student’s management of their learning or self-regulation, or them as individuals. This feedback can be verbal or written or can be given through tests or via digital technology. It can come from a teacher or someone taking a teaching role, or from ‘peers’.’ This initial broad definition, whilst conceptually coherent, does create challenges both in practice for teachers and in terms of identifying and distinguishing between practices when considering research evidence. For example, what is the difference between small group learning and ‘peer feedback’? It seems perfectly reasonable to assume that small group learning must contain conversations between students about their work and the task they have been asked to complete and thus is ‘feedback’. However, in practice, this may not be what teachers think of as ‘feedback’ and in the research literature, ‘small group learning’ is investigated both as a unique pedagogical strategy and as a component of a number of other pedagogical strategies. As the development of the understanding of the scope of the review evolved, the working definition of feedback for the review became modified practically through the exclusion of certain categories of intervention, even though they may contain an element of feedback practice. The inclusion criteria in the methods section outlines the revised definition that the review team used. 1.2 Conceptual framework/Theory of Change There are several ways in which feedback is conceptualised as improving learner performance—i.e. as a Theory of Change. The ‘Feedback’ strand in the EEF Teaching and Learning Toolkit draws most explicitly on the conceptualisation of Hattie and Timperley’s (2007) model. This model emphasises the importance of systems of feedback where the teacher provides feedback to the specific needs of individual students. The searching processes used in this review are consistent with this model as the studies used in the Feedback strand of the EEF Teaching and Learning Toolkit were used to ‘seed’ the search. However, they did not preclude the inclusion of studies that may draw on other ‘models’ of feedback which, though similar to Hattie and Timperley (2007), may be argued to place more emphasis on, for example: developing learner self-regulation (Nicole and Macfarlane-Dick, 2006); students’ intrinsic motivation (Dweck, 2016); and/or are subject specific—for example, ‘Thinking Mathematically’ (Mason, Burton and Stacey, 2010). The coding tools used in the review were informed by the model (in terms of coding about the source and content of the feedback; see Appendix 3). 1.3 Review design A systematic review approach was used to investigate the research questions. The review was undertaken in two stages. First, a systematic map identified and described the feedback characteristics of a subset of studies that investigated the attainment impacts of feedback. The map was used to make decisions about focusing the analysis in the second in-depth systematic review stage. At the second stage an in-depth review, including meta-analysis, was performed on a subset of the studies identified in the map to answer the review questions and explore the variety of intervention and context characteristics that may influence the impact of feedback. This systematic review was designed to complement the work of the EEF Database project. The EEF Database project is currently undertaking a programme to extract and code the individual studies from the meta-synthesis used in the EEF Teaching and Learning Toolkit. The search strategy used in this review was ‘seeded’ from studies identified as being about ‘feedback’ in the database, and this systematic review used the coding tools developed by the Database team (see Appendix 3). The studies newly identified in this review will be subsequently included in the EEF Database. This systematic review was also designed to provide additional research evidence for use in guidance on feedback developed for schools produced by the EEF, and therefore to fit with a particular time window for the review’s production. The results of the meta-analyses were presented to an advisory panel of academics and teaching practitioners, who used the results , their own expertise, a review of practice undertaken by the University of Oxford, and conceptual models (such as Hattie and Timperley) to draft recommendations for practice

AMPK Activators Suppress Cervical Cancer Cell Growth through Inhibition of DVL3 Mediated Wnt/β-Catenin Signaling Activity

published_or_final_versio

AMPK Activators Suppress Cervical Cancer Cell Growth through Inhibition of DVL3 Mediated Wnt/β-Catenin Signaling Activity

published_or_final_versio

Population Factors Affecting Initial Diffusion Patterns of H1N1

published_or_final_versio

An Early Warning System for Detecting H1N1 Disease Outbreak - A Spatio-temporal Approach

The outbreaks of new and emerging infectious diseases in recent decades have caused widespread social and economic disruptions in the global economy. Various guidelines for pandemic influenza planning are based upon traditional infection control, best practice and evidence. This article describes the development of an early warning system for detecting disease outbreaks in the urban setting of Hong Kong, using 216 confirmed cases of H1N1 influenza from 1 May 2009 to 20 June 2009. The prediction model uses two variables – daily influenza cases and population numbers – as input to the spatio-temporal and stochastic SEIR model to forecast impending disease cases. The fairly encouraging forecast accuracy metrics for the 1- and 2-day advance prediction suggest that the number of impending cases could be estimated with some degree of certainty. Much like a weather forecast system, the procedure combines technical and scientific skills using empirical data but the interpretation requires experience and intuitive reasoning.postprin

Fuzzy-logic controlled genetic algorithm for the rail-freight crew-scheduling problem

AbstractThis article presents a fuzzy-logic controlled genetic algorithm designed for the solution of the crew-scheduling problem in the rail-freight industry. This problem refers to the assignment of train drivers to a number of train trips in accordance with complex industrial and governmental regulations. In practice, it is a challenging task due to the massive quantity of train trips, large geographical span and significant number of restrictions. While genetic algorithms are capable of handling large data sets, they are prone to stalled evolution and premature convergence on a local optimum, thereby obstructing further search. In order to tackle these problems, the proposed genetic algorithm contains an embedded fuzzy-logic controller that adjusts the mutation and crossover probabilities in accordance with the genetic algorithm’s performance. The computational results demonstrate a 10% reduction in the cost of the schedule generated by this hybrid technique when compared with a genetic algorithm with fixed crossover and mutation rates

Presynaptic partner selection during retinal circuit reassembly varies with timing of neuronal regeneration in vivo

Whether neurons can restore their original connectivity patterns during circuit repair is unclear. Taking advantage of the regenerative capacity of zebrafish retina, we show here the remarkable specificity by which surviving neurons reassemble their connectivity upon regeneration of their major input. H3 horizontal cells (HCs) normally avoid red and green cones, and prefer ultraviolet over blue cones. Upon ablation of the major (ultraviolet) input, H3 HCs do not immediately increase connectivity with other cone types. Instead, H3 dendrites retract and re-extend to contact new ultraviolet cones. But, if regeneration is delayed or absent, blue-cone synaptogenesis increases and ectopic synapses are made with red and green cones. Thus, cues directing synapse specificity can be maintained following input loss, but only within a limited time period. Further, we postulate that signals from the major input that shape the H3 HC's wiring pattern during development persist to restrict miswiring after damage

The management of an endodontically abscessed tooth: patient health state utility, decision-tree and economic analysis

<p>Abstract</p> <p>Background</p> <p>A frequent encounter in clinical practice is the middle-aged adult patient complaining of a toothache caused by the spread of a carious infection into the tooth's endodontic complex. Decisions about the range of treatment options (conventional crown with a post and core technique (CC), a single tooth implant (STI), a conventional dental bridge (CDB), and a partial removable denture (RPD)) have to balance the prognosis, utility and cost. Little is know about the utility patients attach to the different treatment options for an endontically abscessed mandibular molar and maxillary incisor. We measured patients' dental-health-state utilities and ranking preferences of the treatment options for these dental problems.</p> <p>Methods</p> <p>Forty school teachers ranked their preferences for conventional crown with a post and core technique, a single tooth implant, a conventional dental bridge, and a partial removable denture using a standard gamble and willingness to pay. Data previously reported on treatment prognosis and direct "out-of-pocket" costs were used in a decision-tree and economic analysis</p> <p>Results</p> <p>The Standard Gamble utilities for the restoration of a mandibular 1st molar with either the conventional crown (CC), single-tooth-implant (STI), conventional dental bridge (CDB) or removable-partial-denture (RPD) were 74.47 [± 6.91], 78.60 [± 5.19], 76.22 [± 5.78], 64.80 [± 8.1] respectively (p < 0.05). Their respective Willingness-to-Pay ($CDN) were 1,782.05 [± 361.42], 1,871.79 [± 349.44], 1,605.13 [± 348.10], 1,351.28 [± 368.62] (p < 0.05).</p> <p>The standard gamble utilities for the restoration of a maxillary central incisor with a CC, STI, CDB and RPD were 88.50 [± 6.12], 90.68 [± 3.41], 89.78 [± 3.81] and 91.10 [± 3.57] respectively (p > 0.05). Their respective willingness-to-pay ($CDN) were: 1,782.05 [± 361.42], 1,871.79 [± 349.44], 1,605.13 [± 348.10] and 1,351.28 [± 368.62]. A statistical difference was found between the utility of treating a maxillary central incisor and mandibular 1st-molar (p < 0.05).</p> <p>The expected-utility-value for a 5-year prosthetic survival was highest for the CDB and the STI treatment of an abscessed mandibular molar (74.75 and 71.47 respectively) and maxillary incisor (86.24 and 84.91 respectively). This held up to a sensitivity analysis when the success of root canal therapy and the risk of damage to the adjacent tooth were varied. The RPD for both the molar and incisor was the favored treatment based on a cost-utility (3.85 and 2.74 CND$per year of tooth saved respectively) and cost-benefit analysis (0.92 to 0.60 CND$ of cost per $ of benefit, respectively) for a prosthetic clinical survival of 5-years.</p> <p>Conclusion</p> <p>The position of the abscessed tooth and the amount of insurance coverage influences the utility and rank assigned by patients to the different treatment options. STI and CDB have optimal EUVs for a 5-year survival outcome, and RPD has significantly lower cost providing the better cost:benefit ratio.</p

Search for new phenomena in final states with an energetic jet and large missing transverse momentum in pp collisions at √ s = 8 TeV with the ATLAS detector

Results of a search for new phenomena in final states with an energetic jet and large missing transverse momentum are reported. The search uses 20.3 fb−1 of √ s = 8 TeV data collected in 2012 with the ATLAS detector at the LHC. Events are required to have at least one jet with pT > 120 GeV and no leptons. Nine signal regions are considered with increasing missing transverse momentum requirements between Emiss T > 150 GeV and Emiss T > 700 GeV. Good agreement is observed between the number of events in data and Standard Model expectations. The results are translated into exclusion limits on models with either large extra spatial dimensions, pair production of weakly interacting dark matter candidates, or production of very light gravitinos in a gauge-mediated supersymmetric model. In addition, limits on the production of an invisibly decaying Higgs-like boson leading to similar topologies in the final state are presente

Cured meat, vegetables, and bean-curd foods in relation to childhood acute leukemia risk: A population based case-control study

<p>Abstract</p> <p>Background</p> <p>Consumption of cured/smoked meat and fish leads to the formation of carcinogenic <it>N-</it>nitroso compounds in the acidic stomach. This study investigated whether consumed cured/smoked meat and fish, the major dietary resource for exposure to nitrites and nitrosamines, is associated with childhood acute leukemia.</p> <p>Methods</p> <p>A population-based case-control study of Han Chinese between 2 and 20 years old was conducted in southern Taiwan. 145 acute leukemia cases and 370 age- and sex-matched controls were recruited between 1997 and 2005. Dietary data were obtained from a questionnaire. Multiple logistic regression models were used in data analyses.</p> <p>Results</p> <p>Consumption of cured/smoked meat and fish more than once a week was associated with an increased risk of acute leukemia (OR = 1.74; 95% CI: 1.15–2.64). Conversely, higher intake of vegetables (OR = 0.55; 95% CI: 0.37–0.83) and bean-curd (OR = 0.55; 95% CI: 0.34–0.89) was associated with a reduced risk. No statistically significant association was observed between leukemia risk and the consumption of pickled vegetables, fruits, and tea.</p> <p>Conclusion</p> <p>Dietary exposure to cured/smoked meat and fish may be associated with leukemia risk through their contents of nitrites and nitrosamines among children and adolescents, and intake of vegetables and soy-bean curd may be protective.</p