Randomised controlled trials (RCTs) in education research –methodological debates, questions, challenges

This Special Issue of Educational Research shines a spotlight on a key research design in the 21st Century in the field of education research: the randomised controlled trial (RCT) or ‘true’ experiment. The six papers included here explore the history and future of the design. They focus on challenges and opportunities, methodological developments and innovation, but above all they highlight the immense progress that has been made in rigorous evaluation over the last 60 years. They provide a historical background in the United States of America (USA), in Scandinavia and in the United Kingdom (UK), and cover the aspects of politics and methodology that have shaped the recent education science landscape. In a time when answers are sought to questions of efficacy and effectiveness of education policies and practices, RCTs have a special role. Uniquely among research designs, they are able to obtain unbiased estimates of the average effects of these policies and practices on children and young people’s education and wider outcomes. Furthermore, they serve as a useful introduction to contemporary issues in RCTs for any education researcher who has an interest in them, but may have felt hindered by limited technical knowledge

Systematic reviews of randomized controlled trials in literacy research : methodological challenges

Introduction: In this item a 'tertiary' review of systematic reviews in literacy learning is presented. It explores the methodological quality of the identified systematic reviews and identifies the primary data that are used for the in-depth methodological work in Item 3 on the two main threats to the validity of systematic reviews: publication bias and design bias. Background: Recent governments in the UK have introduced a number of initiatives aimed at improving the literacy levels of children. It is important, therefore, that policy and practice are informed by the most rigorous available evidence, particularly for questions of effectiveness in literacy learning. It is also important that this evidence is subjected to rigorous critical scrutiny. Methods: Systematic reviews undertaken in the field of literacy learning in English in the years between 1983 and 2003 were searched for, located and quality assessed. The scope of the review was limited to systematic reviews of experimental research evaluating literacy interventions with quantifiable literacy outcome measures in English as a first (not second or additional) language and focusing on children and young people in school settings up to the age of 18. Results: A total of 14 systematic reviews containing meta-analyses and meeting all the inclusion criteria were included in the tertiary review. The following data were extracted from the reviews: literacy interventions, outcomes evaluated and effect sizes. The quality of the reviews was examined using an adaptation of the QUORUM statement. Overall the quality of the meta-analyses included in this tertiary review was good. When examining the effect sizes of randomized controlled trials (RCTs) and controlled trials (CTs) separately there was no clear pattern as to whether the RCTs produced a larger or smaller effect size than the CTs. Discussion: Overall the quality of the meta-analyses included in this tertiary review was good. The QUORUM checklist seemed to perform well for the appraisal of educational meta-analyses. All the reviews clearly stated their research question, and their methods of searching for and selecting included studies. Most studies described their data extraction and used some form of quality assessment of included studies. On the other hand, some reviews did have notable methodological weaknesses. Six of the 14 studies did not make an assessment of publication bias, which is potentially a major threat to the validity of any systematic review. In addition, six studies did not provide evidence for reviewer agreement when synthesising the data. There is, therefore, some room for improvement in the methodological quality of systematic reviews in literacy learning. Conclusions: A number of reviews in this tertiary review are judged to be of sufficiently high quality to provide reliable evidence for the effectiveness of literacy interventions

Pragmatic cluster randomised controlled trial of contextualised grammar teaching and small group teaching to improve the writing skills of 11 year old children

Introduction: We evaluated two interventions: a contextualised grammar teaching intervention – Grammar for Writing - to assess whether it improved 11 year old children’s writing skills; and a small group literacy intervention to assess whether or not this was effective. Design and method: We used a pragmatic cluster randomised trial with partial split plot design. Independent concealed randomisation was undertaken at the class level, and, within the intervention group, children were also individually randomised to receive the whole class intervention plus a small group intervention or to receive the intervention in a whole class setting only. The main outcomes were writing and reading assessed by the Progress in English 11 (Long Form) test (GL Assessment). Results: In 2013, 55 schools in England, each with two classes, were recruited and randomised. Within each school, the two classes were randomly allocated to receive either the intervention or the control condition. After randomisation, 2 schools withdrew, leaving 53 schools, 106 classes and 2510 pupils. We observed an effect size (ES) of 0.10 favouring the Grammar for Writing classes; however, this was not statistically significant (95% confidence interval (CI) -0.10 to 0.31). Pupils randomised to the small groups had an increased literacy score when compared with the control classes (ES = 0.24, 95% CI 0.00 to 0.49) and when compared with the intervention children taught in the whole class (ES = 0.21, 95% CI 0.04 to 0.38). Conclusion: There is little evidence that this form of contextualised grammar teaching had an effect on 11 year old children’s writing skills. There was some evidence of an effect for small group teaching

Resequencing Candidate Genes Implicates Rare Variants in Asthma Susceptibility

Common variation in over 100 genes has been implicated in the risk of developing asthma, but the contribution of rare variants to asthma susceptibility remains largely unexplored. We selected nine genes that showed the strongest signatures of weak purifying selection from among 53 candidate asthma-associated genes, and we sequenced the coding exons and flanking noncoding regions in 450 asthmatic cases and 515 nonasthmatic controls. We observed an overall excess of p values <0.05 (p = 0.02), and rare variants in four genes (AGT, DPP10, IKBKAP, and IL12RB1) contributed to asthma susceptibility among African Americans. Rare variants in IL12RB1 were also associated with asthma susceptibility among European Americans, despite the fact that the majority of rare variants in IL12RB1 were specific to either one of the populations. The combined evidence of association with rare noncoding variants in IL12RB1 remained significant (p = 3.7 × 10−4) after correcting for multiple testing. Overall, the contribution of rare variants to asthma susceptibility was predominantly due to noncoding variants in sequences flanking the exons, although nonsynonymous rare variants in DPP10 and in IL12RB1 were associated with asthma in African Americans and European Americans, respectively. This study provides evidence that rare variants contribute to asthma susceptibility. Additional studies are required for testing whether prioritizing genes for resequencing on the basis of signatures of purifying selection is an efficient means of identifying novel rare variants that contribute to complex disease

The translation research in a dental setting (TRiaDS) programme protocol

Background: It is well documented that the translation of knowledge into clinical practice is a slow and haphazard process. This is no less true for dental healthcare than other types of healthcare. One common policy strategy to help promote knowledge translation is the production of clinical guidance, but it has been demonstrated that the simple publication of guidance is unlikely to optimise practice. Additional knowledge translation interventions have been shown to be effective, but effectiveness varies and much of this variation is unexplained. The need for researchers to move beyond single studies to develop a generalisable, theory based, knowledge translation framework has been identified.For dentistry in Scotland, the production of clinical guidance is the responsibility of the Scottish Dental Clinical Effectiveness Programme (SDCEP). TRiaDS (Translation Research in a Dental Setting) is a multidisciplinary research collaboration, embedded within the SDCEP guidance development process, which aims to establish a practical evaluative framework for the translation of guidance and to conduct and evaluate a programme of integrated, multi-disciplinary research to enhance the science of knowledge translation.Methods: Set in General Dental Practice the TRiaDS programmatic evaluation employs a standardised process using optimal methods and theory. For each SDCEP guidance document a diagnostic analysis is undertaken alongside the guidance development process. Information is gathered about current dental care activities. Key recommendations and their required behaviours are identified and prioritised. Stakeholder questionnaires and interviews are used to identify and elicit salient beliefs regarding potential barriers and enablers towards the key recommendations and behaviours. Where possible routinely collected data are used to measure compliance with the guidance and to inform decisions about whether a knowledge translation intervention is required. Interventions are theory based and informed by evidence gathered during the diagnostic phase and by prior published evidence. They are evaluated using a range of experimental and quasi-experimental study designs, and data collection continues beyond the end of the intervention to investigate the sustainability of an intervention effect.Discussion: The TRiaDS programmatic approach is a significant step forward towards the development of a practical, generalisable framework for knowledge translation research. The multidisciplinary composition of the TRiaDS team enables consideration of the individual, organisational and system determinants of professional behaviour change. In addition the embedding of TRiaDS within a national programme of guidance development offers a unique opportunity to inform and influence the guidance development process, and enables TRiaDS to inform dental services practitioners, policy makers and patients on how best to translate national recommendations into routine clinical activities

Whole-Genome Sequencing of Individuals from a Founder Population Identifies Candidate Genes for Asthma

Asthma is a complex genetic disease caused by a combination of genetic and environmental risk factors. We sought to test classes of genetic variants largely missed by genome-wide association studies (GWAS), including copy number variants (CNVs) and low-frequency variants, by performing whole-genome sequencing (WGS) on 16 individuals from asthma-enriched and asthma-depleted families. The samples were obtained from an extended 13-generation Hutterite pedigree with reduced genetic heterogeneity due to a small founding gene pool and reduced environmental heterogeneity as a result of a communal lifestyle. We sequenced each individual to an average depth of 13-fold, generated a comprehensive catalog of genetic variants, and tested the most severe mutations for association with asthma. We identified and validated 1960 CNVs, 19 nonsense or splice-site single nucleotide variants (SNVs), and 18 insertions or deletions that were out of frame. As follow-up, we performed targeted sequencing of 16 genes in 837 cases and 540 controls of Puerto Rican ancestry and found that controls carry a significantly higher burden of mutations in IL27RA (2.0% of controls; 0.23% of cases; nominal p = 0.004; Bonferroni p = 0.21). We also genotyped 593 CNVs in 1199 Hutterite individuals. We identified a nominally significant association (p = 0.03; Odds ratio (OR) = 3.13) between a 6 kbp deletion in an intron of NEDD4L and increased risk of asthma. We genotyped this deletion in an additional 4787 non-Hutterite individuals (nominal p = 0.056; OR = 1.69). NEDD4L is expressed in bronchial epithelial cells, and conditional knockout of this gene in the lung in mice leads to severe inflammation and mucus accumulation. Our study represents one of the early instances of applying WGS to complex disease with a large environmental component and demonstrates how WGS can identify risk variants, including CNVs and low-frequency variants, largely untested in GWAS