Assessing the most powerful analysis method for school-based intervention studies

This article compares four mixed-model analyses valid for group-randomized trials (GRTs) involving a nested cohort design with a single pretest and a single posttest, the most common design used in GRTs. This study makes estimates of intraclass correlations (ICCs) available to investigators planning GRTs with alcohol, tobacco, and other drug measures as the outcomes of interest. It also provides formulae demonstrating the potential benefits to the standard error of the intervention effect of both adjustments for fixed and time-varying covariates, as well as correlations over time. These estimates will allow other researchers using these variables to plan their studies by performing a priori power analyses for any of four common analytic options

Assessing Intervention Effects in a School-Based Nutrition Intervention Trial: Which Analytic Model Is Most Powerful?

This article compares four mixed-model analyses valid for group-randomized trials (GRTs) involving a nested cohort design with a single pretest and posttest. This study makes estimates of intraclass correlations (ICCs) available to investigators planning GRTs addressing dietary outcomes. It also provides formulae demonstrating the potential benefits to the standard error of the intervention effect (σΔ) from adjustments for both fixed and time-varying covariates and correlations over time. These estimates will allow other researchers to use these variables to plan their studies by estimating a priori detectable differences and sample size requirements for any of the four analytic options. These methods are demonstrated using data from the Teens Eating for Energy and Nutrition at School study. Mixed-model analyses of covariance proved to be the most powerful analysis in that data set. The formulae may be applied to any dependent variable in any GRT given corresponding information for those variables on the parameters that define the formulae

A Theater-Based Approach to Primary Prevention of Sexual Behavior for Early Adolescents

This article compares four mixed-model analyses valid for group-randomized trials (GRTs) involving a nested cohort design with a single pretest and posttest. This study makes estimates of intraclass correlations (ICCs) available to investigators planning GRTs addressing dietary outcomes. It also provides formulae demonstrating the potential benefits to the standard error of the intervention effect (σΔ) from adjustments for both fixed and time-varying covariates and correlations over time. These estimates will allow other researchers using these variables to plan their studies by estimating a priori detectable differences and sample size requirements for any of the four analytic options. These methods are demonstrated using data from the Teens Eating for Energy and Nutrition at School study. Mixed-model analyses of covariance proved to be the most powerful analysis in that data set. The formulae may be applied to any dependent variable in any GRT given corresponding information for those variables on the parameters that define the formulae

Dune Evolution Middlesex County, Virginia Rappahannock River and Piankatank River Shorelines

Shoreline evolution is the change in shore position through time. In fact, it is the material resistance of the coastal geologic underpinnings against the impinging hydrodynamic (and aerodynamic) forces. Along the shores of Chesapeake Bay, it is a process-response system. The processes at work include winds, waves, tides and currents, which shape and modify coastlines by eroding, transporting and depositing sediments. The shore line is commonly plotted and measured to provide a rate of change but it is as important to understand the geomorphic patterns of change. Shore analysis provides the basis to know how a particular coast has changed through time and how it might proceed in the future. The purpose of this report is to document how the dunes on Piankatank River and Rappahannock River shores of Middlesex (Figure 1) have evolved since 1937. Aerial imagery was taken for most of the Bay region beginning that year, and it is this imagery that allows one to assess the geomorphic nature of shore change. Aerial imagery shows how the coast has changed, how beaches, dunes, bars, and spits have grown or decayed, how barriers have breached, how inlets have changed course, and how one shore type has displaced another or has not changed at all. Shore change is a natural process but, quite often, the impacts of man through shore hardening or inlet stabilization come to dominate a given shore reach. Most of the change in shore positions where dunes occur will be quantified in this report. Others, particularly very irregular coasts, around inlets, and other complicated areas will be subject to interpretation

Shoreline Evolution Lancaster County, Virginia Chesapeake Bay and Rappahannock River Shorelines 2006

Shoreline evolution is the change in shore position through time. In fact, it is the material resistance of the coastal geologic underpinnings against the impinging hydrodynamic (and aerodynamic) forces. Along the shores of Chesapeake Bay and Rappahannock River, it is a process-response system. The processes at work include winds, waves, tides and currents, which shape and modify coastlines by eroding, transporting and depositing sediments. The shore line is commonly plotted and measured to provide a rate of change but it is as important to understand the geomorphic patterns of change. Shore analysis provides the basis to know how a particular coast has changed through time and how it might proceed in the future. The purpose of this report is to document how the dunes along the Bay and river shores of Lancaster (Figure 1) have evolved since 1937. Aerial imagery was taken for most of the Bay region beginning that year, and it is this imagery that allows one to assess the geomorphic nature of shore change. Aerial imagery shows how the coast has changed, how beaches, dunes, bars, and spits have grown or decayed, how barriers have breached, how inlets have changed course, and how one shore type has displaced another or has not changed at all. Shore change is a natural process but, quite often, the impacts of man through shore hardening or inlet stabilization come to dominate a given shore reach. Most of the change in shore positions will be quantified in this report. Others, particularly very irregular coasts, around inlets, and other complicated areas will be subject to interpretation

Cluster randomised trials in the medical literature: two bibliometric surveys

Background: Several reviews of published cluster randomised trials have reported that about half did not take clustering into account in the analysis, which was thus incorrect and potentially misleading. In this paper I ask whether cluster randomised trials are increasing in both number and quality of reporting. Methods: Computer search for papers on cluster randomised trials since 1980, hand search of trial reports published in selected volumes of the British Medical Journal over 20 years. Results: There has been a large increase in the numbers of methodological papers and of trial reports using the term 'cluster random' in recent years, with about equal numbers of each type of paper. The British Medical Journal contained more such reports than any other journal. In this journal there was a corresponding increase over time in the number of trials where subjects were randomised in clusters. In 2003 all reports showed awareness of the need to allow for clustering in the analysis. In 1993 and before clustering was ignored in most such trials. Conclusion: Cluster trials are becoming more frequent and reporting is of higher quality. Perhaps statistician pressure works

Observed intra-cluster correlation coefficients in a cluster survey sample of patient encounters in general practice in Australia

BACKGROUND: Cluster sample study designs are cost effective, however cluster samples violate the simple random sample assumption of independence of observations. Failure to account for the intra-cluster correlation of observations when sampling through clusters may lead to an under-powered study. Researchers therefore need estimates of intra-cluster correlation for a range of outcomes to calculate sample size. We report intra-cluster correlation coefficients observed within a large-scale cross-sectional study of general practice in Australia, where the general practitioner (GP) was the primary sampling unit and the patient encounter was the unit of inference. METHODS: Each year the Bettering the Evaluation and Care of Health (BEACH) study recruits a random sample of approximately 1,000 GPs across Australia. Each GP completes details of 100 consecutive patient encounters. Intra-cluster correlation coefficients were estimated for patient demographics, morbidity managed and treatments received. Intra-cluster correlation coefficients were estimated for descriptive outcomes and for associations between outcomes and predictors and were compared across two independent samples of GPs drawn three years apart. RESULTS: Between April 1999 and March 2000, a random sample of 1,047 Australian general practitioners recorded details of 104,700 patient encounters. Intra-cluster correlation coefficients for patient demographics ranged from 0.055 for patient sex to 0.451 for language spoken at home. Intra-cluster correlations for morbidity variables ranged from 0.005 for the management of eye problems to 0.059 for management of psychological problems. Intra-cluster correlation for the association between two variables was smaller than the descriptive intra-cluster correlation of each variable. When compared with the April 2002 to March 2003 sample (1,008 GPs) the estimated intra-cluster correlation coefficients were found to be consistent across samples. CONCLUSIONS: The demonstrated precision and reliability of the estimated intra-cluster correlations indicate that these coefficients will be useful for calculating sample sizes in future general practice surveys that use the GP as the primary sampling unit

Electronic search strategies to identify reports of cluster randomized trials in MEDLINE: low precision will improve with adherence to reporting standards

BACKGROUND: Cluster randomized trials (CRTs) present unique methodological and ethical challenges. Researchers conducting systematic reviews of CRTs (e.g., addressing methodological or ethical issues) require efficient electronic search strategies (filters or hedges) to identify trials in electronic databases such as MEDLINE. According to the CONSORT statement extension to CRTs, the clustered design should be clearly identified in titles or abstracts; however, variability in terminology may make electronic identification challenging. Our objectives were to (a) evaluate sensitivity ( recall ) and precision of a well-known electronic search strategy ( randomized controlled trial as publication type) with respect to identifying CRTs, (b) evaluate the feasibility of new search strategies targeted specifically at CRTs, and (c) determine whether CRTs are appropriately identified in titles or abstracts of reports and whether there has been improvement over time. METHODS: We manually examined a wide range of health journals to identify a gold standard set of CRTs. Search strategies were evaluated against the gold standard set, as well as an independent set of CRTs included in previous systematic reviews. RESULTS: The existing strategy (randomized controlled trial.pt) is sensitive (93.8%) for identifying CRTs, but has relatively low precision (9%, number needed to read 11); the number needed to read can be halved to 5 (precision 18.4%) by combining with cluster design-related terms using the Boolean operator AND; combining with the Boolean operator OR maximizes sensitivity (99.4%) but would require 28.6 citations read to identify one CRT. Only about 50% of CRTs are clearly identified as cluster randomized in titles or abstracts; approximately 25% can be identified based on the reported units of randomization but are not amenable to electronic searching; the remaining 25% cannot be identified except through manual inspection of the full-text article. The proportion of trials clearly identified has increased from 28% between the years 2000-2003, to 60% between 2004-2007 (absolute increase 32%, 95% CI 17 to 47%). CONCLUSIONS: CRTs should include the phrase cluster randomized trial in titles or abstracts; this will facilitate more accurate indexing of the publication type by reviewers at the National Library of Medicine, and efficient textword retrieval of the subset employing cluster randomization

Increased risk of type I errors in cluster randomised trials with small or medium numbers of clusters: a review, reanalysis,and simulation study

Background: Cluster randomised trials (CRTs) are commonly analysed using mixed-effects models or generalised estimating equations (GEEs). However, these analyses do not always perform well with the small number of clusters typical of most CRTs. They can lead to increased risk of a type I error (finding a statistically significant treatment effect when it does not exist) if appropriate corrections are not used. Methods: We conducted a small simulation study to evaluate the impact of using small-sample corrections for mixed-effects models or GEEs in CRTs with a small number of clusters. We then reanalysed data from TRIGGER, a CRT with six clusters, to determine the effect of using an inappropriate analysis method in practice. Finally, we reviewed 100 CRTs previously identified by a search on PubMed in order to assess whether trials were using appropriate methods of analysis. Trials were classified as at risk of an increased type I error rate if they did not report using an analysis method which accounted for clustering, or if they had fewer than 40 clusters and performed an individual-level analysis without reporting the use of an appropriate small-sample correction. Results: Our simulation study found that using mixed-effects models or GEEs without an appropriate correction led to inflated type I error rates, even for as many as 70 clusters. Conversely, using small-sample corrections provided correct type I error rates across all scenarios. Reanalysis of the TRIGGER trial found that inappropriate methods of analysis gave much smaller P values (P ≤ 0.01) than appropriate methods (P = 0.04–0.15). In our review, of the 99 trials that reported the number of clusters, 64 (65 %) were at risk of an increased type I error rate; 14 trials did not report using an analysis method which accounted for clustering, and 50 trials with fewer than 40 clusters performed an individual-level analysis without reporting the use of an appropriate correction. Conclusions: CRTs with a small or medium number of clusters are at risk of an inflated type I error rate unless appropriate analysis methods are used. Investigators should consider using small-sample corrections with mixed-effects models or GEEs to ensure valid results. Abbreviations: CRT, Cluster randomised trial; CI, Confidence interval; GEE, Generalised estimating equations; TRIGGER, Trial in Gastrointestinal Transfusio

Guidance for Evidence-Informed Policies about Health Systems: Assessing How Much Confidence to Place in the Research Evidence

In the third paper in a three-part series on health systems guidance, Simon Lewin and colleagues explore the challenge of assessing how much confidence to place in evidence on health systems interventions