A randomised controlled trial is not a pilot trial simply because it uses a surrogate endpoint.

Background: It has been argued that true endpoints (or 'hard' endpoints) for clinical trials, which are meaningful to clinicians, researchers and patients alike, are limited to those that measure health status, survival and cost. Other endpoints are termed 'surrogate' endpoints and are intended to substitute and predict the true endpoint.  A number of trials that describe using surrogate endpoints use the term 'pilot' in the title of the paper but the reason for this, as related by the authors, is the use of these surrogate endpoints in the trial. The conduct and reporting of such a trial may follow the traditional pattern for a conventional randomised controlled trial (RCT) as defined by the original CONSORT statement, with power-based sample size calculations, and significance tests of the results. However, this is contrary to the guidelines of the CONSORT extension for the reporting of pilot trials. Main body: We review the definition of a surrogate endpoint and the use of surrogate endpoints in clinical trials. We consider to what extent a trial could be considered a pilot trial if it uses a surrogate endpoint and discuss two examples that illustrate current practice. Conclusion: Trials which use surrogate endpoints should only be described as 'pilot' when a definitive trial is a distinct possibility and the authors consider conditions which would indicate whether the definitive main trial was worthwhile and feasible. Simply because a trial uses a surrogate endpoint is not justification for calling it a pilot trial

Cluster randomised trials in the medical literature: two bibliometric surveys

Background: Several reviews of published cluster randomised trials have reported that about half did not take clustering into account in the analysis, which was thus incorrect and potentially misleading. In this paper I ask whether cluster randomised trials are increasing in both number and quality of reporting. Methods: Computer search for papers on cluster randomised trials since 1980, hand search of trial reports published in selected volumes of the British Medical Journal over 20 years. Results: There has been a large increase in the numbers of methodological papers and of trial reports using the term 'cluster random' in recent years, with about equal numbers of each type of paper. The British Medical Journal contained more such reports than any other journal. In this journal there was a corresponding increase over time in the number of trials where subjects were randomised in clusters. In 2003 all reports showed awareness of the need to allow for clustering in the analysis. In 1993 and before clustering was ignored in most such trials. Conclusion: Cluster trials are becoming more frequent and reporting is of higher quality. Perhaps statistician pressure works

Managing Injuries of the Neck Trial (MINT) : design of a randomised controlled trial of treatments for whiplash associated disorders

Background: A substantial proportion of patients with whiplash injuries develop chronic symptoms. However, the best treatment of acute injuries to prevent long-term problems is uncertain. A stepped care treatment pathway has been proposed, in which patients are given advice and education at their initial visit to the emergency department (ED), followed by review at three weeks and physiotherapy for those with persisting symptoms. MINT is a two-stage randomised controlled trial to evaluate two components of such a pathway: 1. use of The Whiplash Book versus usual advice when patients first attend the emergency department; 2. referral to physiotherapy versus reinforcement of advice for patients with continuing symptoms at three weeks. Methods: Evaluation of the Whiplash Book versus usual advice uses a cluster randomised design in emergency departments of eight NHS Trusts. Eligible patients are identified by clinicians in participating emergency departments and are sent a study questionnaire within a week of their ED attendance. Three thousand participants will be included. Patients with persisting symptoms three weeks after their ED attendance are eligible to join an individually randomised study of physiotherapy versus reinforcement of the advice given in ED. Six hundred participants will be randomised. Follow-up is at 4, 8 and 12 months after their ED attendance. Primary outcome is the Neck Disability Index (NDI), and secondary outcomes include quality of life and time to return to work and normal activities. An economic evaluation is being carried out. Conclusion: This paper describes the protocol and operational aspects of a complex intervention trial based in NHS emergency and physiotherapy departments, evaluating two components of a stepped-care approach to the treatment of whiplash injuries. The trial uses two randomisations, with the first stage being cluster randomised and the second individually randomised

Methodological bias in cluster randomised trials

Background: Cluster randomised trials can be susceptible to a range of methodological problems. These problems are not commonly recognised by many researchers. In this paper we discuss the issues that can lead to bias in cluster trials. Methods: We used a sample of cluster randomised trials from a recent review and from a systematic review of hip protectors. We compared the mean age of participants between intervention groups in a sample of 'good' cluster trials with a sample of potentially biased trials. We also compared the effect sizes, in a funnel plot, between hip protector trials that used individual randomisation compared with those that used cluster randomisation. Results: There is a tendency for cluster trials, with evidence methodological biases, to also show an age imbalance between treatment groups. In a funnel plot we show that all cluster trials show a large positive effect of hip protectors whilst individually randomised trials show a range of positive and negative effects, suggesting that cluster trials may be producing a biased estimate of effect. Conclusion: Methodological biases in the design and execution of cluster randomised trials is frequent. Some of these biases associated with the use of cluster designs can be avoided through careful attention to the design of cluster trials. Firstly, if possible, individual allocation should be used. Secondly, if cluster allocation is required, then ideally participants should be identified before random allocation of the clusters. Third, if prior identification is not possible, then an independent recruiter should be used to recruit participants

Prostaglandin insert dinoprostone versus trans-cervical balloon catheter for outpatient labour induction: a randomised controlled trial of feasibility (PROBIT-F)

Background The aim was to assess the feasibility of conducting a randomised controlled trial (RCT) of induction of labour comparing use of two methods in the outpatient setting. Methods An open-label feasibility RCT was conducted in two UK maternity units from October 2017 to March 2019. Women aged ≥ 16 years, undergoing induction of labour (IOL) at term, with intact membranes and deemed suitable for outpatient IOL according to local guidelines were considered eligible. They were randomised to cervical ripening balloon catheter (CRB) or vaginal dinoprostone (Propess). The participants completed a questionnaire and a sub-group underwent detailed interview. Service use and cost data were collected via the Adult Service Use Schedule (AD-SUS). Women who declined to participate were requested to complete a decliners’ questionnaire. Results During the study period, 274 eligible women were identified. Two hundred thirty (83.9%) were approached for participation of whom 84/230 (36.5%) agreed and 146 did not. Of these, 38 were randomised to Propess (n = 20) and CRB (n = 18). Decliner data were collected for 93 women. The reasons for declining were declining IOL (n = 22), preference for inpatient IOL (n = 22) and preference for a specific method, Propess (n = 19). The intended sample size of 120 was not reached due to restrictive criteria for suitability for outpatient IOL, participant preference for Propess and shortage of research staff. The intervention as randomised was received by 29/38 (76%) women. Spontaneous vaginal delivery was observed in 9/20 (45%) women in the dinoprostone group and 11/18 (61%) women in the CRB group. Severe maternal adverse events were recorded in one woman in each group. All babies were born with good condition and all except one (37/38, 97.4%) remained with the mother after delivery. No deaths were recorded. − 21% of women in the dinoprostone group were re-admitted prior to diagnosis of active labour compared to 12% in the CRB group. Conclusions A third of the approached eligible women agreed for randomisation. An RCT is not feasible in the current service context. Modifications to the eligibility criteria for outpatient IOL, better information provision and round the clock availability of research staff would be needed to reach sufficient numbers

Increased risk of type I errors in cluster randomised trials with small or medium numbers of clusters: a review, reanalysis,and simulation study

Background: Cluster randomised trials (CRTs) are commonly analysed using mixed-effects models or generalised estimating equations (GEEs). However, these analyses do not always perform well with the small number of clusters typical of most CRTs. They can lead to increased risk of a type I error (finding a statistically significant treatment effect when it does not exist) if appropriate corrections are not used. Methods: We conducted a small simulation study to evaluate the impact of using small-sample corrections for mixed-effects models or GEEs in CRTs with a small number of clusters. We then reanalysed data from TRIGGER, a CRT with six clusters, to determine the effect of using an inappropriate analysis method in practice. Finally, we reviewed 100 CRTs previously identified by a search on PubMed in order to assess whether trials were using appropriate methods of analysis. Trials were classified as at risk of an increased type I error rate if they did not report using an analysis method which accounted for clustering, or if they had fewer than 40 clusters and performed an individual-level analysis without reporting the use of an appropriate small-sample correction. Results: Our simulation study found that using mixed-effects models or GEEs without an appropriate correction led to inflated type I error rates, even for as many as 70 clusters. Conversely, using small-sample corrections provided correct type I error rates across all scenarios. Reanalysis of the TRIGGER trial found that inappropriate methods of analysis gave much smaller P values (P ≤ 0.01) than appropriate methods (P = 0.04–0.15). In our review, of the 99 trials that reported the number of clusters, 64 (65 %) were at risk of an increased type I error rate; 14 trials did not report using an analysis method which accounted for clustering, and 50 trials with fewer than 40 clusters performed an individual-level analysis without reporting the use of an appropriate correction. Conclusions: CRTs with a small or medium number of clusters are at risk of an inflated type I error rate unless appropriate analysis methods are used. Investigators should consider using small-sample corrections with mixed-effects models or GEEs to ensure valid results. Abbreviations: CRT, Cluster randomised trial; CI, Confidence interval; GEE, Generalised estimating equations; TRIGGER, Trial in Gastrointestinal Transfusio

Clustering in surgical trials : database of intracluster correlations

PMID: 22217216 [PubMed - indexed for MEDLINE] PMCID: PMC3311136 Free PMC ArticlePeer reviewedPublisher PD

Electronic search strategies to identify reports of cluster randomized trials in MEDLINE: low precision will improve with adherence to reporting standards

BACKGROUND: Cluster randomized trials (CRTs) present unique methodological and ethical challenges. Researchers conducting systematic reviews of CRTs (e.g., addressing methodological or ethical issues) require efficient electronic search strategies (filters or hedges) to identify trials in electronic databases such as MEDLINE. According to the CONSORT statement extension to CRTs, the clustered design should be clearly identified in titles or abstracts; however, variability in terminology may make electronic identification challenging. Our objectives were to (a) evaluate sensitivity ( recall ) and precision of a well-known electronic search strategy ( randomized controlled trial as publication type) with respect to identifying CRTs, (b) evaluate the feasibility of new search strategies targeted specifically at CRTs, and (c) determine whether CRTs are appropriately identified in titles or abstracts of reports and whether there has been improvement over time. METHODS: We manually examined a wide range of health journals to identify a gold standard set of CRTs. Search strategies were evaluated against the gold standard set, as well as an independent set of CRTs included in previous systematic reviews. RESULTS: The existing strategy (randomized controlled trial.pt) is sensitive (93.8%) for identifying CRTs, but has relatively low precision (9%, number needed to read 11); the number needed to read can be halved to 5 (precision 18.4%) by combining with cluster design-related terms using the Boolean operator AND; combining with the Boolean operator OR maximizes sensitivity (99.4%) but would require 28.6 citations read to identify one CRT. Only about 50% of CRTs are clearly identified as cluster randomized in titles or abstracts; approximately 25% can be identified based on the reported units of randomization but are not amenable to electronic searching; the remaining 25% cannot be identified except through manual inspection of the full-text article. The proportion of trials clearly identified has increased from 28% between the years 2000-2003, to 60% between 2004-2007 (absolute increase 32%, 95% CI 17 to 47%). CONCLUSIONS: CRTs should include the phrase cluster randomized trial in titles or abstracts; this will facilitate more accurate indexing of the publication type by reviewers at the National Library of Medicine, and efficient textword retrieval of the subset employing cluster randomization

Design effect in multicenter studies: gain or loss of power?

<p>Abstract</p> <p>Background</p> <p>In a multicenter trial, responses for subjects belonging to a common center are correlated. Such a clustering is usually assessed through the design effect, defined as a ratio of two variances. The aim of this work was to describe and understand situations where the design effect involves a gain or a loss of power.</p> <p>Methods</p> <p>We developed a design effect formula for a multicenter study aimed at testing the effect of a binary factor (which thus defines two groups) on a continuous outcome, and explored this design effect for several designs (from individually stratified randomized trials to cluster randomized trials, and for other designs such as matched pair designs or observational multicenter studies).</p> <p>Results</p> <p>The design effect depends on the intraclass correlation coefficient (ICC) (which assesses the correlation between data for two subjects from the same center) but also on a statistic <it>S</it>, which quantifies the heterogeneity of the group distributions among centers (thus the level of association between the binary factor and the center) and on the degree of global imbalance (the number of subjects are then different) between the two groups. This design effect may induce either a loss or a gain in power, depending on whether the <it>S </it>statistic is respectively higher or lower than 1.</p> <p>Conclusion</p> <p>We provided a global design effect formula applying for any multicenter study and allowing identifying factors – the ICC and the distribution of the group proportions among centers – that are associated with a gain or a loss of power in such studies.</p