The challenges faced in the design, conduct and analysis of surgical randomised controlled trials

Randomised evaluations of surgical interventions are rare; some interventions have been widely adopted without rigorous evaluation. Unlike other medical areas, the randomised controlled trial (RCT) design has not become the default study design for the evaluation of surgical interventions. Surgical trials are difficult to successfully undertake and pose particular practical and methodological challenges. However, RCTs have played a role in the assessment of surgical innovations and there is scope and need for greater use. This article will consider the design, conduct and analysis of an RCT of a surgical intervention. The issues will be reviewed under three headings: the timing of the evaluation, defining the research question and trial design issues. Recommendations on the conduct of future surgical RCTs are made. Collaboration between research and surgical communities is needed to address the distinct issues raised by the assessmentof surgical interventions and enable the conduct of appropriate and well-designed trials.The Health Services Research Unit is funded by the Scottish Government Health DirectoratesPeer reviewedPublisher PD

Chlorpromazine for schizophrenia: a Cochrane systematic review of 50 years of randomised controlled trials

BACKGROUND: Chlorpromazine (CPZ) remains one of the most common drugs used for people with schizophrenia worldwide, and a benchmark against which other treatments can be evaluated. Quantitative reviews are rare; this one evaluates the effects of chlorpromazine in the treatment of schizophrenia in comparison with placebo. METHODS: We sought all relevant randomised controlled trials (RCT) comparing chlorpromazine to placebo by electronic and reference searching, and by contacting trial authors and the pharmaceutical industry. Data were extracted from selected trials and, where possible, synthesised and random effects relative risk (RR), the number needed to treat (NNT) and their 95% confidence intervals (CI) calculated. RESULTS: Fifty RCTs from 1955–2000 were included with 5276 people randomised to CPZ or placebo. They constitute 2008 person-years spent in trials. Meta-analysis of these trials showed that chlorpromazine promotes a global improvement (n = 1121, 13 RCTs, RR 0.76 CI 0.7 to 0.9, NNT 7 CI 5 to 10), although a considerable placebo response is also seen. People allocated to chlorpromazine tended not to leave trials early in both the short (n = 945, 16 RCTs, RR 0.74 CI 0.5 to 1.1) and medium term (n = 1861, 25 RCTs, RR 0.79 CI 0.6 to 1.1). There were, however, many adverse effects. Chlorpromazine is sedating (n = 1242, 18 RCTs, RR 2.3 CI 1.7 to 3.1, NNH 6 CI 5 to 8), increases a person's chances of experiencing acute movement disorders, Parkinsonism and causes low blood pressure with dizziness and dry mouth. CONCLUSION: It is understandable why the World Health Organization (WHO) have endorsed and included chlorpromazine in their list of essential drugs for use in schizophrenia. Low- and middle-income countries may have more complete evidence upon which to base their practice compared with richer nations using recent innovations

Chiropractic and children: Is more research enough?

Many health science research and review articles end with the words: "More research is needed". However, when it comes to research, it is not as much a question of quantity as of quality. There are a number of important prerequisites before research should be initiated. The three pillars, relevance, quality and ethics should be respected but for a project to be meaningful, it must also be based on plausible rationale

Current practice in analysing and reporting binary outcome data—a review of randomised controlled trial reports

Background Randomised controlled trials (RCTs) need to be reported so that their results can be unambiguously and robustly interpreted. Binary outcomes yield unique challenges, as different analytical approaches may produce relative, absolute, or no treatment effects, and results may be particularly sensitive to the assumptions made about missing data. This review of recently published RCTs aimed to identify the methods used to analyse binary primary outcomes, how missing data were handled, and how the results were reported. Methods Systematic review of reports of RCTs published in January 2019 that included a binary primary outcome measure. We identified potentially eligible English language papers on PubMed, without restricting by journal or medical research area. Papers reporting the results from individually randomised, parallel-group RCTs were included. Results Two hundred reports of RCTs were included in this review. We found that 64% of the 200 reports used a chi-squared-style test as their primary analytical method. Fifty-five per cent (95% confidence interval 48% to 62%) reported at least one treatment effect measure, and 38% presented only a p value without any treatment effect measure. Missing data were not always adequately described and were most commonly handled using available case analysis (69%) in the 140 studies that reported missing data. Imputation and best/worst-case scenarios were used in 21% of studies. Twelve per cent of articles reported an appropriate sensitivity analysis for missing data. Conclusions The statistical analysis and reporting of treatment effects in reports of randomised trials with a binary primary endpoint requires substantial improvement. Only around half of the studied reports presented a treatment effect measure, hindering the understanding and dissemination of the findings. We also found that published trials often did not clearly describe missing data or sensitivity analyses for these missing data. Practice for secondary endpoints or observational studies may differ

The face of equipoise - delivering a structured education programme within a randomized controlled trial: qualitative study

Background: In trials of behavioural interventions, the individuals who deliver the intervention are in a position of key influence on the success of the trial. Their fidelity to the intervention is crucial. Yet little is understood about the experiences of this group of trial personnel. This study aimed to investigate the views and experiences of educators who delivered a structured education intervention to people with type 2 diabetes, which incorporated training in self-monitoring of either blood glucose (SMBG) or urine glucose (SMUG) as part of a randomized controlled trial (RCT). Methods: Educators’ views were explored through focus groups before and after training (N = 18) and approximately 1 year into the trial (N = 14), and semi-structured telephone interviews at approximately 2 years (N = 7). Analysis was based on the constant comparative method. Results: Educators held preferences regarding the intervention variants; thus, they were not in individual equipoise. Training raised awareness of preferences and their potential to impact on delivery. Educators were confident in their unbiased delivery, but acknowledged the challenges involved. Concealing their preferences was helped by a sense of professionalism, the patient-centred nature of the intervention, and concessions in the trial protocol (enabling participants to swap monitoring methods if needed). Commitment to unbiased delivery was explained through a desire for evidence-based knowledge in the contentious area of SMBG. Conclusions: The findings provide insight into a previously unexplored group of trial personnel - intervention deliverers in trials of behavioural interventions - which will be useful to those designing and running similar trials. Rather than individual equipoise, it is intervention deliverers’ awareness of personal preferences and their potential impact on the trial outcome that facilitates unbiased delivery. Further, awareness of community equipoise, the need for evidence, and relevance to the individual enhance commitment to the RCT

“Excellence R Us”: university research and the fetishisation of excellence

The rhetoric of “excellence” is pervasive across the academy. It is used to refer to research outputs as well as researchers, theory and education, individuals and organisations, from art history to zoology. But does “excellence” actually mean anything? Does this pervasive narrative of “excellence” do any good? Drawing on a range of sources we interrogate “excellence” as a concept and find that it has no intrinsic meaning in academia. Rather it functions as a linguistic interchange mechanism. To investigate whether this linguistic function is useful we examine how the rhetoric of excellence combines with narratives of scarcity and competition to show that the hypercompetition that arises from the performance of “excellence” is completely at odds with the qualities of good research. We trace the roots of issues in reproducibility, fraud, and homophily to this rhetoric. But we also show that this rhetoric is an internal, and not primarily an external, imposition. We conclude by proposing an alternative rhetoric based on soundness and capacity-building. In the final analysis, it turns out that that “excellence” is not excellent. Used in its current unqualified form it is a pernicious and dangerous rhetoric that undermines the very foundations of good research and scholarship

Pre-referral rectal artesunate in severe malaria: flawed trial

<p>Abstract</p> <p>Background</p> <p>Immediate injectable treatment is essential for severe malaria. Otherwise, the afflicted risk lifelong impairment or death. In rural areas of Africa and Asia, appropriate care is often miles away. In 2009, Melba Gomes and her colleagues published the findings of a randomized, placebo-controlled trial of rectal artesunate for suspected severe malaria in such remote areas. Enrolling nearly 18,000 cases, the aim was to evaluate whether, as patients were in transit to a health facility, a pre-referral artesunate suppository blocked disease progression sufficiently to reduce these risks. The affirmative findings of this, the only trial on the issue thus far, have led the WHO to endorse rectal artesunate as a pre-referral treatment for severe malaria. In the light of its public health importance and because its scientific quality has not been assessed for a systematic review, our paper provides a detailed evaluation of the design, conduct, analysis, reporting, and practical features of this trial.</p> <p>Results</p> <p>We performed a checklist-based and an in-depth evaluation of the trial. The evaluation criteria were based on the CONSORT statement for reporting clinical trials, the clinical trial methodology literature, and practice in malaria research. Our main findings are: The inclusion and exclusion criteria and the sample size justification are not stated. Many clearly ineligible subjects were enrolled. The training of the recruiters does not appear to have been satisfactory. There was excessive between center heterogeneity in design and conduct. Outcome evaluation schedule was not defined, and in practice, became too wide. Large gaps in the collection of key data were evident. Primary endpoints were inconsistently utilized and reported; an overall analysis of the outcomes was not done; analyses of time to event data had major flaws; the stated intent-to-treat analysis excluded a third of the randomized subjects; the design-indicated stratified or multi-variate analysis was not done; many improper subgroups were analyzed in a post-hoc fashion; the analysis and reporting metric was deficient. There are concerns relating to patient welfare at some centers. Exclusion of many cases from data analysis compromised external validity. A bias-controlled reanalysis of available data does not lend support to the conclusions drawn by the authors.</p> <p>Conclusions</p> <p>This trial has numerous serious deficiencies in design, implementation, and methods of data analysis. Interpretation and manner of reporting are wanting, and the applicability of the findings is unclear. The trial conduct could have been improved to better protect patient welfare. The totality of these problems make it a flawed study whose conclusions remain subject to appreciable doubt.</p

Outcome based subgroup analysis: a neglected concern

A subgroup of clinical trial subjects identified by baseline characteristics is a proper subgroup while a subgroup determined by post randomization events or measures is an improper subgroup. Both types of subgroups are often analyzed in clinical trial papers. Yet, the extensive scrutiny of subgroup analyses has almost exclusively attended to the former. The analysis of improper subgroups thereby not only flourishes in numerous disguised ways but also does so without a corresponding awareness of its pitfalls. Comparisons of the grade of angina in a heart disease trial, for example, usually include only the survivors. This paper highlights some of the distinct ways in which outcome based subgroup analysis occurs, describes the hazards associated with it, and proposes a simple alternative approach to counter its analytic bias. Data from six published trials show that outcome based subgroup analysis, like proper subgroup analysis, may be performed in a post-hoc fashion, overdone, selectively reported, and over interpreted. Six hypothetical trial scenarios illustrate the forms of hidden bias related to it. That bias can, however, be addressed by assigning clinically appropriate scores to the usually excluded subjects and performing an analysis that includes all the randomized subjects. A greater level of awareness about the practice and pitfalls of outcome based subgroup analysis is needed. When required, such an analysis should maintain the integrity of randomization. This issue needs greater practical and methodologic attention than has been accorded to it thus far