Cluster randomised trials in the medical literature: two bibliometric surveys

Background: Several reviews of published cluster randomised trials have reported that about half did not take clustering into account in the analysis, which was thus incorrect and potentially misleading. In this paper I ask whether cluster randomised trials are increasing in both number and quality of reporting. Methods: Computer search for papers on cluster randomised trials since 1980, hand search of trial reports published in selected volumes of the British Medical Journal over 20 years. Results: There has been a large increase in the numbers of methodological papers and of trial reports using the term 'cluster random' in recent years, with about equal numbers of each type of paper. The British Medical Journal contained more such reports than any other journal. In this journal there was a corresponding increase over time in the number of trials where subjects were randomised in clusters. In 2003 all reports showed awareness of the need to allow for clustering in the analysis. In 1993 and before clustering was ignored in most such trials. Conclusion: Cluster trials are becoming more frequent and reporting is of higher quality. Perhaps statistician pressure works

Comparisons against baseline within randomised groups are often used and can be highly misleading

<p>Abstract</p> <p>Background</p> <p>In randomised trials, rather than comparing randomised groups directly some researchers carry out a significance test comparing a baseline with a final measurement separately in each group.</p> <p>Methods</p> <p>We give several examples where this has been done. We use simulation to demonstrate that the procedure is invalid and also show this algebraically.</p> <p>Results</p> <p>This approach is biased and invalid, producing conclusions which are, potentially, highly misleading. The actual alpha level of this procedure can be as high as 0.50 for two groups and 0.75 for three.</p> <p>Conclusions</p> <p>Randomised groups should be compared directly by two-sample methods and separate tests against baseline are highly misleading.</p

Quality research in healthcare: are researchers getting enough statistical support?

BACKGROUND: Reviews of peer-reviewed health studies have highlighted problems with their methodological quality. As published health studies form the basis of many clinical decisions including evaluation and provisions of health services, this has scientific and ethical implications. The lack of involvement of methodologists (defined as statisticians or quantitative epidemiologists) has been suggested as one key reason for this problem and this has been linked to the lack of access to methodologists. This issue was highlighted several years ago and it was suggested that more investments were needed from health care organisations and Universities to alleviate this problem. METHODS: To assess the current level of methodological support available for health researchers in England, we surveyed the 25 National Health Services Trusts in England, that are the major recipients of the Department of Health's research and development (R&D) support funding. RESULTS AND DISCUSSION: The survey shows that the earmarking of resources to provide appropriate methodological support to health researchers in these organisations is not widespread. Neither the level of R&D support funding received nor the volume of research undertaken by these organisations showed any association with the amount they spent in providing a central resource for methodological support for their researchers. CONCLUSION: The promotion and delivery of high quality health research requires that organisations hosting health research and their academic partners put in place funding and systems to provide appropriate methodological support to ensure valid research findings. If resources are limited, health researchers may have to rely on short courses and/or a limited number of advisory sessions which may not always produce satisfactory results

Reporting of loss to follow-up information in randomised controlled trials with time-to-event outcomes: a literature survey

<p>Abstract</p> <p>Background</p> <p>To assess the reporting of loss to follow-up (LTFU) information in articles on randomised controlled trials (RCTs) with time-to-event outcomes, and to assess whether discrepancies affect the validity of study results.</p> <p>Methods</p> <p>Literature survey of all issues of the BMJ, Lancet, JAMA, and New England Journal of Medicine published between 2003 and 2005. Eligible articles were reports of RCTs including at least one Kaplan-Meier plot. Articles were classified as "assessable" if sufficient information was available to assess LTFU. In these articles, LTFU information was derived from Kaplan-Meier plots, extracted from the text, and compared. Articles were then classified as "consistent" or "not consistent". Sensitivity analyses were performed to assess the validity of study results.</p> <p>Results</p> <p>319 eligible articles were identified. 187 (59%) were classified as "assessable", as they included sufficient information for evaluation; 140 of 319 (44%) presented consistent LTFU information between the Kaplan-Meier plot and text. 47 of 319 (15%) were classified as "not consistent". These 47 articles were included in sensitivity analyses. When various imputation methods were used, the results of a chi²-test applied to the corresponding 2 × 2 table changed and hence were not robust in about half of the studies.</p> <p>Conclusions</p> <p>Less than half of the articles on RCTs using Kaplan-Meier plots provide assessable and consistent LTFU information, thus questioning the validity of the results and conclusions of many studies presenting survival analyses. Authors should improve the presentation of both Kaplan-Meier plots and LTFU information, and reviewers of study publications and journal editors should critically appraise the validity of the information provided.</p

How to spot a statistical problem: advice for a non-statistical reviewer

Statistical analyses presented in general medical journals are becoming increasingly sophisticated. BMC Medicine relies on subject reviewers to indicate when a statistical review is required. We consider this policy and provide guidance on when to recommend a manuscript for statistical evaluation. Indicators for statistical review include insufficient detail in methods or results, some common statistical issues and interpretation not based on the presented evidence. Reviewers are required to ensure that the manuscript is methodologically sound and clearly written. Within that context, they are expected to provide constructive feedback and opinion on the statistical design, analysis, presentation and interpretation. If reviewers lack the appropriate background to positively confirm the appropriateness of any of the manuscript’s statistical aspects, they are encouraged to recommend it for expert statistical review

An investigation of minimisation criteria

Minimisation can be used within treatment trials to ensure that prognostic factors are evenly distributed between treatment groups. The technique is relatively straightforward to apply but does require running tallies of patient recruitments to be made and some simple calculations to be performed prior to each allocation. As computing facilities have become more widely available, minimisation has become a more feasible option for many. Although the technique has increased in popularity, the mode of application is often poorly reported and the choice of input parameters not justified in any logical way

Corrections in an electronic environment

The need to publish corrections to scientific articles, and occasionally to retract them, has been recognized for decades. However very little emphasis has been attached to how this is done, provided that the retraction or correction is accessible. We are considering a policy to directly correct our online publications

The effects of laryngeal mask airway passage simulation training on the acquisition of undergraduate clinical skills: a randomised controlled trial

Background\ud Effective use of the laryngeal mask airway (LMA) requires learning proper insertion technique in normal patients undergoing routine surgical procedures. However, there is a move towards simulation training for learning practical clinical skills, such as LMA placement. The evidence linking different amounts of mannequin simulation training to the undergraduate clinical skill of LMA placement in real patients is limited. The purpose of this study was to compare the effectiveness in vivo of two LMA placement simulation courses of different durations. \ud \ud Methods\ud Medical students (n = 126) enrolled in a randomised controlled trial. Seventy-eight of these students completed the trial. The control group (n = 38) received brief mannequin training while the intervention group (n = 40) received additional more intensive mannequin training as part of which they repeated LMA insertion until they were proficient. The anaesthetists supervising LMA placements in real patients rated the participants' performance on assessment forms. Participants completed a self-assessment questionnaire. \ud \ud Results\ud Additional mannequin training was not associated with improved performance (37% of intervention participants received an overall placement rating of > 3/5 on their first patient compared to 48% of the control group, $X^2$ = 0.81, p = 0.37). The agreement between the participants and their instructors in terms of LMA placement success rates was poor to fair. Participants reported that mannequins were poor at mimicking reality. \ud \ud Conclusions\ud The results suggest that the value of extended mannequin simulation training in the case of LMA placement is limited. Educators considering simulation for the training of practical skills should reflect on the extent to which the in vitro simulation mimics the skill required and the degree of difficulty of the procedure. \ud \u

IMPLEmenting a clinical practice guideline for acute low back pain evidence-based manageMENT in general practice (IMPLEMENT) : cluster randomised controlled trial study protocol

Background: Evidence generated from reliable research is not frequently implemented into clinical practice. Evidence-based clinical practice guidelines are a potential vehicle to achieve this. A recent systematic review of implementation strategies of guideline dissemination concluded that there was a lack of evidence regarding effective strategies to promote the uptake of guidelines. Recommendations from this review, and other studies, have suggested the use of interventions that are theoretically based because these may be more effective than those that are not. An evidencebased clinical practice guideline for the management of acute low back pain was recently developed in Australia. This provides an opportunity to develop and test a theory-based implementation intervention for a condition which is common, has a high burden, and for which there is an evidence-practice gap in the primary care setting. Aim: This study aims to test the effectiveness of a theory-based intervention for implementing a clinical practice guideline for acute low back pain in general practice in Victoria, Australia. Specifically, our primary objectives are to establish if the intervention is effective in reducing the percentage of patients who are referred for a plain x-ray, and improving mean level of disability for patients three months post-consultation. Methods/Design: This study protocol describes the details of a cluster randomised controlled trial. Ninety-two general practices (clusters), which include at least one consenting general practitioner, will be randomised to an intervention or control arm using restricted randomisation. Patients aged 18 years or older who visit a participating practitioner for acute non-specific low back pain of less than three months duration will be eligible for inclusion. An average of twenty-five patients per general practice will be recruited, providing a total of 2,300 patient participants. General practitioners in the control arm will receive access to the guideline using the existing dissemination strategy. Practitioners in the intervention arm will be invited to participate in facilitated face-to-face workshops that have been underpinned by behavioural theory. Investigators (not involved in the delivery of the intervention), patients, outcome assessors and the study statistician will be blinded to group allocation. Trial registration: Australian New Zealand Clinical Trials Registry ACTRN012606000098538 (date registered 14/03/2006).The trial is funded by the NHMRC by way of a Primary Health Care Project Grant (334060). JF has 50% of her time funded by the Chief Scientist Office3/2006). of the Scottish Government Health Directorate and 50% by the University of Aberdeen. PK is supported by a NHMRC Health Professional Fellowship (384366) and RB by a NHMRC Practitioner Fellowship (334010). JG holds a Canada Research Chair in Health Knowledge Transfer and Uptake. All other authors are funded by their own institutions