Comparisons against baseline within randomised groups are often used and can be highly misleading

<p>Abstract</p> <p>Background</p> <p>In randomised trials, rather than comparing randomised groups directly some researchers carry out a significance test comparing a baseline with a final measurement separately in each group.</p> <p>Methods</p> <p>We give several examples where this has been done. We use simulation to demonstrate that the procedure is invalid and also show this algebraically.</p> <p>Results</p> <p>This approach is biased and invalid, producing conclusions which are, potentially, highly misleading. The actual alpha level of this procedure can be as high as 0.50 for two groups and 0.75 for three.</p> <p>Conclusions</p> <p>Randomised groups should be compared directly by two-sample methods and separate tests against baseline are highly misleading.</p

Cluster randomised trials in the medical literature: two bibliometric surveys

Background: Several reviews of published cluster randomised trials have reported that about half did not take clustering into account in the analysis, which was thus incorrect and potentially misleading. In this paper I ask whether cluster randomised trials are increasing in both number and quality of reporting. Methods: Computer search for papers on cluster randomised trials since 1980, hand search of trial reports published in selected volumes of the British Medical Journal over 20 years. Results: There has been a large increase in the numbers of methodological papers and of trial reports using the term 'cluster random' in recent years, with about equal numbers of each type of paper. The British Medical Journal contained more such reports than any other journal. In this journal there was a corresponding increase over time in the number of trials where subjects were randomised in clusters. In 2003 all reports showed awareness of the need to allow for clustering in the analysis. In 1993 and before clustering was ignored in most such trials. Conclusion: Cluster trials are becoming more frequent and reporting is of higher quality. Perhaps statistician pressure works

Quality research in healthcare: are researchers getting enough statistical support?

BACKGROUND: Reviews of peer-reviewed health studies have highlighted problems with their methodological quality. As published health studies form the basis of many clinical decisions including evaluation and provisions of health services, this has scientific and ethical implications. The lack of involvement of methodologists (defined as statisticians or quantitative epidemiologists) has been suggested as one key reason for this problem and this has been linked to the lack of access to methodologists. This issue was highlighted several years ago and it was suggested that more investments were needed from health care organisations and Universities to alleviate this problem. METHODS: To assess the current level of methodological support available for health researchers in England, we surveyed the 25 National Health Services Trusts in England, that are the major recipients of the Department of Health's research and development (R&D) support funding. RESULTS AND DISCUSSION: The survey shows that the earmarking of resources to provide appropriate methodological support to health researchers in these organisations is not widespread. Neither the level of R&D support funding received nor the volume of research undertaken by these organisations showed any association with the amount they spent in providing a central resource for methodological support for their researchers. CONCLUSION: The promotion and delivery of high quality health research requires that organisations hosting health research and their academic partners put in place funding and systems to provide appropriate methodological support to ensure valid research findings. If resources are limited, health researchers may have to rely on short courses and/or a limited number of advisory sessions which may not always produce satisfactory results

An investigation of minimisation criteria

Minimisation can be used within treatment trials to ensure that prognostic factors are evenly distributed between treatment groups. The technique is relatively straightforward to apply but does require running tallies of patient recruitments to be made and some simple calculations to be performed prior to each allocation. As computing facilities have become more widely available, minimisation has become a more feasible option for many. Although the technique has increased in popularity, the mode of application is often poorly reported and the choice of input parameters not justified in any logical way

Reporting of loss to follow-up information in randomised controlled trials with time-to-event outcomes: a literature survey

<p>Abstract</p> <p>Background</p> <p>To assess the reporting of loss to follow-up (LTFU) information in articles on randomised controlled trials (RCTs) with time-to-event outcomes, and to assess whether discrepancies affect the validity of study results.</p> <p>Methods</p> <p>Literature survey of all issues of the BMJ, Lancet, JAMA, and New England Journal of Medicine published between 2003 and 2005. Eligible articles were reports of RCTs including at least one Kaplan-Meier plot. Articles were classified as "assessable" if sufficient information was available to assess LTFU. In these articles, LTFU information was derived from Kaplan-Meier plots, extracted from the text, and compared. Articles were then classified as "consistent" or "not consistent". Sensitivity analyses were performed to assess the validity of study results.</p> <p>Results</p> <p>319 eligible articles were identified. 187 (59%) were classified as "assessable", as they included sufficient information for evaluation; 140 of 319 (44%) presented consistent LTFU information between the Kaplan-Meier plot and text. 47 of 319 (15%) were classified as "not consistent". These 47 articles were included in sensitivity analyses. When various imputation methods were used, the results of a chi²-test applied to the corresponding 2 × 2 table changed and hence were not robust in about half of the studies.</p> <p>Conclusions</p> <p>Less than half of the articles on RCTs using Kaplan-Meier plots provide assessable and consistent LTFU information, thus questioning the validity of the results and conclusions of many studies presenting survival analyses. Authors should improve the presentation of both Kaplan-Meier plots and LTFU information, and reviewers of study publications and journal editors should critically appraise the validity of the information provided.</p

How to spot a statistical problem: advice for a non-statistical reviewer

Statistical analyses presented in general medical journals are becoming increasingly sophisticated. BMC Medicine relies on subject reviewers to indicate when a statistical review is required. We consider this policy and provide guidance on when to recommend a manuscript for statistical evaluation. Indicators for statistical review include insufficient detail in methods or results, some common statistical issues and interpretation not based on the presented evidence. Reviewers are required to ensure that the manuscript is methodologically sound and clearly written. Within that context, they are expected to provide constructive feedback and opinion on the statistical design, analysis, presentation and interpretation. If reviewers lack the appropriate background to positively confirm the appropriateness of any of the manuscript’s statistical aspects, they are encouraged to recommend it for expert statistical review

Detection of emphysema progression in alpha 1-antitrypsin deficiency using CT densitometry; Methodological advances

<p>Abstract</p> <p>Background</p> <p>Computer tomography (CT) densitometry is a potential tool for detecting the progression of emphysema but the optimum methodology is uncertain. The level of inspiration affects reproducibility but the ability to adjust for this variable is facilitated by whole lung scanning methods. However, emphysema is frequently localised to sub-regions of the lung and targeted densitometric sampling may be more informative than whole lung assessment.</p> <p>Methods</p> <p>Emphysema progression over a 2-year interval was assessed in 71 patients (alpha 1-antitrypsin deficiency with PiZ phenotype) with CT densitometry, using the 15^thpercentile point (Perc15) and voxel index (VI) -950 Hounsfield Units (HU) and -910 HU (VI -950 and -910) on whole lung, limited single slices, and apical, central and basal thirds. The relationship between whole lung densitometric progression (ΔCT) and change in CT-derived lung volume (ΔCT_Vol) was characterised, and adjustment for lung volume using statistical modelling was evaluated.</p> <p>Results</p> <p>CT densitometric progression was statistically significant for all methods. ΔCT correlated with ΔCT_Voland linear regression indicated that nearly one half of lung density loss was secondary to apparent hyperinflation. The most accurate measure was obtained using a random coefficient model to adjust for lung volume and the greatest progression was detected by targeted sampling of the middle third of the lung.</p> <p>Conclusion</p> <p>Progressive hyperinflation may contribute significantly to loss of lung density, but volume effects and absolute tissue loss can be identified by statistical modelling. Targeted sampling of the middle lung region using Perc15 appears to be the most robust measure of emphysema progression.</p

A kinematic analysis of a haptic handheld stylus in a virtual environment: a study in healthy subjects

<p>Abstract</p> <p>Background</p> <p>Virtual Reality provides new options for conducting motor assessment and training within computer-generated 3 dimensional environments. To date very little has been reported about normal performance in virtual environments. The objective of this study was to evaluate the test-retest reliability of a clinical procedure measuring trajectories with a haptic handheld stylus in a virtual environment and to establish normative data in healthy subjects using this haptic device.</p> <p>Methods</p> <p>Fifty-eight normal subjects; aged from 20 to 69, performed 3 dimensional hand movements in a virtual environment using a haptic device on three occasions within one week. Test-retest stability and standardized normative data were obtained for all subjects.</p> <p>Results</p> <p>No difference was found between test and retest. The limits of agreement revealed that changes in an individual's performance could not be detected. There was a training effect between the first test occasion and the third test occasion. Normative data are presented.</p> <p>Conclusion</p> <p>A new test was developed for recording the kinematics of the handheld haptic stylus in a virtual environment. The normative data will be used for purposes of comparison in future assessments, such as before and after training of persons with neurological deficits.</p

How accurate is an LCD screen version of the Pelli–Robson test?

Purpose: To evaluate the accuracy and repeatability of a computer-generated Pelli–Robson test displayed on liquid crystal display (LCD) systems compared to a standard Pelli–Robson chart. Methods: Two different randomized crossover experiments were carried out for two different LCD systems for 32 subjects: 6 females and 10 males (40.5 ± 13.0 years) and 9 females and 7 males (27.8 ± 12.2 years), respectively, in the first and second experiment. Two repeated measurements were taken with the printed Pelli–Robson test and with the LCDs at 1 and 3 m. To test LCD reliability, measurements were repeated after 1 week. Results: In Experiment 1, contrast sensitivity (CS) measured with LCD1 resulted significantly higher than Pelli–Robson both at 1 and at 3 m of about 0.20 log 1/C in both eyes (p < 0.01). Bland–Altman plots showed a proportional bias for LCD1 measures. LCD1 measurements showed reasonable repeatability: ICC was 0.83 and 0.65 at 1 and 3 m, respectively. In Experiment 2, CS measured with LCD2 resulted significantly lower than Pelli–Robson both at 1 and at 3 m of about 0.10 log 1/C in both eyes (p < 0.01). Bland–Altman plots did not show any proportional bias for LCD2 measures. LCD2 measurements showed sufficient repeatability: ICC resulted 0.51 and 0.65 at 1 and 3 m, respectively. Conclusions: Computer-generated versions of Pelli–Robson test, displayed on LCD systems, do not provide accurate results compared to classic Pelli–Robson printed version. Clinicians should consider that Pelli–Robson computer-generated versions could be non-interchangeable to the printed version