Methodological criteria for the assessment of moderators in systematic reviews of randomised controlled trials : a consensus study

Background: Current methodological guidelines provide advice about the assessment of sub-group analysis within RCTs, but do not specify explicit criteria for assessment. Our objective was to provide researchers with a set of criteria that will facilitate the grading of evidence for moderators, in systematic reviews. Method: We developed a set of criteria from methodological manuscripts (n = 18) using snowballing technique, and electronic database searches. Criteria were reviewed by an international Delphi panel (n = 21), comprising authors who have published methodological papers in this area, and researchers who have been active in the study of sub-group analysis in RCTs. We used the Research ANd Development/University of California Los Angeles appropriateness method to assess consensus on the quantitative data. Free responses were coded for consensus and disagreement. In a subsequent round additional criteria were extracted from the Cochrane Reviewers’ Handbook, and the process was repeated. Results: The recommendations are that meta-analysts report both confirmatory and exploratory findings for subgroups analysis. Confirmatory findings must only come from studies in which a specific theory/evidence based apriori statement is made. Exploratory findings may be used to inform future/subsequent trials. However, for inclusion in the meta-analysis of moderators, the following additional criteria should be applied to each study: Baseline factors should be measured prior to randomisation, measurement of baseline factors should be of adequate reliability and validity, and a specific test of the interaction between baseline factors and interventions must be presented. Conclusions: There is consensus from a group of 21 international experts that methodological criteria to assess moderators within systematic reviews of RCTs is both timely and necessary. The consensus from the experts resulted in five criteria divided into two groups when synthesising evidence: confirmatory findings to support hypotheses about moderators and exploratory findings to inform future research. These recommendations are discussed in reference to previous recommendations for evaluating and reporting moderator studies

How to spot a statistical problem: advice for a non-statistical reviewer

Statistical analyses presented in general medical journals are becoming increasingly sophisticated. BMC Medicine relies on subject reviewers to indicate when a statistical review is required. We consider this policy and provide guidance on when to recommend a manuscript for statistical evaluation. Indicators for statistical review include insufficient detail in methods or results, some common statistical issues and interpretation not based on the presented evidence. Reviewers are required to ensure that the manuscript is methodologically sound and clearly written. Within that context, they are expected to provide constructive feedback and opinion on the statistical design, analysis, presentation and interpretation. If reviewers lack the appropriate background to positively confirm the appropriateness of any of the manuscript’s statistical aspects, they are encouraged to recommend it for expert statistical review

Estimating measures of interaction on an additive scale for preventive exposures

Measures of interaction on an additive scale (relative excess risk due to interaction [RERI], attributable proportion [AP], synergy index [S]), were developed for risk factors rather than preventive factors. It has been suggested that preventive factors should be recoded to risk factors before calculating these measures. We aimed to show that these measures are problematic with preventive factors prior to recoding, and to clarify the recoding method to be used to circumvent these problems. Recoding of preventive factors should be done such that the stratum with the lowest risk becomes the reference category when both factors are considered jointly (rather than one at a time). We used data from a case-control study on the interaction between ACE inhibitors and the ACE gene on incident diabetes. Use of ACE inhibitors was a preventive factor and DD ACE genotype was a risk factor. Before recoding, the RERI, AP and S showed inconsistent results (RERI = 0.26 [95%CI: −0.30; 0.82], AP = 0.30 [95%CI: −0.28; 0.88], S = 0.35 [95%CI: 0.02; 7.38]), with the first two measures suggesting positive interaction and the third negative interaction. After recoding the use of ACE inhibitors, they showed consistent results (RERI = −0.37 [95%CI: −1.23; 0.49], AP = −0.29 [95%CI: −0.98; 0.40], S = 0.43 [95%CI: 0.07; 2.60]), all indicating negative interaction. Preventive factors should not be used to calculate measures of interaction on an additive scale without recoding

GEIRA: gene-environment and gene–gene interaction research application

The GEIRA (Gene-Environment and Gene–Gene Interaction Research Application) algorithm and subsequent program is dedicated to genome-wide gene-environment and gene–gene interaction analysis. It implements concepts of both additive and multiplicative interaction as well as calculations based on dominant, recessive and co-dominant genetic models, respectively. Estimates of interactions are incorporated in a single table to make the output easily read. The algorithm is coded in both SAS and R. GEIRA is freely available to non-commercial users at http://www.epinet.se. Additional information, including user’s manual and example datasets is available online at http://www.epinet.se

Tamoxifen is not effective in good prognosis patients with hepatocellular carcinoma

BACKGROUND: Large randomised clinical trials and systematic reviews substantiate that tamoxifen is ineffective in improving survival of patients with hepatocellular carcinoma (HCC). However, a recent report suggested that the drug might prolong survival among patients with well preserved liver function. The aim of this paper is to validate this hypothesis. METHODS: We used the updated database of the phase 3 randomised CLIP-1 trial that compared tamoxifen with supportive therapy. Primary endpoint was overall survival. Treatment arms were compared within strata defined according to the Okuda stage and the CLIP-score. Survival differences were tested by the Log-rank test. RESULTS: Tamoxifen was not effective in prolonging survival in Okuda I-II subgroup (p = 0.501). Median survival times were equal to 16.8 (95%CI 12.7–18.5) months for tamoxifen and 16.8 (95%CI 13.5–22.4) months for the control arms; 1-year survival probabilities were equal to 58.8% (95%CI 51.7–65.8) and 59.4 (95%CI 52.5–66.2), respectively. Similar results were observed in the better CLIP subgroup (score 0/1), without evidence of difference between the two treatment arms (p = 0.734). Median survival times were equal to 29.2 (95%CI 20.1–36.4) months with tamoxifen and 29.0 (95%CI 23.3–35.2) months without; 1-year survival probabilities were equal to 80.9% (95%CI 72.5–89.3) with tamoxifen and 77.1% (95%CI 68.6–85.7) for the control arm. CONCLUSION: The recent suggestion that tamoxifen might be effective in the subgroup of patients with better prognosis is not supported by a reanalysis of the CLIP-1 trial. Tamoxifen should no longer be considered for the treatment of HCC patients and future trials of medical treatment should concentrate on different drugs

Outcome based subgroup analysis: a neglected concern

A subgroup of clinical trial subjects identified by baseline characteristics is a proper subgroup while a subgroup determined by post randomization events or measures is an improper subgroup. Both types of subgroups are often analyzed in clinical trial papers. Yet, the extensive scrutiny of subgroup analyses has almost exclusively attended to the former. The analysis of improper subgroups thereby not only flourishes in numerous disguised ways but also does so without a corresponding awareness of its pitfalls. Comparisons of the grade of angina in a heart disease trial, for example, usually include only the survivors. This paper highlights some of the distinct ways in which outcome based subgroup analysis occurs, describes the hazards associated with it, and proposes a simple alternative approach to counter its analytic bias. Data from six published trials show that outcome based subgroup analysis, like proper subgroup analysis, may be performed in a post-hoc fashion, overdone, selectively reported, and over interpreted. Six hypothetical trial scenarios illustrate the forms of hidden bias related to it. That bias can, however, be addressed by assigning clinically appropriate scores to the usually excluded subjects and performing an analysis that includes all the randomized subjects. A greater level of awareness about the practice and pitfalls of outcome based subgroup analysis is needed. When required, such an analysis should maintain the integrity of randomization. This issue needs greater practical and methodologic attention than has been accorded to it thus far

Reporting on covariate adjustment in randomised controlled trials before and after revision of the 2001 CONSORT statement: a literature review

<p>Abstract</p> <p>Objectives</p> <p>To evaluate the use and reporting of adjusted analysis in randomised controlled trials (RCTs) and compare the quality of reporting before and after the revision of the CONSORT Statement in 2001.</p> <p>Design</p> <p>Comparison of two cross sectional samples of published articles.</p> <p>Data Sources</p> <p>Journal articles indexed on PubMed in December 2000 and December 2006.</p> <p>Study Selection</p> <p>Parallel group RCTs with a full publication carried out in humans and published in English</p> <p>Main outcome measures</p> <p>Proportion of articles reported adjusted analysis; use of adjusted analysis; the reason for adjustment; the method of adjustment and the reporting of adjusted analysis results in the main text and abstract.</p> <p>Results</p> <p>In both cohorts, 25% of studies reported adjusted analysis (84/355 in 2000 vs 113/422 in 2006). Compared with articles reporting only unadjusted analyses, articles that reported adjusted analyses were more likely to specify primary outcomes, involve multiple centers, perform stratified randomization, be published in general medical journals, and recruit larger sample sizes. In both years a minority of articles explained why and how covariates were selected for adjustment (20% to 30%). Almost all articles specified the statistical methods used for adjustment (99% in 2000 vs 100% in 2006) but only 5% and 10%, respectively, reported both adjusted and unadjusted results as recommended in the CONSORT guidelines.</p> <p>Conclusion</p> <p>There was no evidence of change in the reporting of adjusted analysis results five years after the revision of the CONSORT Statement and only a few articles adhered fully to the CONSORT recommendations.</p

Protocol for a randomised controlled trial examining the impact of a web-based personally controlled health management system on the uptake of influenza vaccination rates

<p>Abstract</p> <p>Background</p> <p>Online social networking and personally controlled health management systems (PCHMS) offer a new opportunity for developing innovative interventions to prevent diseases of public health concern (e.g., influenza) but there are few comparative studies about patterns of use and impact of these systems.</p> <p>Methods/Design</p> <p>A 2010 CONSORT-compliant randomised controlled trial with a two-group parallel design will assess the efficacy of a web-based PCHMS called <it>Healthy.me </it>in facilitating the uptake of influenza vaccine amongst university students and staff. Eligible participants are randomised either to obtain access to <it>Healthy.me </it>or a 6-month waitlist. Participants complete pre-study, post-study and monthly surveys about their health and utilisation of health services. A post-study clinical audit will be conducted to validate self-reports about influenza vaccination and visits to the university health service due to influenza-like illness (ILI) amongst a subset of participants. 600 participants older than 18 years with monthly access to the Internet and email will be recruited. Participants who (i) discontinue the online registration process; (ii) report obtaining an influenza vaccination in 2010 before the commencement of the study; or (iii) report being influenced by other participants to undertake influenza vaccination will be excluded from analysis. The primary outcome measure is the number of participants obtaining influenza vaccination during the study. Secondary outcome measures include: number of participants (i) experiencing ILI symptoms, (ii) absent from or experiencing impairment in work or study due to ILI symptoms, (iii) using health services or medications due to ILI symptoms; (iv) expressing positive or negative attitudes or experiences towards influenza vaccination, via their reasons of receiving (or not receiving) influenza vaccine; and (v) their patterns of usage of <it>Healthy.me </it>(e.g., frequency and timing of hits, duration of access, uptake of specific functions).</p> <p>Discussion</p> <p>This study will provide new insights about the utility of online social networking and PCHMS for public health and health promotion. It will help to assess whether a web-based PCHMS, with connectivity to a health service provider, containing information and self-management tools, can improve the uptake of preventive health services amongst university students and staff.</p> <p>Trial registration</p> <p><a href="http://www.anzctr.org.au/ACTRN12610000386033.aspx">ACTRN12610000386033</a> (Australian New Zealand Clinical Trials Registry)</p