An empirical study using permutation-based resampling in meta-regression

<p>Abstract</p> <p>Background</p> <p>In meta-regression, as the number of trials in the analyses decreases, the risk of false positives or false negatives increases. This is partly due to the assumption of normality that may not hold in small samples. Creation of a distribution from the observed trials using permutation methods to calculate <it>P </it>values may allow for less spurious findings. Permutation has not been empirically tested in meta-regression. The objective of this study was to perform an empirical investigation to explore the differences in results for meta-analyses on a small number of trials using standard large sample approaches verses permutation-based methods for meta-regression.</p> <p>Methods</p> <p>We isolated a sample of randomized controlled clinical trials (RCTs) for interventions that have a small number of trials (herbal medicine trials). Trials were then grouped by herbal species and condition and assessed for methodological quality using the Jadad scale, and data were extracted for each outcome. Finally, we performed meta-analyses on the primary outcome of each group of trials and meta-regression for methodological quality subgroups within each meta-analysis. We used large sample methods and permutation methods in our meta-regression modeling. We then compared final models and final <it>P </it>values between methods.</p> <p>Results</p> <p>We collected 110 trials across 5 intervention/outcome pairings and 5 to 10 trials per covariate. When applying large sample methods and permutation-based methods in our backwards stepwise regression the covariates in the final models were identical in all cases. The <it>P </it>values for the covariates in the final model were larger in 78% (7/9) of the cases for permutation and identical for 22% (2/9) of the cases.</p> <p>Conclusions</p> <p>We present empirical evidence that permutation-based resampling may not change final models when using backwards stepwise regression, but may increase <it>P </it>values in meta-regression of multiple covariates for relatively small amount of trials.</p

Applying the Risk of Bias Tool in a Systematic Review of Combination Long-Acting Beta-Agonists and Inhaled Corticosteroids for Persistent Asthma

Background: The Risk of Bias (RoB) tool is used to assess internal validity of randomized controlled trials (RCTs). Our objectives were to: 1) evaluate inter-rater agreement of the RoB tool; 2) determine the time to access supplemental study information; 3) compare the RoB tool with the Jadad scale and Schulz allocation concealment (AC); and 4) examine the relationship between RoB and effect estimates. Methods: We conducted a systematic review of long-acting beta agonists (LABA) combined with inhaled corticosteroids (ICS) for adults with persistent asthma. Two reviewers independently assessed 107 trials using RoB, Jadad, and AC. One reviewer searched for study protocols. We assessed inter-rater agreement using weighted Kappa (k) and the correlation between tools using Kendall’s Tau (t). Mean differences in effect sizes for RCTs with different RoB were calculated using inverse variance method and random effects model. Results: Trials had good Jadad scores (median 4, IQR 3-4); however, 85 % had unclear AC and 87 % high RoB. The factor that most influenced RoB was the potential inappropriate influence of study sponsors (95 % industry funded). Agreement on RoB domains was fair (k = 0.40) to almost perfect (k = 0.86), and moderate for overall RoB (k = 0.41). Median time to complete RoB assessments was 21 minutes (IQR 14-27) and 12 minutes (IQR 9-16) to search for protocols. Protocols were identified for 5/42 studies (12%); in 3 cases the assessment of selective outcome reporting changed. There was low correlation between overall RoB vs. Jadad (t =0.04

Helping editors, peer reviewers and authors improve the clarity, completeness and transparency of reporting health research

Inadequate reporting is problematic for several reasons. If authors do not provide sufficient details concerning the conduct of their study, readers are left with an incomplete picture of what was done. As such, they are not able to judge the merits of the results and interpret them. The EQUATOR Network is a new initiative aimed at improving the clarity and transparency of reporting health research

A meta-analytic review of stand-alone interventions to improve body image

Objective Numerous stand-alone interventions to improve body image have been developed. The present review used meta-analysis to estimate the effectiveness of such interventions, and to identify the specific change techniques that lead to improvement in body image. Methods The inclusion criteria were that (a) the intervention was stand-alone (i.e., solely focused on improving body image), (b) a control group was used, (c) participants were randomly assigned to conditions, and (d) at least one pretest and one posttest measure of body image was taken. Effect sizes were meta-analysed and moderator analyses were conducted. A taxonomy of 48 change techniques used in interventions targeted at body image was developed; all interventions were coded using this taxonomy. Results The literature search identified 62 tests of interventions (N = 3,846). Interventions produced a small-to-medium improvement in body image (d+ = 0.38), a small-to-medium reduction in beauty ideal internalisation (d+ = -0.37), and a large reduction in social comparison tendencies (d+ = -0.72). However, the effect size for body image was inflated by bias both within and across studies, and was reliable but of small magnitude once corrections for bias were applied. Effect sizes for the other outcomes were no longer reliable once corrections for bias were applied. Several features of the sample, intervention, and methodology moderated intervention effects. Twelve change techniques were associated with improvements in body image, and three techniques were contra-indicated. Conclusions The findings show that interventions engender only small improvements in body image, and underline the need for large-scale, high-quality trials in this area. The review identifies effective techniques that could be deployed in future interventions

Reporting on covariate adjustment in randomised controlled trials before and after revision of the 2001 CONSORT statement: a literature review

<p>Abstract</p> <p>Objectives</p> <p>To evaluate the use and reporting of adjusted analysis in randomised controlled trials (RCTs) and compare the quality of reporting before and after the revision of the CONSORT Statement in 2001.</p> <p>Design</p> <p>Comparison of two cross sectional samples of published articles.</p> <p>Data Sources</p> <p>Journal articles indexed on PubMed in December 2000 and December 2006.</p> <p>Study Selection</p> <p>Parallel group RCTs with a full publication carried out in humans and published in English</p> <p>Main outcome measures</p> <p>Proportion of articles reported adjusted analysis; use of adjusted analysis; the reason for adjustment; the method of adjustment and the reporting of adjusted analysis results in the main text and abstract.</p> <p>Results</p> <p>In both cohorts, 25% of studies reported adjusted analysis (84/355 in 2000 vs 113/422 in 2006). Compared with articles reporting only unadjusted analyses, articles that reported adjusted analyses were more likely to specify primary outcomes, involve multiple centers, perform stratified randomization, be published in general medical journals, and recruit larger sample sizes. In both years a minority of articles explained why and how covariates were selected for adjustment (20% to 30%). Almost all articles specified the statistical methods used for adjustment (99% in 2000 vs 100% in 2006) but only 5% and 10%, respectively, reported both adjusted and unadjusted results as recommended in the CONSORT guidelines.</p> <p>Conclusion</p> <p>There was no evidence of change in the reporting of adjusted analysis results five years after the revision of the CONSORT Statement and only a few articles adhered fully to the CONSORT recommendations.</p

Results from a blind and a non-blind randomised trial run in parallel: experience from the Estonian Postmenopausal Hormone Therapy (EPHT) Trial

<p>Abstract</p> <p>Background</p> <p>The Estonian Postmenopausal Hormone Therapy (EPHT) Trial assigned 4170 potential participants prior to recruitment to blind or non-blind hormone therapy (HT), with placebo or non-treatment the respective alternatives. Before having to decide on participation, women were told whether they had been randomised to the blind or non-blind trial. Eligible women who were still willing to join the trial were recruited. After recruitment participants in the non-blind trial (N = 1001) received open-label HT or no treatment, participants in the blind trial (N = 777) remained blinded until the end of the trial. The aim of this paper is to analyse the effect of blinding on internal and external validity of trial outcomes.</p> <p>Methods</p> <p>Effect of blinding was calculated as the hazard ratio of selected chronic diseases, total mortality and all outcomes. For analysing the effect of blinding on external validity, the hazard ratios from women recruited to the placebo arm and to the non-treatment arm were compared with those not recruited; for analysing the effect of blinding on internal validity, the hazard ratios from the blind trial were compared with those from the non-blind trial.</p> <p>Results</p> <p>The women recruited to the placebo arm had less cerebrovascular disease events (HR 0.43; 95% CI: 0.26-0.71) and all outcomes combined (HR 0.76; 95% CI: 0.63-0.91) than those who were not recruited. Among women recruited or not recruited to the non-treatment arm, no differences were observed for any of the outcomes studied.</p> <p>Among women recruited to the trial, the risk for coronary heart disease events (HR 0.77; 95% CI: 0.64-0.93), cerebrovascular disease events (HR 0.66; 95%CI: 0.47-0.92), and all outcomes combined (HR 0.82; 95% CI: 0.72-0.94) was smaller among participants in the blind trial than in the non-blind trial. There was no difference between the blind and the non-blind trial for total cancer (HR 0.95; 95% CI: 0.64-1.42), bone fractures (0.93; 95% CI: 0.74-1.16), and total mortality (HR 1.03; 95% CI: 0.53-1.98).</p> <p>Conclusions</p> <p>The results from blind and non-blind trials may differ, even if the target population is the same. Blinding may influence both internal and external validity. The effect of blinding may vary for different outcome events.</p> <p>Trial registration</p> <p>[<a href="http://www.controlled-trials.com/ISRCTN35338757">ISRCTN35338757</a>]</p

Recommendations by Cochrane Review Groups for assessment of the risk of bias in studies

<p>Abstract</p> <p>Background</p> <p>Assessing the risk of bias in individual studies in a systematic review can be done using individual components or by summarizing the study quality in an overall score.</p> <p>Methods</p> <p>We examined the instructions to authors of the 50 Cochrane Review Groups that focus on clinical interventions for recommendations on methodological quality assessment of studies.</p> <p>Results</p> <p>Forty-one of the review groups (82%) recommended quality assessment using components and nine using a scale. All groups recommending components recommended to assess concealment of allocation, compared to only two of the groups recommending scales (P < 0.0001). Thirty-five groups (70%) recommended assessment of sequence generation and 21 groups (42%) recommended assessment of intention-to-treat analysis. Only 28 groups (56%) had specific recommendations for using the quality assessment of studies analytically in reviews, with sensitivity analysis, quality as an inclusion threshold and subgroup analysis being the most commonly recommended methods. The scales recommended had problems in the individual items and some of the groups recommending components recommended items not related to bias in their quality assessment.</p> <p>Conclusion</p> <p>We found that recommendations by some groups were not based on empirical evidence and many groups had no recommendations on how to use the quality assessment in reviews. We suggest that all Cochrane Review Groups refer to the Cochrane Handbook for Systematic Reviews of Interventions, which is evidence-based, in their instructions to authors and that their own guidelines are kept to a minimum and describe only how methodological topics that are specific to their fields should be handled.</p

Self-management support interventions to reduce health care utilisation without compromising outcomes: a systematic review and meta-analysis

Background: There is increasing interest in the role of �self-management� interventions to support the management of long-term conditions in health service settings. Self-management may include patient education, support for decision-making, self-monitoring and psychological and social support. Self-management support has potential to improve the efficiency of health services by reducing other forms of utilisation (such as primary care or hospital use), but a shift to self-management may lead to negative outcomes, such as patients who feel more anxious about their health, are less able to cope, or who receive worse quality of care, all of which may impact on their health and quality of life. We sought to determine which models of self-management support are associated with significant reductions in health services utilisation without compromising outcomes among patients with long-term conditions. Methods: We used systematic review with meta-analysis. We included randomised controlled trials in patients with long-term conditions which included self-management support interventions and reported measures of service utilisation or costs, as well as measures of health outcomes (standardized disease specific quality of life, generic quality of life, or depression/anxiety).We searched multiple databases (CENTRAL, CINAHL, Econlit, EMBASE, HEED, MEDLINE, NHS EED and PsycINFO) and the reference lists of published reviews. We calculated effects sizes for both outcomes and costs, and presented the results in permutation plots, as well as conventional meta-analyses. Results: We included 184 studies. Self-management support was associated with small but significant improvements in health outcomes, with the best evidence of effectiveness in patients with diabetic, respiratory, cardiovascular and mental health conditions. Only a minority of self-management support interventions reported reductions in health care utilisation in association with decrements in health. Evidence for reductions in utilisation associated with self-management support was strongest in respiratory and cardiovascular problems. Studies at higher risk of bias were more likely to report benefits. Conclusions: Self-management support interventions can reduce health service utilization without compromising patient health outcomes, although effects were generally small, and the evidence was strongest in respiratory and cardiovascular disorders. Further work is needed to determine which components of self-management support are most effective

The assessment of the quality of reporting of meta-analyses in diagnostic research: a systematic review

<p>Abstract</p> <p>Background</p> <p>Over the last decade there have been a number of guidelines published, aimed at improving the quality of reporting in published studies and reviews. In systematic reviews this may be measured by their compliance with the PRISMA statement. This review aims to evaluate the quality of reporting in published meta-analyses of diagnostic tests, using the PRISMA statement and establish whether there has been a measurable improvement over time.</p> <p>Methods</p> <p>Eight databases were searched for reviews published prior to 31^stDecember 2008. Studies were selected if they evaluated a diagnostic test, measured performance, searched two or more databases, stated the search terms and inclusion criteria, and used a statistical method to summarise a test's performance. Data were extracted on the review characteristics and items of the PRISMA statement. To measure the change in the quality of reporting over time, PRISMA items for two periods of equal duration were compared.</p> <p>Results</p> <p>Compliance with the PRISMA statement was generally poor: none of the reviews completely adhered to all 27 checklist items. Of the 236 meta-analyses included following selection: only 2(1%) reported the study protocol; 59(25%) reported the searches used; 76(32%) reported the results of a risk of bias assessment; and 82(35%) reported the abstract as a structured summary. Only 11 studies were published before 2000. Thus, the impact of QUOROM on the quality of reporting was not evaluated. However, the periods 2001-2004 and 2005-2008 (covering 93% of studies) were compared using relative risks (RR). There was an increase in the proportion of reviews reporting on five PRISMA items: eligibility criteria (RR 1.13, 95% CI 1.00 - 1.27); risk of bias across studies (methods) (RR 1.81, 95% CI 1.34 - 2.44); study selection results (RR 1.48, 95% CI 1.05 - 2.09); results of individual studies (RR 1.37, 95% CI 1.09 - 1.72); risk of bias across studies (results) (RR 1.65, 95% CI 1.20 - 2.25).</p> <p>Conclusion</p> <p>Although there has been an improvement in the quality of meta-analyses in diagnostic research, there are still many deficiencies in the reporting which future reviewers need to address if readers are to trust the validity of the reported findings.</p

The role of polyclonal intravenous immunoglobulin in treating HIV-infected children with severe bacterial infections: A retrospective cohort study

<p>Abstract</p> <p>Background</p> <p>Mortality among HIV-infected children in developing countries remains high after serious bacterial infections despite the use of antibiotics. Intravenous immunoglobulin (IVIG) has been used as an adjuvant therapy to treat these infections, but little data exists regarding its efficacy, and previous studies have focused on IVIG as a prophylactic agent. We examined the impact of IVIG as an adjuvant therapy in reducing mortality and length of hospital stay in HIV-infected children with serious bacterial infections.</p> <p>Methods</p> <p>This retrospective study focused on pediatric admissions at a large urban hospital between 2002 and 2006. Children between the ages of one month and nine years of age with laboratory confirmed HIV-status, serious bacterial infection, no prior exposure to IVIG, and a hospital length of stay of 5 days or more, were eligible for inclusion.</p> <p>Results</p> <p>A total of 140 children (median age 1.2 years) met inclusion criteria; lower respiratory tract infection was diagnosed in 94 (67%) of the children, while 74 (53%) had bacterial sepsis. Fifty-four (39%) children were receiving antiretroviral therapy and 39 (28%) were receiving tuberculosis treatment. Overall 73 (52%) were treated with IVIG, with the majority (74%) of children receiving a single dose. Thirteen (9%) died during their hospital admission. In crude analysis IVIG was significantly associated with increased mortality was (Odds Ratio (OR): 5.8; 95% Confidence Interval (CI): 1.2–27.1) and this association was weakened by adjustment for other predictors of mortality (OR 4.3, 95% CI 0.7–27.9, p = 0.123). IVIG use was also associated with longer hospital stays.</p> <p>Conclusion</p> <p>Administration of one to three doses of IVIG during the acute phase of illness does not appear to reduce mortality or the length of hospital stays in HIV-infected children with serious bacterial infections. However, the retrospective nature of this study makes confounding by indication difficult to control and further studies regarding the timing, dosing, and method of administration are required. Nonetheless the routine use of IVIG in resource-limited settings should be carefully considered given its high cost.</p