Applying the Risk of Bias Tool in a Systematic Review of Combination Long-Acting Beta-Agonists and Inhaled Corticosteroids for Persistent Asthma

Background: The Risk of Bias (RoB) tool is used to assess internal validity of randomized controlled trials (RCTs). Our objectives were to: 1) evaluate inter-rater agreement of the RoB tool; 2) determine the time to access supplemental study information; 3) compare the RoB tool with the Jadad scale and Schulz allocation concealment (AC); and 4) examine the relationship between RoB and effect estimates. Methods: We conducted a systematic review of long-acting beta agonists (LABA) combined with inhaled corticosteroids (ICS) for adults with persistent asthma. Two reviewers independently assessed 107 trials using RoB, Jadad, and AC. One reviewer searched for study protocols. We assessed inter-rater agreement using weighted Kappa (k) and the correlation between tools using Kendall’s Tau (t). Mean differences in effect sizes for RCTs with different RoB were calculated using inverse variance method and random effects model. Results: Trials had good Jadad scores (median 4, IQR 3-4); however, 85 % had unclear AC and 87 % high RoB. The factor that most influenced RoB was the potential inappropriate influence of study sponsors (95 % industry funded). Agreement on RoB domains was fair (k = 0.40) to almost perfect (k = 0.86), and moderate for overall RoB (k = 0.41). Median time to complete RoB assessments was 21 minutes (IQR 14-27) and 12 minutes (IQR 9-16) to search for protocols. Protocols were identified for 5/42 studies (12%); in 3 cases the assessment of selective outcome reporting changed. There was low correlation between overall RoB vs. Jadad (t =0.04

Helping editors, peer reviewers and authors improve the clarity, completeness and transparency of reporting health research

Inadequate reporting is problematic for several reasons. If authors do not provide sufficient details concerning the conduct of their study, readers are left with an incomplete picture of what was done. As such, they are not able to judge the merits of the results and interpret them. The EQUATOR Network is a new initiative aimed at improving the clarity and transparency of reporting health research

Developing a model of mental health self-care support for children and young people through an integrated evaluation of available types of provision involving systematic review, meta-analysis and case study

Background The mental health of children and young people (CYP) is a major UK public health concern. Recent policy reviews have identified that service provision for CYP with mental health needs is not as effective, responsive, accessible or child-centred as it could be. Following on from a previous National Institute for Health Research (NIHR) study into self-care support for CYP with long-term physical health needs, this study explored self-care support’s potential in CYP’s mental health. Objectives To identify and evaluate the types of mental health self-care support used by, and available to, CYP and their parents, and to establish how such support interfaces with statutory and non-statutory service provision. Design Two inter-related systematic literature reviews (an effectiveness review with meta-analysis and a perceptions review), together with a service mapping exercise and case study. Setting Global (systematic reviews); England and Wales (mapping exercise and case study). Participants (case study) Fifty-two individuals (17 CYP, 16 family members and 19 staff) were interviewed across six sites. Main outcome measures (meta-analysis) A measure of CYP’s mental health symptomatology. Data sources (literature reviews) MEDLINE, Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO, All Evidence-Based Medicine (EBM) Reviews, Applied Social Sciences Index and Abstracts (ASSIA) and Education Resources Information Center (ERIC). Review methods Titles and abstracts of papers were screened for relevance then grouped into studies. Two independent reviewers extracted data from studies meeting the inclusion criteria. A descriptive analysis and meta-analysis were conducted for the effectiveness review; descriptive analyses were conducted for the perceptions review. These analyses were integrated to elicit a mixed-methods review. Results Sixty-five of 71 included studies were meta-analysable. These 65 studies elicited 71 comparisons which, when meta-analysed, suggested that self-care support interventions were effective at 6-month [standardised mean difference (SMD) = −0.20; 95% confidence interval (CI) −0.28 to −0.11] and 12-month (SMD = −0.12; 95% CI −0.17 to −0.06) follow-ups. However, judged against Cochrane criteria, the studies were mostly low quality. Key elements of self-care support identified in the perceptions review were the acquisition of knowledge and skills, peer support and the relationship with the self-care support agent; CYP also had different perceptions from adults about what is important in self-care support. The mapping exercise identified 27 providers of 33 self-care support services. According to the case study data, effective self-care support services are predicated on flexibility; straightforward access; non-judgemental, welcoming organisations and staff; the provision of time and attention; opportunities to learn and practise skills relevant to self-care; and systems of peer support. Conclusions Mental health self-care support interventions for CYP are modestly effective in the short to medium term. Self-care support can be conceptualised as a process which has overlap with ‘recovery’. CYP and their families want choice and flexibility in the provision of such interventions and a continued relationship with services after the nominal therapy period. Those delivering self-care support need to have specific child-centred attributes. Future work Future work should focus on under-represented conditions (e.g. psychosis, eating disorders, self-harm); the role of technology, leadership and readiness in self-care support; satisfaction in self-care support; the conceptualisation of self-care support in CYP’s mental health; and efficacy and cost-effectiveness

Methods of Blinding in Reports of Randomized Controlled Trials Assessing Pharmacologic Treatments: A Systematic Review

BACKGROUND: Blinding is a cornerstone of therapeutic evaluation because lack of blinding can bias treatment effect estimates. An inventory of the blinding methods would help trialists conduct high-quality clinical trials and readers appraise the quality of results of published trials. We aimed to systematically classify and describe methods to establish and maintain blinding of patients and health care providers and methods to obtain blinding of outcome assessors in randomized controlled trials of pharmacologic treatments. METHODS AND FINDINGS: We undertook a systematic review of all reports of randomized controlled trials assessing pharmacologic treatments with blinding published in 2004 in high impact-factor journals from Medline and the Cochrane Methodology Register. We used a standardized data collection form to extract data. The blinding methods were classified according to whether they primarily (1) established blinding of patients or health care providers, (2) maintained the blinding of patients or health care providers, and (3) obtained blinding of assessors of the main outcomes. We identified 819 articles, with 472 (58%) describing the method of blinding. Methods to establish blinding of patients and/or health care providers concerned mainly treatments provided in identical form, specific methods to mask some characteristics of the treatments (e.g., added flavor or opaque coverage), or use of double dummy procedures or simulation of an injection. Methods to avoid unblinding of patients and/or health care providers involved use of active placebo, centralized assessment of side effects, patients informed only in part about the potential side effects of each treatment, centralized adapted dosage, or provision of sham results of complementary investigations. The methods reported for blinding outcome assessors mainly relied on a centralized assessment of complementary investigations, clinical examination (i.e., use of video, audiotape, or photography), or adjudication of clinical events. CONCLUSIONS: This review classifies blinding methods and provides a detailed description of methods that could help trialists overcome some barriers to blinding in clinical trials and readers interpret the quality of pharmalogic trials

A meta-analytic review of stand-alone interventions to improve body image

Objective Numerous stand-alone interventions to improve body image have been developed. The present review used meta-analysis to estimate the effectiveness of such interventions, and to identify the specific change techniques that lead to improvement in body image. Methods The inclusion criteria were that (a) the intervention was stand-alone (i.e., solely focused on improving body image), (b) a control group was used, (c) participants were randomly assigned to conditions, and (d) at least one pretest and one posttest measure of body image was taken. Effect sizes were meta-analysed and moderator analyses were conducted. A taxonomy of 48 change techniques used in interventions targeted at body image was developed; all interventions were coded using this taxonomy. Results The literature search identified 62 tests of interventions (N = 3,846). Interventions produced a small-to-medium improvement in body image (d+ = 0.38), a small-to-medium reduction in beauty ideal internalisation (d+ = -0.37), and a large reduction in social comparison tendencies (d+ = -0.72). However, the effect size for body image was inflated by bias both within and across studies, and was reliable but of small magnitude once corrections for bias were applied. Effect sizes for the other outcomes were no longer reliable once corrections for bias were applied. Several features of the sample, intervention, and methodology moderated intervention effects. Twelve change techniques were associated with improvements in body image, and three techniques were contra-indicated. Conclusions The findings show that interventions engender only small improvements in body image, and underline the need for large-scale, high-quality trials in this area. The review identifies effective techniques that could be deployed in future interventions

An empirical study using permutation-based resampling in meta-regression

<p>Abstract</p> <p>Background</p> <p>In meta-regression, as the number of trials in the analyses decreases, the risk of false positives or false negatives increases. This is partly due to the assumption of normality that may not hold in small samples. Creation of a distribution from the observed trials using permutation methods to calculate <it>P </it>values may allow for less spurious findings. Permutation has not been empirically tested in meta-regression. The objective of this study was to perform an empirical investigation to explore the differences in results for meta-analyses on a small number of trials using standard large sample approaches verses permutation-based methods for meta-regression.</p> <p>Methods</p> <p>We isolated a sample of randomized controlled clinical trials (RCTs) for interventions that have a small number of trials (herbal medicine trials). Trials were then grouped by herbal species and condition and assessed for methodological quality using the Jadad scale, and data were extracted for each outcome. Finally, we performed meta-analyses on the primary outcome of each group of trials and meta-regression for methodological quality subgroups within each meta-analysis. We used large sample methods and permutation methods in our meta-regression modeling. We then compared final models and final <it>P </it>values between methods.</p> <p>Results</p> <p>We collected 110 trials across 5 intervention/outcome pairings and 5 to 10 trials per covariate. When applying large sample methods and permutation-based methods in our backwards stepwise regression the covariates in the final models were identical in all cases. The <it>P </it>values for the covariates in the final model were larger in 78% (7/9) of the cases for permutation and identical for 22% (2/9) of the cases.</p> <p>Conclusions</p> <p>We present empirical evidence that permutation-based resampling may not change final models when using backwards stepwise regression, but may increase <it>P </it>values in meta-regression of multiple covariates for relatively small amount of trials.</p

Results from a blind and a non-blind randomised trial run in parallel: experience from the Estonian Postmenopausal Hormone Therapy (EPHT) Trial

<p>Abstract</p> <p>Background</p> <p>The Estonian Postmenopausal Hormone Therapy (EPHT) Trial assigned 4170 potential participants prior to recruitment to blind or non-blind hormone therapy (HT), with placebo or non-treatment the respective alternatives. Before having to decide on participation, women were told whether they had been randomised to the blind or non-blind trial. Eligible women who were still willing to join the trial were recruited. After recruitment participants in the non-blind trial (N = 1001) received open-label HT or no treatment, participants in the blind trial (N = 777) remained blinded until the end of the trial. The aim of this paper is to analyse the effect of blinding on internal and external validity of trial outcomes.</p> <p>Methods</p> <p>Effect of blinding was calculated as the hazard ratio of selected chronic diseases, total mortality and all outcomes. For analysing the effect of blinding on external validity, the hazard ratios from women recruited to the placebo arm and to the non-treatment arm were compared with those not recruited; for analysing the effect of blinding on internal validity, the hazard ratios from the blind trial were compared with those from the non-blind trial.</p> <p>Results</p> <p>The women recruited to the placebo arm had less cerebrovascular disease events (HR 0.43; 95% CI: 0.26-0.71) and all outcomes combined (HR 0.76; 95% CI: 0.63-0.91) than those who were not recruited. Among women recruited or not recruited to the non-treatment arm, no differences were observed for any of the outcomes studied.</p> <p>Among women recruited to the trial, the risk for coronary heart disease events (HR 0.77; 95% CI: 0.64-0.93), cerebrovascular disease events (HR 0.66; 95%CI: 0.47-0.92), and all outcomes combined (HR 0.82; 95% CI: 0.72-0.94) was smaller among participants in the blind trial than in the non-blind trial. There was no difference between the blind and the non-blind trial for total cancer (HR 0.95; 95% CI: 0.64-1.42), bone fractures (0.93; 95% CI: 0.74-1.16), and total mortality (HR 1.03; 95% CI: 0.53-1.98).</p> <p>Conclusions</p> <p>The results from blind and non-blind trials may differ, even if the target population is the same. Blinding may influence both internal and external validity. The effect of blinding may vary for different outcome events.</p> <p>Trial registration</p> <p>[<a href="http://www.controlled-trials.com/ISRCTN35338757">ISRCTN35338757</a>]</p

Reporting Methods of Blinding in Randomized Trials Assessing Nonpharmacological Treatments

BACKGROUND: Blinding is a cornerstone of treatment evaluation. Blinding is more difficult to obtain in trials assessing nonpharmacological treatment and frequently relies on “creative” (nonstandard) methods. The purpose of this study was to systematically describe the strategies used to obtain blinding in a sample of randomized controlled trials of nonpharmacological treatment. METHODS AND FINDINGS: We systematically searched in Medline and the Cochrane Methodology Register for randomized controlled trials (RCTs) assessing nonpharmacological treatment with blinding, published during 2004 in high-impact-factor journals. Data were extracted using a standardized extraction form. We identified 145 articles, with the method of blinding described in 123 of the reports. Methods of blinding of participants and/or health care providers and/or other caregivers concerned mainly use of sham procedures such as simulation of surgical procedures, similar attention-control interventions, or a placebo with a different mode of administration for rehabilitation or psychotherapy. Trials assessing devices reported various placebo interventions such as use of sham prosthesis, identical apparatus (e.g., identical but inactivated machine or use of activated machine with a barrier to block the treatment), or simulation of using a device. Blinding participants to the study hypothesis was also an important method of blinding. The methods reported for blinding outcome assessors relied mainly on centralized assessment of paraclinical examinations, clinical examinations (i.e., use of video, audiotape, photography), or adjudications of clinical events. CONCLUSIONS: This study classifies blinding methods and provides a detailed description of methods that could overcome some barriers of blinding in clinical trials assessing nonpharmacological treatment, and provides information for readers assessing the quality of results of such trials

Reporting on covariate adjustment in randomised controlled trials before and after revision of the 2001 CONSORT statement: a literature review

<p>Abstract</p> <p>Objectives</p> <p>To evaluate the use and reporting of adjusted analysis in randomised controlled trials (RCTs) and compare the quality of reporting before and after the revision of the CONSORT Statement in 2001.</p> <p>Design</p> <p>Comparison of two cross sectional samples of published articles.</p> <p>Data Sources</p> <p>Journal articles indexed on PubMed in December 2000 and December 2006.</p> <p>Study Selection</p> <p>Parallel group RCTs with a full publication carried out in humans and published in English</p> <p>Main outcome measures</p> <p>Proportion of articles reported adjusted analysis; use of adjusted analysis; the reason for adjustment; the method of adjustment and the reporting of adjusted analysis results in the main text and abstract.</p> <p>Results</p> <p>In both cohorts, 25% of studies reported adjusted analysis (84/355 in 2000 vs 113/422 in 2006). Compared with articles reporting only unadjusted analyses, articles that reported adjusted analyses were more likely to specify primary outcomes, involve multiple centers, perform stratified randomization, be published in general medical journals, and recruit larger sample sizes. In both years a minority of articles explained why and how covariates were selected for adjustment (20% to 30%). Almost all articles specified the statistical methods used for adjustment (99% in 2000 vs 100% in 2006) but only 5% and 10%, respectively, reported both adjusted and unadjusted results as recommended in the CONSORT guidelines.</p> <p>Conclusion</p> <p>There was no evidence of change in the reporting of adjusted analysis results five years after the revision of the CONSORT Statement and only a few articles adhered fully to the CONSORT recommendations.</p