Applying the Risk of Bias Tool in a Systematic Review of Combination Long-Acting Beta-Agonists and Inhaled Corticosteroids for Persistent Asthma

Background: The Risk of Bias (RoB) tool is used to assess internal validity of randomized controlled trials (RCTs). Our objectives were to: 1) evaluate inter-rater agreement of the RoB tool; 2) determine the time to access supplemental study information; 3) compare the RoB tool with the Jadad scale and Schulz allocation concealment (AC); and 4) examine the relationship between RoB and effect estimates. Methods: We conducted a systematic review of long-acting beta agonists (LABA) combined with inhaled corticosteroids (ICS) for adults with persistent asthma. Two reviewers independently assessed 107 trials using RoB, Jadad, and AC. One reviewer searched for study protocols. We assessed inter-rater agreement using weighted Kappa (k) and the correlation between tools using Kendall’s Tau (t). Mean differences in effect sizes for RCTs with different RoB were calculated using inverse variance method and random effects model. Results: Trials had good Jadad scores (median 4, IQR 3-4); however, 85 % had unclear AC and 87 % high RoB. The factor that most influenced RoB was the potential inappropriate influence of study sponsors (95 % industry funded). Agreement on RoB domains was fair (k = 0.40) to almost perfect (k = 0.86), and moderate for overall RoB (k = 0.41). Median time to complete RoB assessments was 21 minutes (IQR 14-27) and 12 minutes (IQR 9-16) to search for protocols. Protocols were identified for 5/42 studies (12%); in 3 cases the assessment of selective outcome reporting changed. There was low correlation between overall RoB vs. Jadad (t =0.04

From the Trenches: A Cross-Sectional Study Applying the GRADE Tool in Systematic Reviews of Healthcare Interventions

Background: GRADE was developed to address shortcomings of tools to rate the quality of a body of evidence. While much has been published about GRADE, there are few empirical and systematic evaluations. Objective: To assess GRADE for systematic reviews (SRs) in terms of inter-rater agreement and identify areas of uncertainty. Design: Cross-sectional, descriptive study. Methods: We applied GRADE to three SRs (n = 48, 66, and 75 studies, respectively) with 29 comparisons and 12 outcomes overall. Two reviewers graded evidence independently for outcomes deemed clinically important a priori. Inter-rater reliability was assessed using kappas for four main domains (risk of bias, consistency, directness, and precision) and overall quality of evidence. Results: For the first review, reliability was: k = 0.41 for risk of bias; 0.84 consistency; 0.18 precision; and 0.44 overall quality. Kappa could not be calculated for directness as one rater assessed all items as direct; assessors agreed in 41 % of cases. For the second review reliability was: 0.37 consistency and 0.19 precision. Kappa could not be assessed for other items; assessors agreed in 33 % of cases for risk of bias; 100 % directness; and 58 % overall quality. For the third review, reliability was: 0.06 risk of bias; 0.79 consistency; 0.21 precision; and 0.18 overall quality. Assessors agreed in 100 % of cases for directness. Precision created the most uncertainty due to difficulties in identifying ‘‘optimal’ ’ information size and ‘‘clinica

A protocol for a systematic review of knowledge translation strategies in the allied health professions

<p>Abstract</p> <p>Background</p> <p>Knowledge translation (KT) aims to close the gap between knowledge and practice in order to realize the benefits of research through (a) improved health outcomes, (b) more effective health services and products, and (c) strengthened healthcare systems. While there is some understanding of strategies to put research findings into practice within nursing and medicine, we have limited knowledge of KT strategies in allied health professions. Given the interprofessional nature of healthcare, a lack of guidance for supporting KT strategies in the allied health professions is concerning. Our objective in this study is to systematically review published research on KT strategies in five allied health disciplines.</p> <p>Methods</p> <p>A medical research librarian will develop and implement search strategies designed to identify evidence that is relevant to each question of the review. Two reviewers will perform study selection and quality assessment using standard forms. For study selection, data will be extracted by two reviewers. For quality assessment, data will be extracted by one reviewer and verified by a second. Disagreements will be resolved through discussion or third party adjudication. Within each profession, data will be grouped and analyzed by research design and KT strategies using the Effective Practice and Organisation of Care Review Group classification scheme. An overall synthesis across professions will be conducted.</p> <p>Significance</p> <p>A uniprofessional approach to KT does not represent the interprofessional context it targets. Our findings will provide the first systematic overview of KT strategies used in allied health professionals' clinical practice, as well as a foundation to inform future KT interventions in allied healthcare settings.</p

Risk factors for methamphetamine use in youth: a systematic review

<p>Abstract</p> <p>Background</p> <p>Methamphetamine (MA) is a potent stimulant that is readily available. Its effects are similar to cocaine, but the drug has a profile associated with increased acute and chronic toxicities. The objective of this systematic review was to identify and synthesize literature on risk factors that are associated with MA use among youth.</p> <p>More than 40 electronic databases, websites, and key journals/meeting abstracts were searched. We included studies that compared children and adolescents (≤ 18 years) who used MA to those who did not. One reviewer extracted the data and a second checked for completeness and accuracy. For discrete risk factors, odds ratios (OR) were calculated and when appropriate, a pooled OR with 95% confidence intervals (95% CI) was calculated. For continuous risk factors, mean difference and 95% CI were calculated and when appropriate, a weighted mean difference (WMD) and 95% CI was calculated. Results were presented separately by comparison group: low-risk (no previous drug abuse) and high-risk children (reported previous drug abuse or were recruited from a juvenile detention center).</p> <p>Results</p> <p>Twelve studies were included. Among low-risk youth, factors associated with MA use were: history of heroin/opiate use (OR = 29.3; 95% CI: 9.8–87.8), family history of drug use (OR = 4.7; 95% CI: 2.8–7.9), risky sexual behavior (OR = 2.79; 95% CI: 2.25, 3.46) and some psychiatric disorders. History of alcohol use and smoking were also significantly associated with MA use. Among high-risk youth, factors associated with MA use were: family history of crime (OR = 2.0; 95% CI: 1.2–3.3), family history of drug use (OR = 4.7; 95% CI: 2.8–7.9), family history of alcohol abuse (OR = 3.2; 95% CI: 1.8–5.6), and psychiatric treatment (OR = 6.8; 95% CI: 3.6–12.9). Female sex was also significantly associated with MA use.</p> <p>Conclusion</p> <p>Among low-risk youth, a history of engaging in a variety of risky behaviors was significantly associated with MA use. A history of a psychiatric disorder was a risk factor for MA for both low- and high-risk youth. Family environment was also associated with MA use. Many of the included studies were cross-sectional making it difficult to assess causation. Future research should utilize prospective study designs so that temporal relationships between risk factors and MA use can be established.</p

Quantifying bias in randomized controlled trials in child health: a meta-epidemiological study.

OBJECTIVE: To quantify bias related to specific methodological characteristics in child-relevant randomized controlled trials (RCTs). DESIGN: Meta-epidemiological study. DATA SOURCES: We identified systematic reviews containing a meta-analysis with 10-40 RCTs that were relevant to child health in the Cochrane Database of Systematic Reviews. DATA EXTRACTION: Two reviewers independently assessed RCTs using items in the Cochrane Risk of Bias tool and other study factors. We used meta-epidemiological methods to assess for differences in effect estimates between studies classified as high/unclear vs. low risk of bias. RESULTS: We included 287 RCTs from 17 meta-analyses. The proportion of studies at high/unclear risk of bias was: 79% sequence generation, 83% allocation concealment, 67% blinding of participants, 47% blinding of outcome assessment, 49% incomplete outcome data, 32% selective outcome reporting, 44% other sources of bias, 97% overall risk of bias, 56% funding, 35% baseline imbalance, 13% blocked randomization in unblinded trials, and 1% early stopping for benefit. We found no significant differences in effect estimates for studies that were high/unclear vs. low risk of bias for any of the risk of bias domains, overall risk of bias, or other study factors. CONCLUSIONS: We found no differences in effect estimates between studies based on risk of bias. A potential explanation is the number of trials included, in particular the small number of studies with low risk of bias. Until further evidence is available, reviewers should not exclude RCTs from systematic reviews and meta-analyses based solely on risk of bias particularly in the area of child health

Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews

Abstract Background Systematic reviews (SRs) are an important source of information about healthcare interventions. A key component of a well-conducted SR is a comprehensive literature search. There is limited evidence on the contribution of non-English reports, unpublished studies, and dissertations and their impact on results of meta-analyses. Methods Our sample included SRs from three Cochrane Review Groups: Acute Respiratory Infections (ARI), Infectious Diseases (ID), Developmental Psychosocial and Learning Problems (DPLP) (n = 129). Outcomes included: 1) proportion of reviews that searched for and included each study type; 2) proportion of relevant studies represented by each study type; and 3) impact on results and conclusions of the primary meta-analysis for each study type. Results Most SRs searched for non-English studies; however, these were included in only 12% of reviews and represented less than 5% of included studies. There was a change in results in only four reviews (total sample = 129); in two cases the change did not have an impact on the statistical or clinical significance of results. Most SRs searched for unpublished studies but the majority did not include these (only 6%) and they represented 2% of included studies. In most cases the impact of including unpublished studies was small; a substantial impact was observed in one case that relied solely on unpublished data. Few reviews in ARI (9%) and ID (3%) searched for dissertations compared to 65% in DPLP. Overall, dissertations were included in only nine SRs and represented less than 2% of included studies. In the majority of cases the change in results was negligible or small; in the case where a large change was noted, the estimate was more conservative without dissertations. Conclusions The majority of SRs searched for non-English and unpublished studies; however, these represented a small proportion of included studies and rarely impacted the results and conclusions of the review. Inclusion of these study types may have an impact in situations where there are few relevant studies, or where there are questionable vested interests in the published literature. We found substantial variation in whether SRs searched for dissertations; in most reviews that included dissertations, these had little impact on results

The contribution of databases to the results of systematic reviews: a cross-sectional study

Abstract Background One of the best sources for high quality information about healthcare interventions is a systematic review. A well-conducted systematic review includes a comprehensive literature search. There is limited empiric evidence to guide the extent of searching, in particular the number of electronic databases that should be searched. We conducted a cross-sectional quantitative analysis to examine the potential impact of selective database searching on results of meta-analyses. Methods Our sample included systematic reviews (SRs) with at least one meta-analysis from three Cochrane Review Groups: Acute Respiratory Infections (ARI), Infectious Diseases (ID), Developmental Psychosocial and Learning Problems (DPLP) (n = 129). Outcomes included: 1) proportion of relevant studies indexed in each of 10 databases; and 2) changes in results and statistical significance of primary meta-analysis for studies identified in Medline only and in Medline plus each of the other databases. Results Due to variation across topics, we present results by group (ARI n = 57, ID n = 38, DPLP n = 34). For ARI, identification of relevant studies was highest for Medline (85 %) and Embase (80 %). Restricting meta-analyses to trials that appeared in Medline + Embase yielded fewest changes in statistical significance: 53/55 meta-analyses showed no change. Point estimates changed in 12 cases; in 7 the change was less than 20 %. For ID, yield was highest for Medline (92 %), Embase (81 %), and BIOSIS (67 %). Restricting meta-analyses to trials that appeared in Medline + BIOSIS yielded fewest changes with 1 meta-analysis changing in statistical significance. Point estimates changed in 8 of 31 meta-analyses; change less than 20 % in all cases. For DPLP, identification of relevant studies was highest for Medline (75 %) and Embase (62 %). Restricting meta-analyses to trials that appeared in Medline + PsycINFO resulted in only one change in significance. Point estimates changed for 13 of 33 meta-analyses; less than 20 % in 9 cases. Conclusions Majority of relevant studies can be found within a limited number of databases. Results of meta-analyses based on the majority of studies did not differ in most cases. There were very few cases of changes in statistical significance. Effect estimates changed in a minority of meta-analyses but in most the change was small. Results did not change in a systematic manner (i.e., regularly over- or underestimating treatment effects), suggesting that selective searching may not introduce bias in terms of effect estimates