Applying the ROBINS-I tool to natural experiments: an example from public health

Background: A new tool to assess Risk of Bias In Non-randomised Studies of Interventions (ROBINS-I) was published in Autumn 2016. ROBINS-I uses the Cochrane-approved risk of bias (RoB) approach and focusses on internal validity. As such, ROBINS-I represents an important development for those conducting systematic reviews which include non-randomised studies (NRS), including public health researchers. We aimed to establish the applicability of ROBINS-I using a group of NRS which have evaluated non-clinical public health natural experiments. Methods: Five researchers, all experienced in critical appraisal of non-randomised studies, used ROBINS-I to independently assess risk of bias in five studies which had assessed the health impacts of a domestic energy efficiency intervention. ROBINS-I assessments for each study were entered into a database and checked for consensus across the group. Group discussions were used to identify reasons underpinning lack of consensus for specific questions and bias domains. Results: ROBINS-I helped to systematically articulate sources of bias in NRS. However, the lack of consensus in assessments for all seven bias domains raised questions about ROBINS-I’s reliability and applicability for natural experiment studies. The two RoB domains with least consensus were selection (Domain 2) and performance (Domain 4). Underlying the lack of consensus were difficulties in applying an intention to treat or per protocol effect of interest to the studies. This was linked to difficulties in determining whether the intervention status was classified retrospectively at follow-up, i.e. post hoc. The overall risk of bias ranged from moderate to critical; this was most closely linked to the assessment of confounders. Conclusion: The ROBINS-I tool is a conceptually rigorous tool which focusses on risk of bias due to the counterfactual. Difficulties in applying ROBINS-I may be due to poor design and reporting of evaluations of natural experiments. While the quality of reporting may improve in the future, improved guidance on applying ROBINS-I is needed to enable existing evidence from natural experiments to be assessed appropriately and consistently. We hope future refinements to ROBINS-I will address some of the issues raised here to allow wider use of the tool

Applying the Risk of Bias Tool in a Systematic Review of Combination Long-Acting Beta-Agonists and Inhaled Corticosteroids for Persistent Asthma

Background: The Risk of Bias (RoB) tool is used to assess internal validity of randomized controlled trials (RCTs). Our objectives were to: 1) evaluate inter-rater agreement of the RoB tool; 2) determine the time to access supplemental study information; 3) compare the RoB tool with the Jadad scale and Schulz allocation concealment (AC); and 4) examine the relationship between RoB and effect estimates. Methods: We conducted a systematic review of long-acting beta agonists (LABA) combined with inhaled corticosteroids (ICS) for adults with persistent asthma. Two reviewers independently assessed 107 trials using RoB, Jadad, and AC. One reviewer searched for study protocols. We assessed inter-rater agreement using weighted Kappa (k) and the correlation between tools using Kendall’s Tau (t). Mean differences in effect sizes for RCTs with different RoB were calculated using inverse variance method and random effects model. Results: Trials had good Jadad scores (median 4, IQR 3-4); however, 85 % had unclear AC and 87 % high RoB. The factor that most influenced RoB was the potential inappropriate influence of study sponsors (95 % industry funded). Agreement on RoB domains was fair (k = 0.40) to almost perfect (k = 0.86), and moderate for overall RoB (k = 0.41). Median time to complete RoB assessments was 21 minutes (IQR 14-27) and 12 minutes (IQR 9-16) to search for protocols. Protocols were identified for 5/42 studies (12%); in 3 cases the assessment of selective outcome reporting changed. There was low correlation between overall RoB vs. Jadad (t =0.04

Impact of Age and HIV Status on Immune Activation, Senescence and Apoptosis

Introduction: Residual immune dysfunctions, resembling those that occur during normal aging, may persist even in well-treated people with HIV (PWH), and accelerated aging has been proposed. We aimed to determine if HIV infection is an independent risk factor for T-cell immune dysfunctions including increased immune activation, senescence and apoptosis. Moreover, in PWH we aimed to identify the associations between age and immune activation, senescence and apoptosis. Materials and Methods: We included 780 PWH with suppressed viral replication (<50 copies/mL) and absence of hepatitis B and hepatitis C co-infection and 65 uninfected controls from the Copenhagen Co-morbidity in HIV Infection (COCOMO) Study. Flow cytometry was used to determine T-cell activation (CD38+HLA-DR+), senescence (CD28-CD57+), and apoptosis (CD28-CD95+). T-cell subsets are reported as proportions of CD4+ and CD8+ T-cells. We defined an elevated proportion of a given T-cell subset as above the 75th percentile. Regression models were used to determine the association between HIV status and T-cell subset and in PWH to determine the association between age or HIV-specific risk factors and T-cell subsets. Furthermore, an interaction between HIV status and age on T-cell subsets was investigated with an interaction term in models including both PWH and controls. Models were adjusted for age, sex, BMI, and smoking status. Results: In adjusted models a positive HIV status was associated with elevated proportions of CD8+ activated (p = 0.009), CD4+ senescent (p = 0.004), CD4+ apoptotic (p = 0.002), and CD8+ apoptotic (p = 0.003) T-cells. In PWH a 10-year increase in age was associated with higher proportions of CD4+ and CD8+ senescent (p = 0.001 and p < 0.001) and CD4+ and CD8+ apoptotic T-cells (p < 0.001 and p < 0.001). However, no interaction between HIV status and age was found. Furthermore, in PWH a CD4+/CD8+ ratio < 1 was associated with elevated proportions of T-cell activation, senescence, and apoptosis. Discussion: We found evidence of residual T-cell immune dysfunction in well-treated PWH without HBV or HCV co-infection, and age was associated with T-cell senescence and apoptosis. Our data supports that HIV infection has similar effects as aging on T-cell subsets. However, since no interaction between HIV status and age was found on these parameters, we found no evidence to support accelerated immunological aging in PWH

From the Trenches: A Cross-Sectional Study Applying the GRADE Tool in Systematic Reviews of Healthcare Interventions

Background: GRADE was developed to address shortcomings of tools to rate the quality of a body of evidence. While much has been published about GRADE, there are few empirical and systematic evaluations. Objective: To assess GRADE for systematic reviews (SRs) in terms of inter-rater agreement and identify areas of uncertainty. Design: Cross-sectional, descriptive study. Methods: We applied GRADE to three SRs (n = 48, 66, and 75 studies, respectively) with 29 comparisons and 12 outcomes overall. Two reviewers graded evidence independently for outcomes deemed clinically important a priori. Inter-rater reliability was assessed using kappas for four main domains (risk of bias, consistency, directness, and precision) and overall quality of evidence. Results: For the first review, reliability was: k = 0.41 for risk of bias; 0.84 consistency; 0.18 precision; and 0.44 overall quality. Kappa could not be calculated for directness as one rater assessed all items as direct; assessors agreed in 41 % of cases. For the second review reliability was: 0.37 consistency and 0.19 precision. Kappa could not be assessed for other items; assessors agreed in 33 % of cases for risk of bias; 100 % directness; and 58 % overall quality. For the third review, reliability was: 0.06 risk of bias; 0.79 consistency; 0.21 precision; and 0.18 overall quality. Assessors agreed in 100 % of cases for directness. Precision created the most uncertainty due to difficulties in identifying ‘‘optimal’ ’ information size and ‘‘clinica

Effectiveness of early intervention programs for parents of preterm infants: a meta-review of systematic reviews

Background: Various intervention programs exist for parents of preterm babies and some systematic reviews (SRs) have synthesised the evidence of their effectiveness. These reviews are, however, limited to specific interventions, components, or outcomes, and a comprehensive evidence base is lacking. The aim of this meta-review was to appraise and meta-synthesise the evidence from existing SRs to provide a comprehensive evidence base on the effectiveness of interventions for parents of preterm infants on parental and infant outcomes. Methods: We conducted a comprehensive search of the following databases to identify relevant SRs: Cochrane library, Web of science, EMBASE, CINAHL, British Nursing Index, PsycINFO, Medline, ScienceDirect, Scopus, IBSS, DOAJ, ERIC, EPPI-Centre, PROSPERO, WHO Library. Additional searches were conducted using authors’ institutional libraries, Google Scholar, and the reference lists of identified reviews. Identified articles were screened in two stages against an inclusion criteria with titles and abstracts screened first followed by full-text screening. Selected SRs were appraised using the AMSTAR tool. Extracted data using a predesigned tool were synthesised narratively examining the direction of impact on outcomes. Results: We found 11 SRs eligible for inclusion that synthesised a total of 343 quantitative primary studies. The average quality of the SRs was ‘medium’. Thirty four interventions were reported across the SRs with considerable heterogeneity in the structural framework and the targeted outcomes that included maternal-infant dyadic, maternal/parental, and infant outcomes. Among all interventions, Kangaroo Care (KC) showed the most frequent positive impact across outcomes (n = 19) followed by Mother Infant Transaction Program (MITP) (n = 14). Other interventions with most consistent positive impact on infant outcomes were Modified-Mother Infant Transaction Program (M-MITP) (n = 6), Infant Health and Development Program (IHDP) (n = 5) and Creating Opportunities for Parent Empowerment (COPE) (n = 5). Overall, interventions with both home and facility based components showed the most frequent positive impact across outcomes. Conclusions: Neonatal care policy and planning for preterm babies should consider the implementation of interventions with most positive impact on outcomes. The heterogeneity in interventions and outcomes calls for the development and implementation of an integrated program for parents of preterm infants with a clearly defined global set of parental and infant outcomes

Racemic epinephrine compared to salbutamol in hospitalized young children with bronchiolitis; a randomized controlled clinical trial [ISRCTN46561076]

BACKGROUND: Bronchiolitis is the most common cause of lower respiratory tract illness in infancy, and hospital admission rates appear to be increasing in Canada and the United States. Inhaled beta agonists offer only modest short-term improvement. Trials of racemic epinephrine have shown conflicting results. We sought to determine if administration of racemic epinephrine during hospital stay for bronchiolitis improved respiratory distress, was safe, and shortened length of stay. METHODS: The study was a randomized, double-blind controlled trial of aerosolized racemic epinephrine compared to salbutamol every one to 4 hours in previously well children aged 6 weeks to ≤ 2 years of age hospitalized with bronchiolitis. The primary outcome was symptom improvement as measured by the Respiratory Distress Assessment Instrument (RDAI); secondary outcomes were length of stay in hospital, adverse events, and report of symptoms by structured parental telephone interview one week after discharge. RESULTS: 62 children with a mean age of 6.4 months were enrolled; 80% of children had Respiratory Syncytial Virus (RSV). Racemic epinephrine resulted in significant improvement in wheezing and the total RDAI score on day 2 and over the entire stay (p < 0.05). The mean LOS in the epinephrine arm was 2.6 days (95% CI 2, 3.2) v. 3.4 days in those in the salbutamol group (95% CI 2.6, 4.2) (p > 0.05). Adverse events were not significantly different in the two arms. At one week post-discharge, over half of parents reported that their child still had a respiratory symptom and 40% had less than normal feeding. CONCLUSION: Racemic epinephrine relieves respiratory distress in hospitalized infants with bronchiolitis and is safe but does not abbreviate hospital stay. Morbidity associated with bronchiolitis as identified by parents persists for at least one week after hospital discharge in most infants

A descriptive analysis of oral health systematic reviews published 1991-2012: cross sectional study

OBJECTIVES: To identify all systematic reviews (SRs) published in the domain of oral health research and describe them in terms of their epidemiological and descriptive characteristics. DESIGN: Cross sectional, descriptive study. METHODS: An electronic search of seven databases was performed from inception through May 2012; bibliographies of relevant publications were also reviewed. Studies were considered for inclusion if they were oral health SRs defined as therapeutic or non-therapeutic investigations that studied a topic or an intervention related to dental, oral or craniofacial diseases/disorders. Data were extracted from all the SRs based on a number of epidemiological and descriptive characteristics. Data were analysed descriptively for all the SRs, within each of the nine dental specialities, and for Cochrane and non-Cochrane SRs separately. RESULTS: 1,188 oral health (126 Cochrane and 1062 non-Cochrane) SRs published from 1991 through May 2012 were identified, encompassing the nine dental specialties. Over half (n = 676; 56.9%) of the SRs were published in specialty oral health journals, with almost all (n = 1,178; 99.2%) of the SRs published in English and almost none of the non-Cochrane SRs (n = 11; 0.9%) consisting of updates of previously published SRs. 75.3% of the SRs were categorized as therapeutic, with 64.5% examining non-drug interventions, while approximately half (n = 150/294; 51%) of the non-therapeutic SRs were classified as epidemiological SRs. The SRs included a median of 15 studies, with a meta-analysis conducted in 43.6%, in which a median of 9 studies/1 randomized trial were included in the largest meta-analysis conducted. Funding was received for 25.1% of the SRs, including nearly three-quarters (n = 96; 76.2%) of the Cochrane SRs. CONCLUSION: Epidemiological and descriptive characteristics of the 1,188 oral health SRs varied across the nine dental specialties and by SR category (Cochrane vs. non-Cochrane). There is a clear need for more updates of SRs in all the dental specialties.Humam Saltaji, Greta G. Cummings, Susan Armijo-Olivo, Michael P. Major, Maryam Amin, Paul W. Major, Lisa Hartling, Carlos Flores-Mi

Fraction of all hospital admissions and deaths attributable to malnutrition among children in rural Kenya235

Background: Malnutrition is common in the developing world and associated with disease and mortality. Because malnutrition frequently occurs among children in the community as well as those with acute illness, and because anthropometric indicators of nutritional status are continuous variables that preclude a single definition of malnutrition, malnutrition-attributable fractions of admissions and deaths cannot be calculated by simply enumerating individual children

Epidemiology, Quality and Reporting Characteristics of Systematic Reviews of Traditional Chinese Medicine Interventions Published in Chinese Journals

Systematic reviews (SRs) of TCM have become increasingly popular in China and have been published in large numbers. This review provides the first examination of epidemiological characteristics of these SRs as well as compliance with the PRISMA and AMSTAR guidelines.To examine epidemiological and reporting characteristics as well as methodological quality of SRs of TCM published in Chinese journals.Four Chinese databases were searched (CBM, CSJD, CJFD and Wanfang Database) for SRs of TCM, from inception through Dec 2009. Data were extracted into Excel spreadsheets. The PRISMA and AMSTAR checklists were used to assess reporting characteristics and methodological quality, respectively.A total of 369 SRs were identified, most (97.6%) of which used the terms systematic review or meta-analysis in the title. None of the reviews had been updated. Half (49.8%) were written by clinicians and nearly half (47.7%) were reported in specialty journals. The impact factors of 45.8% of the journals published in were zero. The most commonly treated conditions were diseases of the circulatory and digestive disease. Funding sources were not reported for any reviews. Most (68.8%) reported information about quality assessment, while less than half (43.6%) reported assessing for publication bias. Statistical mistakes appeared in one-third (29.3%) of reviews and most (91.9%) did not report on conflict of interest.While many SRs of TCM interventions have been published in Chinese journals, the quality of these reviews is troubling. As a potential key source of information for clinicians and researchers, not only were many of these reviews incomplete, some contained mistakes or were misleading. Focusing on improving the quality of SRs of TCM, rather than continuing to publish them in great quantity, is urgently needed in order to increase the value of these studies