A proposed framework to guide evidence synthesis practice for meta-analysis with zero-events studies

ObjectiveIn evidence synthesis practice, researchers often face the problem of how to deal with zero-events. Inappropriately dealing with zero-events studies may lead to research waste and mislead healthcare practice. We propose a framework to guide researchers to better deal with zero-events in meta-analysis. Study Design and SettingWe used two dimensions, one with respect to the total events count across all studies in the comparative arms in a meta-analysis, and a second with respect to whether included studies have single or both arms with zero-events, to establish the framework for the classification of meta-analysis with zero-events studies. A dataset from Cochrane systematic reviews was used to evaluate the classification. ResultsThe proposed framework classifies meta-analysis with zero-events studies into six subtypes. The classification matched well to the large real-world dataset. The applicability of existing methods for zero-events were then presented under each meta-analysis subtype based on this framework, with a 5-step principle to help researchers in evidence synthesis practice. ConclusionsThe proposed framework should be considered by researchers when making decisions on the selection of the synthesis methods in a meta-analysis. It also provides a reasonable basis for the development of methodological guidelines to deal with zero-events in meta-analysis

Some improvements are apparent in identifying adverse effects in systematic reviews from 1994 to 2011

An increasing amount of research and guidelines has been published on search methodology and the reporting of search strategies in systematic reviews. This research assessed whether this has lead to any improvements in the reporting and quality of searching in systematic reviews of adverse effects

Foreign body in children’s airways

OBJECTIVE: To determine the clinical characteristics and the results of bronchoscopic treatment of children due to foreign body aspiration in a university hospital. METHOD: Time series of children who underwent bronchoscopies for foreign bodies aspirated into the airway between March 1993 and July 2002. Each patient was analyzed for age, sex, initial clinical diagnosis, nature and location of the foreign body, duration of symptoms between aspiration and bronchoscopy, radiological findings, results of bronchoscopic removal, complications of bronchoscopy and presence of foreign bodies in the airways. RESULTS: Thirty-four children, 20 (59%) boys, ages ranging from nine months to nine years (median = 23 months). In 32 (94%) children the foreign body was removed by rigid bronchoscope, and two resulted in thoracotomy. Foreign bodies were more frequent in children under three years of age (66%). A clinical history of foreign body inhalation was obtained in 27 (80%) cases. Most of the foreign bodies removed were organic (65%) and more frequently found in the right bronchial tree (59%). Foreign bodies were removed within 24 hours in 18 (53%) cases. The most frequent radiographic findings were: unilateral air trapping, atelectasis and radiopac foreign body. Major bronchoscopy complications occurred in seven children (22%), and there were no deaths. CONCLUSIONS: More attention is necessary to the respiratory symptoms of aspirations, mainly in boys at early ages, with clinical history and compatible radiological findings. Most foreign bodies removed were of organic nature. In this case series, therapeutic rigid bronchoscopy was effective with few complications

Validity of data extraction in evidence synthesis practice of adverse events: reproducibility study

OBJECTIVES: To investigate the validity of data extraction in systematic reviews of adverse events, the effect of data extraction errors on the results, and to develop a classification framework for data extraction errors to support further methodological research. DESIGN: Reproducibility study. DATA SOURCES: PubMed was searched for eligible systematic reviews published between 1 January 2015 and 1 January 2020. Metadata from the randomised controlled trials were extracted from the systematic reviews by four authors. The original data sources (eg, full text and ClinicalTrials.gov) were then referred to by the same authors to reproduce the data used in these meta-analyses. ELIGIBILITY CRITERIA FOR SELECTING STUDIES: Systematic reviews were included when based on randomised controlled trials for healthcare interventions that reported safety as the exclusive outcome, with at least one pair meta-analysis that included five or more randomised controlled trials and with a 2×2 table of data for event counts and sample sizes in intervention and control arms available for each trial in the meta-analysis. MAIN OUTCOME MEASURES: The primary outcome was data extraction errors summarised at three levels: study level, meta-analysis level, and systematic review level. The potential effect of such errors on the results was further investigated. RESULTS: 201 systematic reviews and 829 pairwise meta-analyses involving 10 386 randomised controlled trials were included. Data extraction could not be reproduced in 1762 (17.0%) of 10 386 trials. In 554 (66.8%) of 829 meta-analyses, at least one randomised controlled trial had data extraction errors; 171 (85.1%) of 201 systematic reviews had at least one meta-analysis with data extraction errors. The most common types of data extraction errors were numerical errors (49.2%, 867/1762) and ambiguous errors (29.9%, 526/1762), mainly caused by ambiguous definitions of the outcomes. These categories were followed by three others: zero assumption errors, misidentification, and mismatching errors. The impact of these errors were analysed on 288 meta-analyses. Data extraction errors led to 10 (3.5%) of 288 meta-analyses changing the direction of the effect and 19 (6.6%) of 288 meta-analyses changing the significance of the P value. Meta-analyses that had two or more different types of errors were more susceptible to these changes than those with only one type of error (for moderate changes, 11 (28.2%) of 39 v 26 (10.4%) 249, P=0.002; for large changes, 5 (12.8%) of 39 v 8 (3.2%) of 249, P=0.01). CONCLUSION: Systematic reviews of adverse events potentially have serious issues in terms of the reproducibility of the data extraction, and these errors can mislead the conclusions. Implementation guidelines are urgently required to help authors of future systematic reviews improve the validity of data extraction

PRISMA harms checklist: improving harms reporting in systematic reviews

Introduction: For any health intervention, accurate knowledge of both benefits and harms is needed. Systematic reviews often compound poor reporting of harms in primary studies by failing to report harms or doing so inadequately. While the PRISMA statement (Preferred Reporting Items for Systematic reviews and Meta-Analyses) helps systematic review authors ensure complete and transparent reporting, it is focused mainly on efficacy. Thus, a PRISMA harms checklist has been developed to improve harms reporting in systematic reviews, promoting a more balanced assessment of benefits and harms. Methods: A development strategy, endorsed by the EQUATOR Network and existing reporting guidelines (including the PRISMA statement, PRISMA for abstracts, and PRISMA for protocols), was used. After the development of a draft checklist of items, a modified Delphi process was initiated. The Delphi consisted of three rounds of electronic feedback followed by an in-person meeting. Results: The PRISMA harms checklist contains four essential reporting elements to be added to the original PRISMA statement to improve harms reporting in reviews. These are reported in the title (“Specifically mention ‘harms’ or other related terms, or the harm of interest in the review”), synthesis of results (“Specify how zero events were handled, if relevant”), study characteristics (“Define each harm addressed, how it was ascertained (eg, patient report, active search), and over what time period”), and synthesis of results (“Describe any assessment of possible causality”). Additional guidance regarding existing PRISMA items was developed to demonstrate relevance when synthesising information about harms. Conclusion: The PRISMA harms checklist identifies a minimal set of items to be reported when reviewing adverse events. This guideline extension is intended to improve harms reporting in systematic reviews, whether harms are a primary or secondary outcome

Utilization of the evidence from studies with no events in meta-analyses of adverse events: an empirical investigation

Abstract Backgrounds Zero-events studies frequently occur in systematic reviews of adverse events, which consist of an important source of evidence. We aimed to examine how evidence of zero-events studies was utilized in the meta-analyses of systematic reviews of adverse events. Methods We conducted a survey of systematic reviews published in two periods: January 1, 2015, to January 1, 2020, and January 1, 2008, to April 25, 2011. Databases were searched for systematic reviews that conducted at least one meta-analysis of any healthcare intervention and used adverse events as the exclusive outcome. An adverse event was defined as any untoward medical occurrence in a patient or subject in healthcare practice. We summarized the frequency of occurrence of zero-events studies in eligible systematic reviews and how these studies were dealt with in the meta-analyses of these systematic reviews. Results We included 640 eligible systematic reviews. There were 406 (63.45%) systematic reviews involving zero-events studies in their meta-analyses, among which 389 (95.11%) involved single-arm-zero-events studies and 223 (54.93%) involved double-arm-zero-events studies. The majority (98.71%) of these systematic reviews incorporated single-arm-zero-events studies into the meta-analyses. On the other hand, the majority (76.23%) of them excluded double-arm-zero-events studies from the meta-analyses, of which the majority (87.06%) did not discuss the potential impact of excluding such studies. Systematic reviews published at present (2015-2020) tended to incorporate zero-events studies in meta-analyses than those published in the past (2008-2011), but the difference was not significant (proportion difference=−0.09, 95% CI −0.21 to 0.03, p = 0.12). Conclusion Systematic review authors routinely treated studies with zero-events in both arms as “non-informative” carriers and excluded them from their reviews. Whether studies with no events are “informative” or not largely depends on the methods and assumptions applied, thus sensitivity analyses using different methods should be considered in future meta-analyses