The judgement of biases included in the category “other bias” in Cochrane systematic reviews of interventions: a systematic survey

Abstract Background Clinical decisions are made based on Cochrane reviews, but the implementation of results of evidence syntheses such as Cochrane reviews is problematic if the evidence is not prepared consistently. All systematic reviews should assess the risk of bias (RoB) in included studies, and in Cochrane reviews, this is done by using Cochrane RoB tool. However, the tool is not necessarily applied according to the instructions. In this study, we aimed to determine the types of bias and their corresponding judgements noted in the ‘other bias’ domain of Cochrane RoB tool. Methods We analyzed Cochrane reviews that included randomized controlled trials (RCTs) and extracted data regarding ‘other bias’ from the RoB table and accompanying support for the judgment. We categorized different types of other bias. Results We analyzed 768 Cochrane reviews that included 11,369 RCTs. There were 602 (78%) Cochrane reviews that had ‘other bias’ domain in the RoB tool, and they included a total of 7811 RCTs. In the RoB table of 337 Cochrane reviews for at least one of the included trials it was indicated that no other bias was found and supporting explanations were inconsistently judged as low, unclear or high RoB. In the 524 Cochrane reviews that described various sources of other bias, there were 5762 individual types of explanations which we categorized into 31 groups. The judgments of the same supporting explanations were highly inconsistent. We found numerous other inconsistencies in reporting of sources of other bias in Cochrane reviews. Conclusion Cochrane authors mention a wide range of sources of other bias in the RoB tool and they inconsistently judge the same supporting explanations. Inconsistency in appraising risk of other bias hinders reliability and comparability of Cochrane systematic reviews. Discrepant and erroneous judgments of bias in evidence synthesis may hinder implementation of evidence in routine clinical practice and reduce confidence in otherwise trustworthy sources of information. These results can help authors of Cochrane and non-Cochrane reviews to gain insight into various sources of other bias that can be found in trials, and also to help them avoid mistakes that were recognized in published Cochrane reviews

Assessments of attrition bias in Cochrane systematic reviews are highly inconsistent and thus hindering trial comparability

Abstract Background An important part of the systematic review methodology is appraisal of the risk of bias in included studies. Cochrane systematic reviews are considered golden standard regarding systematic review methodology, but Cochrane’s instructions for assessing risk of attrition bias are vague, which may lead to inconsistencies in authors’ assessments. The aim of this study was to analyze consistency of judgments and support for judgments of attrition bias in Cochrane reviews of interventions published in the Cochrane Database of Systematic Reviews (CDSR). Methods We analyzed Cochrane reviews published from July 2015 to June 2016 in the CDSR. We extracted data on number of included trials, judgment of attrition risk of bias for each included trial (low, unclear or high) and accompanying support for the judgment (supporting explanation). We also assessed how many Cochrane reviews had different judgments for the same supporting explanations. Results In the main analysis we included 10,292 judgments and supporting explanations for attrition bias from 729 Cochrane reviews. We categorized supporting explanations for those judgments into four categories and we found that most of the supporting explanations were unclear. Numerical indicators for percent of attrition, as well as statistics related to attrition were judged very differently. One third of Cochrane review authors had more than one category of supporting explanation; some had up to four different categories. Inconsistencies were found even with the number of judgments, names of risk of bias domains and different judgments for the same supporting explanations in the same Cochrane review. Conclusion We found very high inconsistency in methods of appraising risk of attrition bias in recent Cochrane reviews. Systematic review authors need clear guidance about different categories they should assess and judgments for those explanations. Clear instructions about appraising risk of attrition bias will improve reliability of the Cochrane’s risk of bias tool, help authors in making decisions about risk of bias and help in making reliable decisions in healthcare

AMSTAR 2 is only partially applicable to systematic reviews of non-intervention studies : a meta-research study

Objectives: A measurement tool to assess systematic reviews 2 (AMSTAR 2) was originally developed for systematic reviews (SRs) of health-care interventions. The aim of this study was to assess the applicability of AMSTAR 2 to SRs of non-intervention studies. Study design and setting: This was a meta-research study. We used 20 SRs for each of the following four types of SRs: Diagnostic Test Accuracy reviews, Etiology and/or Risk reviews, Prevalence and/or Incidence reviews, and Prognostic reviews (80 in total). Three authors applied AMSTAR 2 independently to each included SRs. Then, the authors assessed the applicability of each item to that SR type and any SR type. Results: Researchers unanimously indicated that 7 of 16 AMSTAR 2 items were applicable for all four specific SR types and any SR type (items 2, 5, 6, 7, 10, 14 and 16), but 8 of 16 items for any SR type. These items could cover generic SR methods that do not depend on a specific SR type. Conclusion: AMSTAR 2 is only partially applicable for non-intervention SRs. There is a need to adapt/extend AMSTAR 2 for SRs of non-intervention studies. Our study can help to further define generic methodological aspects shared across SR types and methodological expectations for non-intervention SRs

A systematic review of meta-research studies finds substantial methodological heterogeneity in citation analyses to monitor evidence-based research

Objectives: This systematic review aimed to identify the characteristics and application of citation analyses in evaluating the justification, design, and placement of the research results of clinical health studies in the context of earlier similar studies. Study Design and Setting: We searched MEDLINE (Ovid), Embase (Ovid), and the Cochrane Methodology Register for meta-research studies. We included meta-research studies assessing whether researchers used earlier similar studies and/or systematic reviews of such studies to inform the justification or design of a new study, whether researchers used systematic reviews to inform the interpretation of new results, and meta-research studies assessing whether redundant studies were published within a specific area. The results are presented as a narrative synthesis. Results: A total of 27 studies were included. How authors of citation analyses define their outcomes appears rather arbitrary, as does how the reference of a landmark review or adherence to reporting guidelines was expected to contribute to the initiation, justification, design, or contextualization of relevant clinical trials. Conclusion: Continued and improved efforts to promote evidence-based research are needed, including clearly defined and justified outcomes in meta-research studies to monitor the implementation of an evidence-based approach

Clinical, laboratory and radiological characteristics and outcomes of novel coronavirus (SARS-CoV-2) infection in humans: A systematic review and series of meta-analyses.

New evidence on the COVID-19 pandemic is being published daily. Ongoing high-quality assessment of this literature is therefore needed to enable clinical practice to be evidence-based. This review builds on a previous scoping review and aimed to identify associations between disease severity and various clinical, laboratory and radiological characteristics. We searched MEDLINE, CENTRAL, EMBASE, Scopus and LILACS for studies published between January 1, 2019 and March 22, 2020. Clinical studies including ≥10 patients with confirmed COVID-19 of any study design were eligible. Two investigators independently extracted data and assessed risk of bias. A quality effects model was used for the meta-analyses. Subgroup analysis and meta-regression identified sources of heterogeneity. For hospitalized patients, studies were ordered by overall disease severity of each population and this order was used as the modifier variable in meta-regression. Overall, 86 studies (n = 91,621) contributed data to the meta-analyses. Severe disease was strongly associated with fever, cough, dyspnea, pneumonia, any computed tomography findings, any ground glass opacity, lymphocytopenia, elevated C-reactive protein, elevated alanine aminotransferase, elevated aspartate aminotransferase, older age and male sex. These variables typically increased in prevalence by 30-73% from mild/early disease through to moderate/severe disease. Among hospitalized patients, 30-78% of heterogeneity was explained by severity of disease. Elevated white blood cell count was strongly associated with more severe disease among moderate/severe hospitalized patients. Elevated lymphocytes, low platelets, interleukin-6, erythrocyte sedimentation rate and D-dimers showed potential associations, while fatigue, gastrointestinal symptoms, consolidation and septal thickening showed non-linear association patterns. Headache and sore throat were associated with the presence of disease, but not with more severe disease. In COVID-19, more severe disease is strongly associated with several clinical, laboratory and radiological characteristics. Symptoms and other variables in early/mild disease appear non-specific and highly heterogeneous. Clinical Trial Registration: PROSPERO CRD42020170623