The Number of Patients and Events Required to Limit the Risk of Overestimation of Intervention Effects in Meta-Analysis—A Simulation Study

BACKGROUND: Meta-analyses including a limited number of patients and events are prone to yield overestimated intervention effect estimates. While many assume bias is the cause of overestimation, theoretical considerations suggest that random error may be an equal or more frequent cause. The independent impact of random error on meta-analyzed intervention effects has not previously been explored. It has been suggested that surpassing the optimal information size (i.e., the required meta-analysis sample size) provides sufficient protection against overestimation due to random error, but this claim has not yet been validated. METHODS: We simulated a comprehensive array of meta-analysis scenarios where no intervention effect existed (i.e., relative risk reduction (RRR) = 0%) or where a small but possibly unimportant effect existed (RRR = 10%). We constructed different scenarios by varying the control group risk, the degree of heterogeneity, and the distribution of trial sample sizes. For each scenario, we calculated the probability of observing overestimates of RRR>20% and RRR>30% for each cumulative 500 patients and 50 events. We calculated the cumulative number of patients and events required to reduce the probability of overestimation of intervention effect to 10%, 5%, and 1%. We calculated the optimal information size for each of the simulated scenarios and explored whether meta-analyses that surpassed their optimal information size had sufficient protection against overestimation of intervention effects due to random error. RESULTS: The risk of overestimation of intervention effects was usually high when the number of patients and events was small and this risk decreased exponentially over time as the number of patients and events increased. The number of patients and events required to limit the risk of overestimation depended considerably on the underlying simulation settings. Surpassing the optimal information size generally provided sufficient protection against overestimation. CONCLUSIONS: Random errors are a frequent cause of overestimation of intervention effects in meta-analyses. Surpassing the optimal information size will provide sufficient protection against overestimation

Statistical Multiplicity in Systematic Reviews of Anaesthesia Interventions: A Quantification and Comparison between Cochrane and Non-Cochrane Reviews

BACKGROUND: Systematic reviews with meta-analyses often contain many statistical tests. This multiplicity may increase the risk of type I error. Few attempts have been made to address the problem of statistical multiplicity in systematic reviews. Before the implications are properly considered, the size of the issue deserves clarification. Because of the emphasis on bias evaluation and because of the editorial processes involved, Cochrane reviews may contain more multiplicity than their non-Cochrane counterparts. This study measured the quantity of statistical multiplicity present in a population of systematic reviews and aimed to assess whether this quantity is different in Cochrane and non-Cochrane reviews. METHODS/PRINCIPAL FINDINGS: We selected all the systematic reviews published by the Cochrane Anaesthesia Review Group containing a meta-analysis and matched them with comparable non-Cochrane reviews. We counted the number of statistical tests done in each systematic review. The median number of tests overall was 10 (interquartile range (IQR) 6 to 18). The median was 12 in Cochrane and 8 in non-Cochrane reviews (difference in medians 4 (95% confidence interval (CI) 2.0-19.0). The proportion that used an assessment of risk of bias as a reason for doing extra analyses was 42% in Cochrane and 28% in non-Cochrane reviews (difference in proportions 14% (95% CI -8 to 36). The issue of multiplicity was addressed in 6% of all the reviews. CONCLUSION/SIGNIFICANCE: Statistical multiplicity in systematic reviews requires attention. We found more multiplicity in Cochrane reviews than in non-Cochrane reviews. Many of the reasons for the increase in multiplicity may well represent improved methodological approaches and greater transparency, but multiplicity may also cause an increased risk of spurious conclusions. Few systematic reviews, whether Cochrane or non-Cochrane, address the issue of multiplicity

A new tool to assess Clinical Diversity In Meta‐analyses (CDIM) of interventions

OBJECTIVE: To develop and validate Clinical Diversity In Meta-analyses (CDIM), a new tool for assessing clinical diversity between trials in meta-analyses of interventions.STUDY DESIGN AND SETTING: The development of CDIM was based on consensus work informed by empirical literature and expertise. We drafted the CDIM tool, refined it, and validated CDIM for interrater scale reliability and agreement in three groups.RESULTS: CDIM measures clinical diversity on a scale that includes four domains with 11 items overall: setting (time of conduct/country development status/units type); population (age, sex, patient inclusion criteria/baseline disease severity, comorbidities); interventions (intervention intensity/strength/duration of intervention, timing, control intervention, cointerventions); and outcome (definition of outcome, timing of outcome assessment). The CDIM is completed in two steps: first two authors independently assess clinical diversity in the four domains. Second, after agreeing upon scores of individual items a consensus score is achieved. Interrater scale reliability and agreement ranged from moderate to almost perfect depending on the type of raters.CONCLUSION: CDIM is the first tool developed for assessing clinical diversity in meta-analyses of interventions. We found CDIM to be a reliable tool for assessing clinical diversity among trials in meta-analysis.</p

Trial registration and abstracts from the American Society of Anesthesiologists meetings 2010-2013: a retrospective observational study of methods

This project is part of a Doctor of Medical Science research higher degree. There are three main parts to this project, of which this is part three. Part Three will involve a systematic search for trial registration of all randomised controlled trial abstracts presented at the ASA Annual Meetings 2010-2013. This will involve searches of the clinicaltrials.gov website (United States Trial Registry website), the International Standard Randomised Controlled Trial Number Register (ISRCTN), and the World Health Organization International Clinical Trials Registry Platform. The primary endpoint will be a comparison between the proportion of positive outcome trials which did not have prospective registration versus the proportion of positive outcome trials with prospective registration. We define a 20% reduction in positive outcomes as clinically significant when comparing trials without prospective registration to trials with prospective registration. We will also calculate the proportion of trials prospectively registered, that displayed a major discrepancy between their trial registration entry and their published paper.Purpose: Publication bias of positive studies has been demonstrated in many areas of medicine. In order to estimate publication bias in the anaesthesia literature and the effect of mandatory trial registration, we will pose the following research questions: Part One: What amount of publication bias is present in the anaesthesia literature, as seen in the abstracts presented at the 2001-2004 American Society of Anesthesiologists (ASA) annual meetings? Part Two: Has the introduction of compulsory trial registration in 2004 resulted in a decrease in publication bias, as seen in the abstracts presented at the 2010-2013 ASA meetings? Part Three: What is the proportion of prospective trial registration in abstracts presented at the 2010-2013 ASA annual meetings? What is the proportion of major discrepancies between abstract trial registration entries and the corresponding published papers? Hypotheses: That there is significant publication bias present in the anaesthesia literature. That the introduction of mandatory trial registration has not resulted in a decrease in publication bias in the anaesthesia literature. That trials that are not prospectively registered (but have a reported outcome) are more likely to have a positive outcome. Methods: All abstracts from 2001-2004 and 2010-2013 ASA annual meetings performed as randomized-controlled trials in humans will be reviewed. Their outcome results will be scored as positive or negative and the abstract quality will be assessed using a 13-point scoring system. A systematic literature search to identify any subsequent publication of the studies will be performed and the odds ratio for journal publication calculated by comparing positive vs negative studies. The odds ratio from the 2010-2013 abstracts period will be compared to the 2001-2004 period. We define a 33% decrease in the odds ratio as clinically significant. A systematic search for trial registration of the 2010-2013 abstracts will also be conducted, and any major discrepancies between registry entries and published papers recorded. We will perform a comparison between the proportion of positive outcome trials which did not have prospective registration versus the proportion of positive outcome trials with prospective registration. We define a 20% reduction in positive outcomes as clinically significant when comparing trials without prospective registration to trials with prospective registration

The relationship of study findings to publication outcome in anaesthesia research post implementation of mandatory trial registration

This project is part of a Doctor of Medical Science research higher degree. There are three main parts to this project, of which this is part two. Part Two will involve a further systematic review of all randomised controlled trial abstracts presented at the ASA Annual Meetings 2010-2013. This time period has been chosen as it is five years after the implementation of mandatory trial registration in 2004. From these abstracts, the odds ratio for abstracts with positive results proceeding to journal publication will be calculated. The primary endpoint will be a comparison between the odds ratios from Part one and from Part two. We define a 33% decrease in odds ratio as clinically significant. This corresponds to a new odds ratio of 1.33.Purpose: Publication bias of positive studies has been demonstrated in many areas of medicine. In order to estimate publication bias in the anaesthesia literature and the effect of mandatory trial registration, we will pose the following research questions: Part One: What amount of publication bias is present in the anaesthesia literature, as seen in the abstracts presented at the 2001-2004 American Society of Anesthesiologists (ASA) annual meetings? Part Two: Has the introduction of compulsory trial registration in 2004 resulted in a decrease in publication bias, as seen in the abstracts presented at the 2010-2013 ASA meetings? Part Three: What is the proportion of prospective trial registration in abstracts presented at the 2010-2013 ASA annual meetings? What is the proportion of major discrepancies between abstract trial registration entries and the corresponding published papers? Hypotheses: That there is significant publication bias present in the anaesthesia literature. That the introduction of mandatory trial registration has not resulted in a decrease in publication bias in the anaesthesia literature. That trials that are not prospectively registered (but have a reported outcome) are more likely to have a positive outcome. Methods: All abstracts from 2001-2004 and 2010-2013 ASA annual meetings performed as randomized-controlled trials in humans will be reviewed. Their outcome results will be scored as positive or negative and the abstract quality will be assessed using a 13-point scoring system. A systematic literature search to identify any subsequent publication of the studies will be performed and the odds ratio for journal publication calculated by comparing positive vs negative studies. The odds ratio from the 2010-2013 abstracts period will be compared to the 2001-2004 period. We define a 33% decrease in the odds ratio as clinically significant. A systematic search for trial registration of the 2010-2013 abstracts will also be conducted, and any major discrepancies between registry entries and published papers recorded. We will perform a comparison between the proportion of positive outcome trials which did not have prospective registration versus the proportion of positive outcome trials with prospective registration. We define a 20% reduction in positive outcomes as clinically significant when comparing trials without prospective registration to trials with prospective registration

Additional file 1: of Viscoelastic testing for hepatic surgery: a systematic review with meta-analysis—a protocol

PRISMA checklist for submission. (DOCX 17 kb

Preoperative evaluation of the adult patient undergoing non-cardiac surgery: guidelines from the European Society of Anaesthesiology

The purpose of these guidelines on the preoperative evaluation of the adult non-cardiac surgery patient is to present recommendations based on available relevant clinical evidence. The ultimate aims of preoperative evaluation are two-fold. First, we aim to identify those patients for whom the perioperative period may constitute an increased risk of morbidity and mortality, aside from the risks associated with the underlying disease. Second, this should help us to design perioperative strategies that aim to reduce additional perioperative risks. Very few well performed randomised studies on the topic are available and many recommendations rely heavily on expert opinion and are adapted specifically to the healthcare systems in individual countries. This report aims to provide an overview of current knowledge on the subject with an assessment of the quality of the evidence in order to allow anaesthetists all over Europe to integrate - wherever possible - this knowledge into daily patient care. The Guidelines Committee of the European Society of Anaesthesiology (ESA) formed a task force with members of subcommittees of scientific subcommittees and individual members of the ESA. Electronic databases were searched from the year 2000 until July 2010 without language restrictions. These searches produced 15 425 abstracts. Relevant systematic reviews with meta-analyses, randomised controlled trials, cohort studies, case-control studies and cross-sectional surveys were selected. The Scottish Intercollegiate Guidelines Network grading system was used to assess the level of evidence and to grade recommendations. The final draft guideline was posted on the ESA website for 4 weeks and the link was sent to all ESA members, individual or national (thus including most European national anaesthesia societies). Comments were collated and the guidelines amended as appropriate. When the final draft was complete, the Guidelines Committee and ESA Board ratified the guideline

The relationship between study findings and publication outcome in anesthesia research following implementation of mandatory trial registration: A systematic review of publication bias.

Previously, we reviewed 1052 randomized-controlled trial abstracts presented at the American Society of Anesthesiologists annual meetings from 2001-2004. We found significant positive publication bias in the period examined, with the odds ratio for abstracts with positive results proceeding to journal publication over those with null results being 2.01 [95% confidence interval: 1.52, 2.66; P < 0.001]. Mandatory trial registration was introduced in 2005 as a required standard for publication. We sought to examine whether mandatory trial registration has decreased publication bias in the anesthesia and perioperative medicine literature. We reviewed all abstracts from the 2010-2016 American Society of Anesthesiologists meetings that reported on randomized-controlled trials in humans. We scored the result of each abstract as positive or null according to a priori definitions. We systematically searched for any subsequent publication of the studies and calculated the odds ratio for journal publication, comparing positive vs null studies. We compared the odds ratio from the 2010-2016 abstracts (post-mandatory trial registration) with the odds ratio from the 2001-2004 abstracts (pre-mandatory trial registration) as a ratio of odds ratios. We defined a 33% decrease in the odds ratio as significant, corresponding to a new odds ratio of 1.33. We reviewed 9789 abstracts; 1049 met inclusion criteria as randomized-controlled trials, with 542 (51.7%) of the abstracts going on to publication. The odds ratio for abstracts with positive results proceeding to journal publication was 1.28 [95% CI: 0.97, 1.67; P = 0.076]. With adjustment for sample size and abstract quality, the difference in publication rate between positive and null abstracts was statistically significant (odds ratio 1.34; 95% CI: 1.02, 1.76; P = 0.037). The ratio of odds ratios, comparing the odds ratio from the 2010-2016 abstracts (post-mandatory trial registration) to the odds ratio from the 2001-2004 abstracts (pre-mandatory trial registration), was 0.63 (95% CI: 0.43, 0.93); P = 0.021). We present the first study in the anesthesia and perioperative medicine literature that examines and compares publication bias over two discrete periods of time, prior to and after the implementation of mandatory trial registration. Our results suggest that the amount of publication bias has decreased markedly following implementation of mandatory trial registration. However, some positive publication bias in the anesthesia and perioperative medicine literature remains