The Number of Patients and Events Required to Limit the Risk of Overestimation of Intervention Effects in Meta-Analysis—A Simulation Study

BACKGROUND: Meta-analyses including a limited number of patients and events are prone to yield overestimated intervention effect estimates. While many assume bias is the cause of overestimation, theoretical considerations suggest that random error may be an equal or more frequent cause. The independent impact of random error on meta-analyzed intervention effects has not previously been explored. It has been suggested that surpassing the optimal information size (i.e., the required meta-analysis sample size) provides sufficient protection against overestimation due to random error, but this claim has not yet been validated. METHODS: We simulated a comprehensive array of meta-analysis scenarios where no intervention effect existed (i.e., relative risk reduction (RRR) = 0%) or where a small but possibly unimportant effect existed (RRR = 10%). We constructed different scenarios by varying the control group risk, the degree of heterogeneity, and the distribution of trial sample sizes. For each scenario, we calculated the probability of observing overestimates of RRR>20% and RRR>30% for each cumulative 500 patients and 50 events. We calculated the cumulative number of patients and events required to reduce the probability of overestimation of intervention effect to 10%, 5%, and 1%. We calculated the optimal information size for each of the simulated scenarios and explored whether meta-analyses that surpassed their optimal information size had sufficient protection against overestimation of intervention effects due to random error. RESULTS: The risk of overestimation of intervention effects was usually high when the number of patients and events was small and this risk decreased exponentially over time as the number of patients and events increased. The number of patients and events required to limit the risk of overestimation depended considerably on the underlying simulation settings. Surpassing the optimal information size generally provided sufficient protection against overestimation. CONCLUSIONS: Random errors are a frequent cause of overestimation of intervention effects in meta-analyses. Surpassing the optimal information size will provide sufficient protection against overestimation

Estimating required information size by quantifying diversity in random-effects model meta-analyses

<p>Abstract</p> <p>Background</p> <p>There is increasing awareness that meta-analyses require a sufficiently large information size to detect or reject an anticipated intervention effect. The required information size in a meta-analysis may be calculated from an anticipated <it>a priori </it>intervention effect or from an intervention effect suggested by trials with low-risk of bias.</p> <p>Methods</p> <p>Information size calculations need to consider the total model variance in a meta-analysis to control type I and type II errors. Here, we derive an adjusting factor for the required information size under any random-effects model meta-analysis.</p> <p>Results</p> <p>We devise a measure of diversity (<it>D</it>²) in a meta-analysis, which is the relative variance reduction when the meta-analysis model is changed from a random-effects into a fixed-effect model. <it>D</it>²is the percentage that the between-trial variability constitutes of the sum of the between-trial variability and a sampling error estimate considering the required information size. <it>D</it>²is different from the intuitively obvious adjusting factor based on the common quantification of heterogeneity, the inconsistency (<it>I</it>²), which may underestimate the required information size. Thus, <it>D</it>²and <it>I</it>²are compared and interpreted using several simulations and clinical examples. In addition we show mathematically that diversity is equal to or greater than inconsistency, that is <it>D</it>²≥ <it>I</it>², for all meta-analyses.</p> <p>Conclusion</p> <p>We conclude that <it>D</it>²seems a better alternative than <it>I</it>²to consider model variation in any random-effects meta-analysis despite the choice of the between trial variance estimator that constitutes the model. Furthermore, <it>D</it>²can readily adjust the required information size in any random-effects model meta-analysis.</p

Intermediate Dose Low-Molecular-Weight Heparin for Thrombosis Prophylaxis:Systematic Review with Meta-Analysis and Trial Sequential Analysis

Different doses of low-molecular-weight heparin (LMWH) are registered and used for thrombosis prophylaxis. We assessed benefits and harms of thrombosis prophylaxis with a predefined intermediate-dose LMWH compared with placebo or no treatment in patients at risk of venous thromboembolism (VTE). We performed a systematic review with meta-analyses and trial sequential analyses (TSA) following The Cochrane Handbook for Systematic Reviews of Interventions . Medline, Cochrane CENTRAL, Web of Science, and Embase were searched up to December 2018. Trials were evaluated for risk of bias and quality of evidence was assessed following the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. Seventy randomized trials with 34,046 patients were included. Eighteen (26%) had overall low risk of bias. There was a small statistically significant effect of LMWH on all-cause mortality (risk ratio [RR]: 0.96; TSA-adjusted confidence interval [TSA-adjusted CI]: 0.94-0.98) which disappeared in sensitivity analyses excluding ambulatory cancer patients (RR: 0.99; TSA-adjusted CI: 0.84-1.16). There was moderate-quality evidence for a statistically significant beneficial effect on symptomatic VTE (odds ratio [OR]: 0.59; TSA-adjusted CI: 0.53-0.67; number needed to treat [NNT]: 76; 95% CI: 60-106) and a statistically significant harmful effect on major bleeding (Peto OR: 1.66; TSA-adjusted CI: 1.31-2.10; number needed to harm [NNH]: 212; 95% CI: 142-393). There were no significant intervention effects on serious adverse events. The use of intermediate-dose LMWH for thrombosis prophylaxis compared with placebo or no treatment was associated with a small statistically significant reduction of all-cause mortality that disappeared in sensitivity analyses excluding trials that evaluated LMWH for anticancer treatment. Intermediate-dose LMWH provides benefits in terms of VTE prevention while it increases major bleeding

Statistical Multiplicity in Systematic Reviews of Anaesthesia Interventions: A Quantification and Comparison between Cochrane and Non-Cochrane Reviews

BACKGROUND: Systematic reviews with meta-analyses often contain many statistical tests. This multiplicity may increase the risk of type I error. Few attempts have been made to address the problem of statistical multiplicity in systematic reviews. Before the implications are properly considered, the size of the issue deserves clarification. Because of the emphasis on bias evaluation and because of the editorial processes involved, Cochrane reviews may contain more multiplicity than their non-Cochrane counterparts. This study measured the quantity of statistical multiplicity present in a population of systematic reviews and aimed to assess whether this quantity is different in Cochrane and non-Cochrane reviews. METHODS/PRINCIPAL FINDINGS: We selected all the systematic reviews published by the Cochrane Anaesthesia Review Group containing a meta-analysis and matched them with comparable non-Cochrane reviews. We counted the number of statistical tests done in each systematic review. The median number of tests overall was 10 (interquartile range (IQR) 6 to 18). The median was 12 in Cochrane and 8 in non-Cochrane reviews (difference in medians 4 (95% confidence interval (CI) 2.0-19.0). The proportion that used an assessment of risk of bias as a reason for doing extra analyses was 42% in Cochrane and 28% in non-Cochrane reviews (difference in proportions 14% (95% CI -8 to 36). The issue of multiplicity was addressed in 6% of all the reviews. CONCLUSION/SIGNIFICANCE: Statistical multiplicity in systematic reviews requires attention. We found more multiplicity in Cochrane reviews than in non-Cochrane reviews. Many of the reasons for the increase in multiplicity may well represent improved methodological approaches and greater transparency, but multiplicity may also cause an increased risk of spurious conclusions. Few systematic reviews, whether Cochrane or non-Cochrane, address the issue of multiplicity

Plexus anesthesia versus general anesthesia in patients for carotid endarterectomy with patch angioplasty:Protocol for a systematic review with meta-analyses and Trial Sequential Analysis of randomized clinical trials

Introduction: Traditional carotid endarterectomy is considered to be the standard technique for prevention of a new stroke in patients with a symptomatic carotid stenosis. Use of plexus anesthesia or general anesthesia in traditional carotid endarterectomy is, to date, not unequivocally proven to be superior to one other. A systematic review is needed for evaluation of benefits and harms to determine which technique, plexus anesthesia or general anesthesia is more effective for traditional carotid endarterectomy in patients with symptomatic carotid stenosis. Methods and outcomes: The review will be conducted according to this protocol following the recommendations of the ‘Cochrane Handbook for Systematic Reviews’ and reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses. Randomized Clinical Trials comparing plexus anesthesia versus general anesthesia in traditional carotid endarterectomy will be included. Primary outcomes will be postoperative death and/ or stroke (<30 days) and serious adverse events. Secondary outcomes will be non-serious adverse events. We will primarily base our conclusions on meta-analyses of trials with overall low risk of bias. We will use Trial Sequential Analysis to assist the evaluation of imprecision in Grading of Recommendations Assessment, Development and Evaluation. However, if pooled point-estimates of all trials are similar to pooled point-estimates of trials with overall low risk of bias and there is lack of a statistical significant interaction between estimates from trials with overall high risk of bias and trials with overall low risk of bias we will consider the Trial Sequential Analysis adjusted confidence interval precision of the estimate achieved in all trials as the result of our meta-analyses. Ethics and dissemination: The proposed systematic review will collect and analyze secondary data from already performed studies therefore ethical approval is not required. The results of the systematic review will be disseminated by publication in a peer-review journal and submitted for presentation at relevant conferences

Glucocorticosteroids for sepsis:systematic review with meta-analysis and trial sequential analysis

Glucocorticosteroids (steroids) are widely used for sepsis patients. However, the potential benefits and harms of both high and low dose steroids remain unclear. A systematic review of randomised clinical trials with meta-analysis and trial sequential analysis (TSA) might shed light on this clinically important question. A systematic review was conducted according to a published protocol and The Cochrane Handbook methodology including meta-analyses, TSA of randomised clinical trials, and external validity estimation (GRADE). Randomised clinical trials evaluating steroids were included for sepsis patients (systemic inflammatory response syndrome, sepsis, severe sepsis or septic shock) aged > 18 years. Cochrane Central Register of Controlled Trials (CENTRAL), PubMed/Medline, Embase, Web of Science and Cinahl were searched until 18 February 2015. No language restrictions were applied. Primary outcomes were mortality at longest follow-up and serious adverse events. A total of 35 trials randomising 4682 patients were assessed and reviewed in full text. All trials but two had high risk of bias. No statistically significant effect was found for any dose of steroids versus placebo or no intervention on mortality at maximal follow-up [relative risk (RR) 0.89; TSA adjusted confidence interval (CI) 0.74-1.08]. Two trials with low risk of bias also showed no statistically significant difference (random-effects model RR 0.38, 95 % CI 0.06-2.42). Similar results were obtained in subgroups of trials stratified according to high (> 500 mg) or low (a parts per thousand currency sign500 mg) dose hydrocortisone (or equivalent) (RR 0.87; TSA-adjusted CI 0.38-1.99; and RR 0.90; TSA-adjusted CI 0.49-1.67, respectively). There were also no statistically significant effects on serious adverse events other than mortality (RR 1.02; TSA-adjusted CI 0.7-1.48). The effects did not vary according to the degree of sepsis. TSA showed that many more randomised patients are needed before definitive conclusions may be drawn. Evidence to support or negate the use of steroids in any dose in sepsis patients is lacking. The results of ongoing and future well-designed, large randomised clinical trials are needed

Evidence at a glance: error matrix approach for overviewing available evidence

Contains fulltext : 88651.pdf (publisher's version ) (Open Access)BACKGROUND: Clinical evidence continues to expand and is increasingly difficult to overview. We aimed at conceptualizing a visual assessment tool, i.e., a matrix for overviewing studies and their data in order to assess the clinical evidence at a glance. METHODS: A four-step matrix was constructed using the three dimensions of systematic error, random error, and design error. Matrix step I ranks the identified studies according to the dimensions of systematic errors and random errors. Matrix step II orders the studies according to the design errors. Matrix step III assesses the three dimensions of errors in studies. Matrix step IV assesses the size and direction of the intervention effect. RESULTS: The application of this four-step matrix is illustrated with two examples: peri-operative beta-blockade initialized in relation to surgery versus placebo for major non-cardiac surgery, and antiarrhythmics for maintaining sinus rhythm after cardioversion of atrial fibrillation. When clinical evidence is deemed both internally and externally valid, the size of the intervention effect is to be assessed. CONCLUSION: The error matrix provides an overview of the validity of the available evidence at a glance, and may assist in deciding which interventions to use in clinical practice

Association between AIRE gene polymorphism and rheumatoid arthritis: a systematic review and meta-analysis of case-control studies.

Autoimmune regulator (AIRE) is a transcription factor that functions as a novel player in immunological investigations. In the thymus, it has a pivotal role in the negative selection of naive T-cells during central tolerance. Experimental studies have shown that single nucleotide polymorphism (SNP) alters transcription of the AIRE gene. SNPs thereby provide a less efficient negative selection, propagate higher survival of autoimmune T-cells, and elevate susceptibility to autoimmune diseases. To date, only rheumatoid arthritis (RA) has been analysed by epidemiological investigations in relation to SNPs in AIRE. In our meta-analysis, we sought to encompass case-control studies and confirm that the association between SNP occurrence and RA. After robust searches of Embase, PubMed, Cochrane Library, and Web of Science databases, we found 19 articles that included five independent studies. Out of 11 polymorphisms, two (rs2075876, rs760426) were common in the five case-control studies. Thus, we performed a meta-analysis for rs2075876 (7145 cases and 8579 controls) and rs760426 (6696 cases and 8164 controls). Our results prove that rs2075876 and rs760426 are significantly associated with an increased risk of RA in allelic, dominant, recessive, codominant heterozygous, and codominant homozygous genetic models. These findings are primarily based on data from Asian populations

Standardized EEG interpretation accurately predicts prognosis after cardiac arrest.

OBJECTIVE: To identify reliable predictors of outcome in comatose patients after cardiac arrest using a single routine EEG and standardized interpretation according to the terminology proposed by the American Clinical Neurophysiology Society. METHODS: In this cohort study, 4 EEG specialists, blinded to outcome, evaluated prospectively recorded EEGs in the Target Temperature Management trial (TTM trial) that randomized patients to 33°C vs 36°C. Routine EEG was performed in patients still comatose after rewarming. EEGs were classified into highly malignant (suppression, suppression with periodic discharges, burst-suppression), malignant (periodic or rhythmic patterns, pathological or nonreactive background), and benign EEG (absence of malignant features). Poor outcome was defined as best Cerebral Performance Category score 3-5 until 180 days. RESULTS: Eight TTM sites randomized 202 patients. EEGs were recorded in 103 patients at a median 77 hours after cardiac arrest; 37% had a highly malignant EEG and all had a poor outcome (specificity 100%, sensitivity 50%). Any malignant EEG feature had a low specificity to predict poor prognosis (48%) but if 2 malignant EEG features were present specificity increased to 96% (p &lt; 0.001). Specificity and sensitivity were not significantly affected by targeted temperature or sedation. A benign EEG was found in 1% of the patients with a poor outcome. CONCLUSIONS: Highly malignant EEG after rewarming reliably predicted poor outcome in half of patients without false predictions. An isolated finding of a single malignant feature did not predict poor outcome whereas a benign EEG was highly predictive of a good outcome

Clinical Examination for the Prediction of Mortality in the Critically Ill:The Simple Intensive Care Studies-I

Objectives: Caregivers use clinical examination to timely recognize deterioration of a patient, yet data on the prognostic value of clinical examination are inconsistent. In the Simple Intensive Care Studies-I, we evaluated the association of clinical examination findings with 90-day mortality in critically ill patients. Design: Prospective single-center cohort study. Setting: ICU of a single tertiary care level hospital between March 27, 2015, and July 22, 2017. Patients: All consecutive adults acutely admitted to the ICU and expected to stay for at least 24 hours. Interventions: A protocolized clinical examination of 19 clinical signs conducted within 24 hours of admission. Measurements Main Results: Independent predictors of 90-day mortality were identified using multivariable logistic regression analyses. Model performance was compared with established prognostic risk scores using area under the receiver operating characteristic curves. Robustness of our findings was tested by internal bootstrap validation and adjustment of the threshold for statistical significance. A total of 1,075 patients were included, of whom 298 patients (28%) had died at 90-day follow-up. Multivariable analyses adjusted for age and norepinephrine infusion rate demonstrated that the combination of higher respiratory rate, higher systolic blood pressure, lower central temperature, altered consciousness, and decreased urine output was independently associated with 90-day mortality (area under the receiver operating characteristic curves = 0.74; 95% CI, 0.71-0.78). Clinical examination had a similar discriminative value as compared with the Simplified Acute Physiology Score-II (area under the receiver operating characteristic curves = 0.76; 95% CI, 0.73-0.79; p = 0.29) and Acute Physiology and Chronic Health Evaluation-IV (using area under the receiver operating characteristic curves = 0.77; 95% CI, 0.74-0.80; p = 0.16) and was significantly better than the Sequential Organ Failure Assessment (using area under the receiver operating characteristic curves = 0.67; 95% CI, 0.64-0.71; p <0.001). Conclusions: Clinical examination has reasonable discriminative value for assessing 90-day mortality in acutely admitted ICU patients. In our study population, a single, protocolized clinical examination had similar prognostic abilities compared with the Simplified Acute Physiology Score-II and Acute Physiology and Chronic Health Evaluation-IV and outperformed the Sequential Organ Failure Assessment score