Inconsistent Results for Peto Odds Ratios in Multi-Arm Studies, Network Meta-Analysis and Indirect Comparisons.

The Peto odds ratio is a well known effect measure in meta-analysis of binary outcomes. For pairwise comparisons, the Peto odds ratio estimator can be severely biased in the situation of unbalanced sample sizes in the two treatment groups or large treatment effects. In this publication, we evaluate Peto odds ratio estimators in the setting of multi-armstudies and in network meta-analysis using illustrative examples. We observe that Peto odds ratio estimators in a multi-arm study are inconsistent if the observed event probabilities are different or the sample sizes of treatment groups are unbalanced. The same problem emerges in network meta-analysis including only two-arm studies and translates to indirect comparisons of pairwise meta-analyses. We conclude that the Peto odds ratio should not be used as effect measure in network meta-analysis or indirect comparisons of pairwise meta-analyses. This article is protected by copyright. All rights reserved

Treatment as required versus regular monthly treatment in the management of neovascular age-related macular degeneration: a systematic review and meta-analysis

Background: To investigate whether treatment as required ‘pro re nata’ (PRN) versus regular monthly treatment regimens lead to differences in outcomes in neovascular age-related macular degeneration (nAMD). Regular monthly administration of vascular endothelial growth factor (VEGF) inhibitors is an established gold standard treatment, but this approach is costly. Replacement of monthly by PRN treatment can only be justified if there is no difference in patient relevant outcomes. Methods: Systematic review and meta-analysis. The intervention was PRN treatment and the comparator was monthly treatment with VEGF-inhibitors. Four bibliographic databases were searched for randomised controlled trials comparing both treatment regimens directly (head-to-head studies). The last literature search was conducted in December 2014. Risk of bias assessment was performed after the Cochrane Handbook for Systematic Reviews of Interventions. Findings: We included 3 head-to-head studies (6 reports) involving more than 2000 patients. After 2 years, the weighted mean difference in best corrected visual acuity (BCVA) was 1.9 (95% CI 0.5 to 3.3) ETDRS letters in favour of monthly treatment. Systemic adverse events were higher in PRN treated patients, but these differences were not statistically significant. After 2 years, the total number of intravitreal injections required by the patients in the PRN arms were 8.4 (95% CI 7.9 to 8.9) fewer than those having monthly treatment. The studies were considered to have a moderate risk of bias. Conclusions: PRN treatment resulted in minor but statistically significant decrease in mean BCVA which may not be clinically meaningful. There is a small increase in risk of systemic adverse events for PRN treated patients. Overall, the results indicate that an individualized treatment approach with anti-VEGF using visual acuity and OCT-guided re-treatment criteria may be appropriate for most patients with nAMD

A forward search algorithm for detecting extreme study effects in network meta-analysis

In a quantitative synthesis of studies via meta-analysis, it is possible that some studies provide a markedly different relative treatment effect or have a large impact on the summary estimate and/or heterogeneity. Extreme study effects (outliers) can be detected visually with forest/funnel plots and by using statistical outlying detection methods. A forward search (FS) algorithm is a common outlying diagnostic tool recently extended to meta-analysis. FS starts by fitting the assumed model to a subset of the data which is gradually incremented by adding the remaining studies according to their closeness to the postulated data-generating model. At each step of the algorithm, parameter estimates, measures of fit (residuals, likelihood contributions), and test statistics are being monitored and their sharp changes are used as an indication for outliers. In this article, we extend the FS algorithm to network meta-analysis (NMA). In NMA, visualization of outliers is more challenging due to the multivariate nature of the data and the fact that studies contribute both directly and indirectly to the network estimates. Outliers are expected to contribute not only to heterogeneity but also to inconsistency, compromising the NMA results. The FS algorithm was applied to real and artificial networks of interventions that include outliers. We developed an R package (NMAoutlier) to allow replication and dissemination of the proposed method. We conclude that the FS algorithm is a visual diagnostic tool that helps to identify studies that are a potential source of heterogeneity and inconsistency

Network meta-analysis of rare events using the Mantel-Haenszel method

The Mantel-Haenszel (MH) method has been used for decades to synthesize data obtained from studies that compare two interventions with respect to a binary outcome. It has been shown to perform better than the inverse-variance method or Peto's odds ratio when data is sparse. Network meta-analysis (NMA) is increasingly used to compare the safety of medical interventions, synthesizing, eg, data on mortality or serious adverse events. In this setting, sparse data occur often and yet there is to-date, no extension of the MH method for the case of NMA. In this paper, we fill this gap by presenting a MH-NMA method for odds ratios. Similarly to the pairwise MH method, we assume common treatment effects. We implement our approach in R, and we provide freely available easy-to-use routines. We illustrate our approach using data from two previously published networks. We compare our results to those obtained from three other approaches to NMA, namely, NMA with noncentral hypergeometric likelihood, an inverse-variance NMA, and a Bayesian NMA with a binomial likelihood. We also perform simulations to assess the performance of our method and compare it with alternative methods. We conclude that our MH-NMA method offers a reliable approach to the NMA of binary outcomes, especially in the case or sparse data, and when the assumption of methodological and clinical homogeneity is justifiable

The statistical importance of a study for a network meta-analysis estimate.

BACKGROUND In pairwise meta-analysis, the contribution of each study to the pooled estimate is given by its weight, which is based on the inverse variance of the estimate from that study. For network meta-analysis (NMA), the contribution of direct (and indirect) evidence is easily obtained from the diagonal elements of a hat matrix. It is, however, not fully clear how to generalize this to the percentage contribution of each study to a NMA estimate. METHODS We define the importance of each study for a NMA estimate by the reduction of the estimate's variance when adding the given study to the others. An equivalent interpretation is the relative loss in precision when the study is left out. Importances are values between 0 and 1. An importance of 1 means that the study is an essential link of the pathway in the network connecting one of the treatments with another. RESULTS Importances can be defined for two-stage and one-stage NMA. These numbers in general do not add to one and thus cannot be interpreted as 'percentage contributions'. After briefly discussing other available approaches, we question whether it is possible to obtain unique percentage contributions for NMA. CONCLUSIONS Importances generalize the concept of weights in pairwise meta-analysis in a natural way. Moreover, they are uniquely defined, easily calculated, and have an intuitive interpretation. We give some real examples for illustration

netmeta: An R Package for Network Meta-Analysis Using Frequentist Methods

Network meta-analysis compares different interventions for the same condition, by combining direct and indirect evidence derived from all eligible studies. Network metaanalysis has been increasingly used by applied scientists and it is a major research topic for methodologists. This article describes the R package netmeta, which adopts frequentist methods to fit network meta-analysis models. We provide a roadmap to perform network meta-analysis, along with an overview of the main functions of the package. We present three worked examples considering different types of outcomes and different data formats to facilitate researchers aiming to conduct network meta-analysis with netmeta

Comparative effectiveness of psychological treatments for depressive disorders in primary care: network meta-analysis

Background: A variety of psychological interventions to treat depressive disorders have been developed and are used in primary care. In a systematic review, we compared the effectiveness of psychological treatments grouped by theoretical background, intensity of contact with the health care professional, and delivery mode for depressed patients in this setting. Methods: Randomized trials comparing a psychological treatment with usual care, placebo, another psychological treatment, pharmacotherapy, or a combination treatment in adult depressed primary care patients were identified by database searches up to December 2013. We performed both conventional pairwise meta-analysis and network meta-analysis combining direct and indirect evidence. Outcome measures were response to treatment (primary outcome),remission of symptoms, post-treatment depression scores and study discontinuation. Results: A total of 37 studies with 7, 024 patients met the inclusion criteria. Among the psychological treatments investigated in at least 150 patients face-to-face cognitive behavioral therapy (CBT;OR 1.80;95 % credible interval 1.35-2.39),face-to-face counselling and psychoeducation (1.65;1.27-2.13),remote therapist lead CBT (1.87;1.38-2.53),guided self-help CBT (1.68;1.22-2.30) and no/minimal contact CBT (1.53;1.07-2.17) were superior to usual care or placebo, but not face-to-face problem-solving therapy and face-to-face interpersonal therapy. There were no statistical differences between psychological treatments apart from face-to-face interpersonal psychotherapy being inferior to remote therapist-lead CBT (0.60;0.37-0.95). Remote therapist-led (0.86;0.21-3.67),guided self-help (0.93;0.62-1.41) and no/minimal contact CBT (0.85;0.54-1.36) had similar effects as face-to-face CBT. Conclusions: The limited available evidence precludes a sufficiently reliable assessment of the comparative effectiveness of psychological treatments in depressed primary care patients. Findings suggest that psychological interventions with a cognitive behavioral approach are promising, and primarily indirect evidence indicates that it applies also when they are delivered with a reduced number of therapist contacts or remotely

Meta-analysis of diagnostic accuracy studies with multiple thresholds – comparison of different approaches

Methods for standard meta-analysis of diagnostic test accuracy studies are well established and understood. For the more complex case in which studies report test accuracy across multiple thresholds, several approaches have recently been proposed. These are based on similar ideas, but make different assumptions. In this article, we apply four different approaches to data from a recent systematic review in the area of nephrology and compare the results. The four approaches use: a linear mixed effects model, a Bayesian multinomial random effects model, a time-to-event model and a nonparametric model, respectively. In the case study data, the accuracy of neutrophil gelatinase-associated lipocalin for the diagnosis of acute kidney injury was assessed in different scenarios, with sensitivity and specificity estimates available for three thresholds in each primary study. All approaches led to plausible and mostly similar summary results. However, we found considerable differences in results for some scenarios, for example, differences in the area under the receiver operating characteristic curve (AUC) of up to 0.13. The Bayesian approach tended to lead to the highest values of the AUC, and the nonparametric approach tended to produce the lowest values across the different scenarios. Though we recommend using these approaches, our findings motivate the need for a simulation study to explore optimal choice of method in various scenarios

Reporting of eligibility criteria of randomised trials: cohort study comparing trial protocols with subsequent articles

Objective To determine whether and how eligibility criteria of participants prespecified in protocols of randomised trials are reported in subsequent articles