Evaluating the quality of evidence from a network meta-analysis

Systematic reviews that collate data about the relative effects of multiple interventions via network meta-analysis are highly informative for decision-making purposes. A network meta-analysis provides two types of findings for a specific outcome: the relative treatment effect for all pairwise comparisons, and a ranking of the treatments. It is important to consider the confidence with which these two types of results can enable clinicians, policy makers and patients to make informed decisions. We propose an approach to determining confidence in the output of a network meta-analysis. Our proposed approach is based on methodology developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group for pairwise meta-analyses. The suggested framework for evaluating a network meta-analysis acknowledges (i) the key role of indirect comparisons (ii) the contributions of each piece of direct evidence to the network meta-analysis estimates of effect size; (iii) the importance of the transitivity assumption to the validity of network meta-analysis; and (iv) the possibility of disagreement between direct evidence and indirect evidence. We apply our proposed strategy to a systematic review comparing topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations. The proposed framework can be used to determine confidence in the results from a network meta-analysis. Judgements about evidence from a network meta-analysis can be different from those made about evidence from pairwise meta-analyses. © 2014 Salanti et al

Allowing for uncertainty due to missing and LOCF imputed outcomes in meta-analysis

The use of the last observation carried forward (LOCF) method for imputing missing outcome data in randomized clinical trials has been much criticized and its shortcomings are well understood. However, only recently have published studies widely started using more appropriate imputation methods. Consequently, meta-analyses often include several studies reporting their results according to LOCF. The results from such meta-analyses are potentially biased and overprecise. We develop methods for estimating summary treatment effects for continuous outcomes in the presence of both missing and LOCF-imputed outcome data. Our target is the treatment effect if complete follow-up was obtained even if some participants drop out from the protocol treatment. We extend a previously developed meta-analysis model, which accounts for the uncertainty due to missing outcome data via an informative missingness parameter. The extended model includes an extra parameter that reflects the level of prior confidence in the appropriateness of the LOCF imputation scheme. Neither parameter can be informed by the data and we resort to expert opinion and sensitivity analysis. We illustrate the methodology using two meta-analyses of pharmacological interventions for depression

Allowing for informative missingness in aggregate data meta-analysis with continuous or binary outcomes: Extensions to metamiss

Missing outcome data can invalidate the results of randomized trials and their meta-analysis. However, addressing missing data is often a challenging issue because it requires untestable assumptions. The impact of missing outcome data on the meta-analysis summary effect can be explored by assuming a relationship between the outcome in the observed and the missing participants via an informative missingness parameter. The informative missingness parameters cannot be estimated from the observed data, but they can be specified, with associated uncertainty, using evidence external to the meta-analysis, such as expert opinion. The use of informative missingness parameters in pairwise meta-analysis of aggregate data with binary outcomes has been previously implemented in Stata by the metamiss command. In this article, we present the new command metamiss2, which is an extension of metamiss for binary or continuous data in pairwise or network meta-analysis. The command can be used to explore the robustness of results to different assumptions about the missing data via sensitivity analysis

A Microsoft-Excel-based tool for running and critically appraising network meta-analyses--an overview and application of NetMetaXL.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.BACKGROUND: The use of network meta-analysis has increased dramatically in recent years. WinBUGS, a freely available Bayesian software package, has been the most widely used software package to conduct network meta-analyses. However, the learning curve for WinBUGS can be daunting, especially for new users. Furthermore, critical appraisal of network meta-analyses conducted in WinBUGS can be challenging given its limited data manipulation capabilities and the fact that generation of graphical output from network meta-analyses often relies on different software packages than the analyses themselves. METHODS: We developed a freely available Microsoft-Excel-based tool called NetMetaXL, programmed in Visual Basic for Applications, which provides an interface for conducting a Bayesian network meta-analysis using WinBUGS from within Microsoft Excel. . This tool allows the user to easily prepare and enter data, set model assumptions, and run the network meta-analysis, with results being automatically displayed in an Excel spreadsheet. It also contains macros that use NetMetaXL's interface to generate evidence network diagrams, forest plots, league tables of pairwise comparisons, probability plots (rankograms), and inconsistency plots within Microsoft Excel. All figures generated are publication quality, thereby increasing the efficiency of knowledge transfer and manuscript preparation. RESULTS: We demonstrate the application of NetMetaXL using data from a network meta-analysis published previously which compares combined resynchronization and implantable defibrillator therapy in left ventricular dysfunction. We replicate results from the previous publication while demonstrating result summaries generated by the software. CONCLUSIONS: Use of the freely available NetMetaXL successfully demonstrated its ability to make running network meta-analyses more accessible to novice WinBUGS users by allowing analyses to be conducted entirely within Microsoft Excel. NetMetaXL also allows for more efficient and transparent critical appraisal of network meta-analyses, enhanced standardization of reporting, and integration with health economic evaluations which are frequently Excel-based.CC is a recipient of a Vanier Canada Graduate Scholarship from the Canadian Institutes of Health Research (funding reference number—CGV 121171) and is a trainee on the Canadian Institutes of Health Research Drug Safety and Effectiveness Network team grant (funding reference number—116573). BH is funded by a New Investigator award from the Canadian Institutes of Health Research and the Drug Safety and Effectiveness Network. This research was partly supported by funding from CADTH as part of a project to develop Excel-based tools to support the conduct of health technology assessments. This research was also supported by Cornerstone Research Group

Comparative effects of different dietary approaches on blood pressure in hypertensive and pre-hypertensive patients: A systematic review and network meta-analysis

Pairwise meta-analyses have shown beneficial effects of individual dietary approaches on blood pressure but their comparative effects have not been established. Objective: Therefore we performed a systematic review of different dietary intervention trials and estimated the aggregate blood pressure effects through network meta-analysis including hypertensive and pre-hypertensive patients. Design: PubMed, Cochrane CENTRAL, and Google Scholar were searched until June 2017. The inclusion criteria were defined as follows: i) Randomized trial with a dietary approach; ii) hypertensive and pre-hypertensive adult patients; and iii) minimum intervention period of 12 weeks. In order to determine the pooled effect of each intervention relative to each of the other intervention for both diastolic and systolic blood pressure (SBP and DBP), random effects network meta-analysis was performed. Results: A total of 67 trials comparing 13 dietary approaches (DASH, lowfat, moderate-carbohydrate, high-protein, low-carbohydrate, Mediterranean, Palaeolithic, vegetarian, low-GI/GL, low-sodium, Nordic, Tibetan, and control) enrolling 17,230 participants were included. In the network metaanalysis, the DASH, Mediterranean, low-carbohydrate, Palaeolithic, high-protein, low-glycaemic index, lowsodium, and low-fat dietary approaches were significantly more effective in reducing SBP (¡8.73 to ¡2.32 mmHg) and DBP (¡4.85 to ¡1.27 mmHg) compared to a control diet. According to the SUCRAs, the DASH diet was ranked the most effective dietary approach in reducing SBP (90%) and DBP (91%), followed by the Palaeolithic, and the low-carbohydrate diet (ranked 3rd for SBP) or the Mediterranean diet (ranked 3rd for DBP). For most comparisons, the credibility of evidence was rated very low to moderate, with the exception for the DASH vs. the low-fat dietary approach for which the quality of evidence was rated high. Conclusion: The present network meta-analysis suggests that the DASH dietary approach might be the most effective dietary measure toreduce blood pressure among hypertensive and pre-hypertensive patients based on high quality evidence

What guidance are researchers given on how to present network meta-analyses to end-users such as policymakers and clinicians? A systematic review

© 2014 Sullivan et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Introduction: Network meta-analyses (NMAs) are complex methodological approaches that may be challenging for non-technical end-users, such as policymakers and clinicians, to understand. Consideration should be given to identifying optimal approaches to presenting NMAs that help clarify analyses. It is unclear what guidance researchers currently have on how to present and tailor NMAs to different end-users. Methods: A systematic review of NMA guidelines was conducted to identify guidance on how to present NMAs. Electronic databases and supplementary sources were searched for NMA guidelines. Presentation format details related to sample formats, target audiences, data sources, analysis methods and results were extracted and frequencies tabulated. Guideline quality was assessed following criteria developed for clinical practice guidelines. Results: Seven guidelines were included. Current guidelines focus on how to conduct NMAs but provide limited guidance to researchers on how to best present analyses to different end-users. None of the guidelines provided reporting templates. Few guidelines provided advice on tailoring presentations to different end-users, such as policymakers. Available guidance on presentation formats focused on evidence networks, characteristics of individual trials, comparisons between direct and indirect estimates and assumptions of heterogeneity and/or inconsistency. Some guidelines also provided examples of figures and tables that could be used to present information. Conclusions: Limited guidance exists for researchers on how best to present NMAs in an accessible format, especially for non-technical end-users such as policymakers and clinicians. NMA guidelines may require further integration with end-users' needs, when NMAs are used to support healthcare policy and practice decisions. Developing presentation formats that enhance understanding and accessibility of NMAs could also enhance the transparency and legitimacy of decisions informed by NMAs.The Canadian Institute of Health Research (CIHR) Drug Safety and Effectiveness Network (Funding reference number – 116573)

Comparative efficacy, cognitive effects and acceptability of electroconvulsive therapies for the treatment of depression: Protocol for a systematic review and network meta-analysis

Introduction There have been important advances in the use of electroconvulsive therapy (ECT) to treat major depressive episodes. These include variations to the type of stimulus the brain regions stimulated, and the stimulus parameters (eg, stimulus duration/pulse width). Our aim is to investigate ECT types using a network meta-analysis (NMA) approach and report on comparative treatment efficacy, cognitive side effects and acceptability. Method We will conduct a systematic review to identify randomised controlled trials that compared two or more ECT protocols to treat depression. This will be done using the following databases: Embase, MEDLINE PubMed, Web of Science, Scopus, PsycINFO, Cochrane CENTRAL and will be supplemented by personal contacts with researchers in the field. All authors will be contacted to provide missing information. Primary outcomes will be symptom severity on a validated continuous clinician-rated scale of depression, cognitive functioning measured using anterograde verbal recall, and acceptability calculated using all-cause drop-outs. Secondary outcomes will include response and remission rates, autobiographical memory following a course of ECT, and anterograde visuospatial recall. Bayesian random effects hierarchical models will compare ECT types. Additional meta-regressions may be conducted to determine the impact of effect modifiers and patient-specific prognostic factors if sufficient data are available. Discussion This NMA will facilitate clinician decision making and allow more sophisticated selection of ECT type according to the balance of efficacy, cognitive side effects and acceptability. Ethics This systematic review and NMA does not require research ethics approval as it will use published aggregate data and will not collect nor disclose individually identifiable participant data. PROSPERO registration number CRD42022357098

Impact of placebo arms on outcomes in antidepressant trials:systematic review and meta-regression analysis

Background: There is debate in the literature as to whether inclusion of a placebo arm may alter characteristics of antidepressant trials. However, previous research has focused on response rates of various antidepressants on average only, ignoring potential differences among drugs or other aspects of trial findings. Little is known about the impact of a placebo arm on all-cause dropout and dropout due to adverse events.Methods: We carried out a systematic review of published and unpublished double-blind randomized controlled trials (RCTs) for the acute treatment of unipolar major depression (update: January 2016). The probability of being allocated to placebo (π) was the exposure of interest, and we examined its influence on responders (efficacy), all-cause dropouts (acceptability) and dropouts due to adverse events (tolerability), while accounting for differences in drugs, trials and patient characteristics in multivariate random effects meta-regression.Results: We included 421 studies (68 305 participants) comparing 16 antidepressants or placebo; π ranged from 20% to 50%. Response rate was lower [risk ratio (RR) 0.87; 95% confidence interval (CI) 0.83, 0.92] and all-cause dropout rate higher (RR 1.19; 95% CI 1.08, 1.31) for the same antidepressants in placebo-controlled trials compared with head-to-head trials. The probability of responding decreased by 3% (95% CI 2-5%) for every 10% increase in π, whereas the risk of all-cause dropout increased by 4% (95% CI 1-7%). Tolerability was unaffected by π. Response rate was inversely correlated with dropouts due to any cause (correlation coefficient -0.48; 95% CI -0.58, -0.36) and due to adverse events (-0.34; 95% CI -0.44, -0.23).Conclusions: For the same antidepressant, response rate was on average smaller and dropouts higher when placebo was included; however, no association was found with dropouts due to adverse events. Decreased patient expectations, larger dropout rates and use of inappropriate statistical methods to impute missing data may explain this phenomenon. The findings call for caution in the integration of randomized evidence involving placebo arms.</p