Meta-analysis is a statistical analysis that combines the results of relevant studies performed for same purpose and is used as a tool for evidence-based medicine to support clinical decision making. Because a meta-analysis includes advanced statistical analysis, the report is not easily understandable to clinicians. In this study, we aimed to write a guideline for clinicians to help them understand the statistical analysis part of a meta-analysis and interpret graphs and results of a meta-analysis

Kul, Seval

PhilPapers

DOI: 10.5152/EurJTher.2018.342European Journal of TherapeuticsHow to Read a Meta-Analysis? A Guideline for CliniciansBir Meta-analizi Nasıl Okunur? Klinisyenler için Bir KılavuzSeval Kul Department of Biostatistics, Gaziantep University School of Medicine, Gaziantep, TurkeyABSTRACTMeta-analysis is a statistical analysis that combines the results of relevant studies performed for same purpose and is used as a tool for evidence-based medicine to support clinical decision making. Because a meta-analysis includes advanced statistical analysis, the report is not easily understandable to clinicians. In this study, we aimed to write a guideline for clinicians to help them under-stand the statistical analysis part of a meta-analysis and interpret graphs and results of a meta-analysis.Keywords: Forest Plot, funnel plot, meta-analysis.ÖZMeta-analizi aynı amaçla gerçekleştirilen ilgili çalışmaların sonuçlarını birleştiren ve kanıta dayalı tıpta karar vermeyi desteklemek için kullanılan bir araçtır. Meta-analizleri ileri istatistiksel analizler içerdiğinden, sonuçlar klinisyenler için kolayca anlaşılabilir değil-dir. Bu çalışmada, klinisyenlerin bir meta-analizin istatistiksel analiz bölümünü anlamalarına,  sonuçlarını ve grafiklerini yorumla-malarına yardımcı yardımcı olmak bir kılavuz yazmayı amaçladık.Anahtar kelimeler: Forest Plot, funnel plot, meta-analiziORCID IDs of the authors: S.K. 0000-0002-4716-9554.Corresponding Author / Sorumlu Yazar: Seva Kul E-posta/E-mail: sevalkul@gantep.edu.tr Received / Geliş Tarihi: 25.01.2018 • Accepted / Kabul Tarihi: 27.02.2018©Copyright by 2018 Gaziantep University School of Medicine - Available online at www.eurjther.com©Telif Hakkı 2018 Gaziantep Üniversitesi Tıp Fakültesi - Makale metnine www.eurjther.com web sayfasından ulaşılabilirReview  / Derleme1INTRODUCTIONClinicians always seek for the best evidence to make a clinical de-cision for a specific disease, treatment, or patients. Meta-analysis is a statistical analysis that combines the results of relevant stud-ies performed for the same purpose and achieves overall results (1). It is used as a tool for evidencebased medicine to support clinical decision making (2). Meta-analyses are considered final studies, which show the efficacy of a drug or success of an ex-periments (2, 3). In addition to experimental studies, meta-anal-yses are also performed for observational studies to determinate important risk factors, diseases, or determinants of events. The report of a meta-analysis is usually very complex and includes several statistical stages (4). Because a meta-analysis includes advanced statistical analysis, the report is not easily understand-able to clinicians. In this study, we aimed to write a guideline for clinicians to facilitate understanding of the statistical analysis part of a meta-analysis. Cochrane collaboration has an extremely good reputation for the past 20 years to gather and summarize the best evidence by performing meta-analyses (5). The steps of a Cochrane review are as follows (6):Define questions, plan eligibility criteria, plan methods, search for studies, apply eligibility criteria, collect data, assess studies for bias risk, analyze and present results, interpret results and draw conclusions, and improve and update review. The PRISMA checklist can be used to verify the quality of a meta-analysis. It was prepared for transparent reporting of systematic reviews and meta-analysis and is widely used by researchers (7). It con-tains 27 items; of them, 7 are for the results section, which show the importance of the result section in meta-analysis. Several papers were published to explain researchers how to perform and report meta-analysis; however, a few studies focused on understanding the statistical part, which is a big challenge for clinicians (1, 6, 8, 9). In this article, we shall only focus on the un-derstanding of statistical analysis and interpretation of results by advising some steps.After reading the title of the meta-analysis, researchers should view the forest plots to understand the findings of the review. Several statistical packages, such as RevMan, R, and Medcalc, are available to perform meta-analysis, but all the packages provide extremely similar forest plots.CLINICAL AND RESEARCH CONSEQUENCESUnderstanding Forest PlotsForest plot is the most commonly used graph and is highly valu-able to visualize the results of a meta-analysis. In a forest plot, the result of each included study and the total effect can be eas-ily seen. To explain the statistical part of meta-analysis the forest plots of the publication with the title “Effects of care pathways on the in-hospital treatment of heart failure: a systematic review” were used (10).Example for Dichotomous (Variable with Two Categories) Outcome:For dichotomous outcomes, such as hospital mortality (yes-no), readmission (yes-no), or having a complication (yes-no), we gain risk ratios (RR) or, preferably, risk differences (RDs) and 95% con-fidence intervals for each study and for the overall results (2). For retrospective studies, the RR is considered odds ratio (OR); for prospective studies, the RR is considered the relative risk (11). The RD is the difference between the risk of an event in experi-mental and control groups (6). In addition to the overall compar-ison, it is also possible to show results for subgroups in the same forest plot. As it is mentioned, the aforementioned two main outcomes of “Effects of care pathways on the in-hospital treat-ment of heart failure: a systematic review” were used to show the interpretation of the statistical part of a meta-analysis with different outcomes (10). The aim of the study was to show the impact of care pathways (CPs) on chronic heart failure patient outcomes, such as hospital mortality, readmission rate, length of hospital stay, and hospitalization cost. Studies that compared CPs vs standard treatment were included in the meta-analysis. Standard care is used as the reference group to evaluate the ef-fectiveness for CPs. Therefore, RRs show relative changes in the outcome interest in the CP group vs standard treatment. RRs<1 show decrease in the risk, which is demanded for mortality. Fig-ure 1 shows forest plot to compare the rate of hospital mortality (10). For readers to understand the forest plot for hospital mor-tality, which is a dichotomous outcome, five steps were defined in this article. The advised steps are explained in Figure 1 using labeled arrows and boxes.Step 1: Understanding Groups Compared in a Meta-analysis. Look at the box shown by arrow 1. In this part, one can easily see the CP compared to standard care. Because the outcome of interest is the rate of mortality, number of the event and number of the total participants were given for each included study. For example, in the first study performed by Azad et al. (Figure 1) in 2008, among 45 participants of the CP group, no one died in hospital; however, among 46 participants of the standard care group, 2 participants died. The total number of the participants was 91 for this study.Step 2: Checking Significance for the Overall Effect. Look at the box shown by arrow 2. In this section, the p value for the combined result is given. For our example, p=0.03, which in-dicates that there is a significant difference between CP and standard care in terms of mortality rate. When the p value of heterogeneity is less than 0.05, researchers prefer the random effect model for effect size estimation to eliminate the variation between studies. In this example, the p values for heterogene-ity are 0.005, which indicates that the included studies vary from each other.Step 3: Understanding the Effect Size Estimation Method Used in the Meta-analysis. Look at the box shown by arrow 3. RRs and 95% confidence intervals (CIs) are given to demonstrate the ef-fect size for each study and the overall effect. In this meta-anal-ysis, randomized clinical trials (RCTs) and controlled clinical tri-als (CCTs) were included. They are both prospectively designed study types. Therefore, RRs can be interpreted as relative RRs. The random effect model was used to estimate RRs and CIs because of the heterogeneity detected in step 2.2Kul S. Understanding a Meta-analysis result Eur J Ther 2018; 24: 1-4Figure 1. Forest Plot of Comparison for the Rate of Hospital Mortality (10)Step 4: Interpretation of the Overall Effect Size (Overall RR). Look at the box shown by arrow  4. RR>1 is interpreted as in-crease, RR<1 is interpreted as decrease in the risk, and RR=1 is interpreted as no significant effect (11). For this study, the RR for the total effect is estimated as 0.45, which indicates a 55% de-crease of mortality rate in the CP group. In step 3, we have noted the significance of this effect, but it can be also verified by look-ing at the CI of RR. Because the CI of RR does not include “1,” it is considered significant.Step 5: Understanding of Diagram. Look at the boxes shown by arrow 5a, 5b, and 5c. At the right part of forest plot, a diagram is given to visualize the meta-analysis result. Because RR=1 is con-sidered non-significant, the relative changes are given based on the vertical line located at 1 on the scale. Diamonds () are used to show the overall effect size and effect size for subgroup analy-sis. Wider diamond demonstrates wider CI (6). There are three di-amonds in Figure 1, one for the overall effect (arrow 5a), one for RCT studies (arrow 5b), and one for CCT studies (arrow 5c). When the diamond touches the vertical line, it indicates that CI includes “1,” and p value is not significant for the effect (6). For example, for RCTs (arrow 5b), the diamond touches the vertical line and the p value is 0.230, which is non-significant. However, the diamond for CCT studies (arrow 5-c) does not touch the line and p=0.004.Example for Numerical Outcome:The mean and standard deviations are the most commonly used descriptive statistics for numerical variables, such as length of hospital stay or hospitalization cost (11). Therefore, a meta-anal-ysis gathers the means and standard deviations of individual studies to compare the overall effect. The mean difference (MD) or standardized MD (SMD) statistics are used to measure the dif-ference in the means in two groups (6). SMD, alternately, Cohen’s d is calculated as follows. Difference in mean outcome between groursSMD=  Standard deviation of outcome among participantsFor interpreting the magnitude of the SMD, SMD<0.20 is inter-preted as a small effect size and SMD>0.80 is interpreted as a large effect size (6).The forest plot of comparison for the length of hospital stay is pre-sented in Figure 2 (10). To ensure that the readers understand a forest plot, five steps are defined in this article. The advised steps are explained in Figure 2 by using labeled arrows and boxes.Step 1: Understanding Groups Compared in the Meta-analysis. Look at the box shown by arrow 1. In this part, one can easily view the CP compared to standard care. Because the outcome of interest is the length of hospital stay, mean and standard devia-tions and the number of the total participants are given for each included study. For example, in the first study performed by Kul et al. (10) in 2009, the mean length of hospital stay for the CP group was 10.4 days, and 214 patients were included in the CP group; the mean is 11.4 day and 215 patients were included in the standard care group.Step 2: Checking the Significance for the Overall Effect. Look at the box shown by arrow 2. In this section, the p value for the combined result is given. For our example, p<0.001, which means there is a highly significant difference between CP and standard care in terms of length of hospital stay.Step 3: Understanding the Effect Size Estimation Method Used in the Meta-analysis. Look at the box shown by arrow 3. The MD and 95% CIs are given to demonstrate the effect size for each study and the overall effect. For this outcome, only one RCT reported the length of hospital stay and 4 CCTs reported the outcome.Kul S. Understanding a Meta-analysis resultEur J Ther 2018; 24: 1-43Figure 2. Forest plot of comparison for the length of hospital stay (10)Step 4: Interpretation of the Overall Effect Size (Overall MD). Look at the box shown by arrow 4. The MDs are calculated by ex-tracting standard care group means from CPs. Negative MD shows a decrease in the CP group, which is desired. The MD for the overall effect is -1.89 days, which could vary between -2.44 and -1.33 days. In step 3, we have already seen the significance of this effect, but it can be also verified by noting the CI of MD. Because the CI of MD does not include “0,” it is considered significant.As mentioned above, some studies prefer to report SMD instead of MD.Step 5: Understanding of Diagram. Look at the boxes shown by arrow 5a, 5b, and 5c. At the right part of the forest plot, a diagram is provided to visualize the result of the meta-analy-sis. Because MD=0 is considered non-significant, the relative changes are given based on the vertical line located at 0 on the scale. Diamonds () are used to show the overall effect size and effect size for the subgroup analysis. Wider diamond demonstrates wider CI (6). There are three diamonds in Figure 2, one for the overall effect (arrow 5a), one for RCT studies (arrow 5b), and one for CCT studies (arrow 5c). When the di-amond touches the vertical line, which indicates that CI in-cludes “0,” and the p value is not significant for the effect (6). For example, for RCTs (arrow 5b), the diamond touches the vertical line and the p value is 0.07, which is non-significant. Nevertheless, the diamond for CCT studies (arrow 5c) does not touch the line and p<0.001.CONCLUSIONIn this tutorial article, the interpretation of the meta-analysis re-sult for numerical and dichotomous outcomes were explained in a step-by-step manner to increase the critical appraise abilities of readers for meta-analyses. We believe that the paper will be very useful for understanding and performing meta-analysis.Peer-review: Externally peer-reviewed.Conflict of Interest: No conflict interest was declared by the author.Financial Disclosure: The author declared that this study has received no financial supportHakem Değerlendirmesi: Dış bağımsız.Çıkar Çatışması: Yazar çıkar çatışması bildirmemiştir.Finansal Destek: Yazar bu çalışma için finansal destek almadığını belirt-miştir.REFERENCES 1. Leandro G. Meta-analysis in Medical Research: The handbook for the understanding and practice of meta-analysis. 2th ed. John Wiley & Sons; 2008, p. 57-61.2. Haidich AB. Meta-analysis in medical research. Hippokratia. 2010; 14: 29.3. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986; 7: 177-8.4. Egger M, Schneider M, Smith GD. Spurious precision? Meta-analysis of observational studies. BMJ 1998; 316:140-4.5. Van Tulder M, Furlan A, Bombardier C, Bouter L, Editorial Board of the Cochrane Collaboration Back Review Group. Updated method guidelines for systematic reviews in the cochrane collaboration back review group. Spine (Phila Pa 1976) 2003; 28: 1290-9.6. Higgins JPT, Green S. Cochrane handbook for systematic reviews of inter-ventions version 5.1.0. (Updated March 2011) The Cochrane collabora-tion. http://www.cochrane-handbook.org. Accessed February 2, 2018.7. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred report-ing items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009; 6: e1000097.8. Russo MW. How to review a meta-analysis. Gastroenterol Hepatol 2007; 3: 637-42.9. Ried K. Interpreting and understanding meta-analysis graphs: a practical guide. Aust Fam Physician 2006; 35: 635-8.10. Kul S, Barbieri A, Milan E, Montag I, Vanhaecht K, Panella M. Effects of care pathways on the in-hospital treatment of heart failure: a sys-tematic review. BMC cardiovascular disorders. 2012; 12: 81.11. Zhang, Jun, F. Yu Kai. “What’s the relative risk?: A method of correct-ing the odds ratio in cohort studies of common outcomes.” Jama, 1998, 280.19: 1690-1.How to cite:Kul S. Fast Reading a Meta-Analysis Result: a Guideline. Eu-rJT Eur J Ther 2018; 24: 1-4.4Kul S. Understanding a Meta-analysis result Eur J Ther 2018; 24: 1-4

How to Read a Meta-Analysis? A Guideline for Clinicians

https://philpapers.org/archive/KULHTR.pdf

How to Read a Meta-Analysis? A Guideline for Clinicians

Abstract

Similar works

Full text

Available Versions

PhilPapers