Detecting small-study effects and funnel plot asymmetry in meta-analysis of survival data: A comparison of new and existing tests.

Small-study effects are a common threat in systematic reviews and may indicate publication bias. Their existence is often verified by visual inspection of the funnel plot. Formal tests to assess the presence of funnel plot asymmetry typically estimate the association between the reported effect size and their standard error, the total sample size, or the inverse of the total sample size. In this paper, we demonstrate that the application of these tests may be less appropriate in meta-analysis of survival data, where censoring influences statistical significance of the hazard ratio. We subsequently propose 2 new tests that are based on the total number of observed events and adopt a multiplicative variance component. We compare the performance of the various funnel plot asymmetry tests in an extensive simulation study where we varied the true hazard ratio (0.5 to 1), the number of published trials (N=10 to 100), the degree of censoring within trials (0% to 90%), and the mechanism leading to participant dropout (noninformative versus informative). Results demonstrate that previous well-known tests for detecting funnel plot asymmetry suffer from low power or excessive type-I error rates in meta-analysis of survival data, particularly when trials are affected by participant dropout. Because our novel test (adopting estimates of the asymptotic precision as study weights) yields reasonable power and maintains appropriate type-I error rates, we recommend its use to evaluate funnel plot asymmetry in meta-analysis of survival data. The use of funnel plot asymmetry tests should, however, be avoided when there are few trials available for any meta-analysis

Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study

<p>Abstract</p> <p>Background</p> <p>In meta-analysis, the presence of funnel plot asymmetry is attributed to publication or other small-study effects, which causes larger effects to be observed in the smaller studies. This issue potentially mean inappropriate conclusions are drawn from a meta-analysis. If meta-analysis is to be used to inform decision-making, a reliable way to adjust pooled estimates for potential funnel plot asymmetry is required.</p> <p>Methods</p> <p>A comprehensive simulation study is presented to assess the performance of different adjustment methods including the novel application of several regression-based methods (which are commonly applied to detect publication bias rather than adjust for it) and the popular Trim & Fill algorithm. Meta-analyses with binary outcomes, analysed on the log odds ratio scale, were simulated by considering scenarios with and without i) publication bias and; ii) heterogeneity. Publication bias was induced through two underlying mechanisms assuming the probability of publication depends on i) the study effect size; or ii) the p-value.</p> <p>Results</p> <p>The performance of all methods tended to worsen as unexplained heterogeneity increased and the number of studies in the meta-analysis decreased. Applying the methods conditional on an initial test for the presence of funnel plot asymmetry generally provided poorer performance than the unconditional use of the adjustment method. Several of the regression based methods consistently outperformed the Trim & Fill estimators.</p> <p>Conclusion</p> <p>Regression-based adjustments for publication bias and other small study effects are easy to conduct and outperformed more established methods over a wide range of simulation scenarios.</p

an education review on funnel plot-based methods

Publication bias refers to a systematic deviation from the truth in the results of a meta-analysis due to the higher likelihood for published studies to be included in meta-analyses than unpublished studies. Publication bias can lead to misleading recommendations for decision and policy making. In this education review, we introduce, explain, and provide solutions to the pervasive misuses and misinterpretations of publication bias that afict evidence syntheses in sport and exercise medicine, with a focus on the commonly used funnel-plot based methods. Publication bias is more routinely assessed by visually inspecting funnel plot asymmetry, although it has been consistently deemed unreliable, leading to the development of statistical tests to assess publication bias. However, most statistical tests of publication bias (i) cannot rule out alternative explanations for funnel plot asymmetry (e.g., between-study heterogeneity, choice of metric, chance) and (ii) are grossly underpowered, even when using an arbitrary minimum threshold of ten or more studies. We performed a cross-sectional meta-research investigation of how publication bias was assessed in systematic reviews with meta-analyses published in the top two sport and exercise medicine journals throughout 2021. This analysis highlights that publication bias is frequently misused and misinterpreted, even in top tier journals. Because of conceptual and methodological problems when assessing and interpreting publication bias, preventive strategies (e.g., pre-registration, registered reports, disclosing protocol deviations, and reporting all study fndings regardless of direction or magnitude) ofer the best and most efcient solution to mitigate the misuse and misinterpretation of publication bias. Because true publication bias is very difcult to determine, we recommend that future publications use the term “risk of publication bias”.9E1A-F9DD-3EB8 | Filipe Manuel ClementeN/

A meta-analysis of gabapentin and multimodal analgesics

Multimodal analgesia has been proposed as a useful strategy to reduce postoperative pain while decreasing opioid consumption and thus opioid adverse events. Gabapentin is one such agent although previous results have been heterogeneous. This thesis aimed to review randomised controlled trials of gabapentin for reducing pain, opioid adverse effects and the haemodynamic response to intubation while attempted to predict clinical effectiveness from these trials using meta-regression. Extending this principle, we evaluated other multimodal analgesic agents to identify whether heterogeneity could be explained by various clinical and methodological covariates. Our gabapentin review included 133 randomised controlled trials and demonstrated its efficacy in reducing pain scores, opioid consumption and opioid adverse events such as nausea, vomiting and pruritus. However, gabapentin increased the risk of sedation. Gabapentin was effective at reducing the haemodynamic response to intubation in 29 randomised controlled trials although trials failed to report on clinically relevant outcomes. Gabapentin exhibited no pre-emptive analgesic effect in 4 randomised controlled trials. There was evidence of considerable statistical heterogeneity on meta-analysis of gabapentin for pain scores and 24-hour morphine consumption. Meta-regression analysis showed however that baseline risk predicted the majority of the heterogeneity between studies. Extending this approach to other multimodal analgesics from 344 randomised controlled trials; we demonstrated this was true for analgesic agents in general. In addition to baseline risk, methodological limitations, especially inadequate allocation concealment, explained some of the residual heterogeneity. There was evidence of funnel plot asymmetry for most analgesic agents, suggesting publication bias. However, this may be a product of trials with higher baseline risk having larger standard errors, rather than true publication bias. Indeed, when we simulated meta-analyses with no publication bias, with both effect size and standard deviations dependent on baseline risk, funnel plot asymmetry was still evident (p<0.001). Therefore, conventional funnel plots may be an unsuitable method of detecting publication bias where baseline risk predicts between-study heterogeneity. We present an alternative method using meta-regression residuals that corrects funnel plot asymmetry in the presence of no publication bias. Finally, due to concerns that methodological limitations exaggerated effect estimates, we used trial sequential analysis to determine whether sufficient low risk of bias evidence exists to reject type I and type II errors in the analyses of analgesic adjuncts. We demonstrated there is currently insufficient evidence from low risk of bias trials to be confident of the efficacy of the majority of analgesic adjuncts

A meta-analysis of gabapentin and multimodal analgesics

Multimodal analgesia has been proposed as a useful strategy to reduce postoperative pain while decreasing opioid consumption and thus opioid adverse events. Gabapentin is one such agent although previous results have been heterogeneous. This thesis aimed to review randomised controlled trials of gabapentin for reducing pain, opioid adverse effects and the haemodynamic response to intubation while attempted to predict clinical effectiveness from these trials using meta-regression. Extending this principle, we evaluated other multimodal analgesic agents to identify whether heterogeneity could be explained by various clinical and methodological covariates. Our gabapentin review included 133 randomised controlled trials and demonstrated its efficacy in reducing pain scores, opioid consumption and opioid adverse events such as nausea, vomiting and pruritus. However, gabapentin increased the risk of sedation. Gabapentin was effective at reducing the haemodynamic response to intubation in 29 randomised controlled trials although trials failed to report on clinically relevant outcomes. Gabapentin exhibited no pre-emptive analgesic effect in 4 randomised controlled trials. There was evidence of considerable statistical heterogeneity on meta-analysis of gabapentin for pain scores and 24-hour morphine consumption. Meta-regression analysis showed however that baseline risk predicted the majority of the heterogeneity between studies. Extending this approach to other multimodal analgesics from 344 randomised controlled trials; we demonstrated this was true for analgesic agents in general. In addition to baseline risk, methodological limitations, especially inadequate allocation concealment, explained some of the residual heterogeneity. There was evidence of funnel plot asymmetry for most analgesic agents, suggesting publication bias. However, this may be a product of trials with higher baseline risk having larger standard errors, rather than true publication bias. Indeed, when we simulated meta-analyses with no publication bias, with both effect size and standard deviations dependent on baseline risk, funnel plot asymmetry was still evident (p<0.001). Therefore, conventional funnel plots may be an unsuitable method of detecting publication bias where baseline risk predicts between-study heterogeneity. We present an alternative method using meta-regression residuals that corrects funnel plot asymmetry in the presence of no publication bias. Finally, due to concerns that methodological limitations exaggerated effect estimates, we used trial sequential analysis to determine whether sufficient low risk of bias evidence exists to reject type I and type II errors in the analyses of analgesic adjuncts. We demonstrated there is currently insufficient evidence from low risk of bias trials to be confident of the efficacy of the majority of analgesic adjuncts

Testing for Publication Bias in Diagnostic Meta-Analysis: A Simulation Study

The present study investigates the performance of several statistical tests to detect publication bias in diagnostic meta-analysis by means of simulation. While bivariate models should be used to pool data from primary studies in diagnostic meta-analysis, univariate measures of diagnostic accuracy are preferable for the purpose of detecting publication bias. In contrast to earlier research, which focused solely on the diagnostic odds ratio or its logarithm ($\ln\omega$), the tests are combined with four different univariate measures of diagnostic accuracy. For each combination of test and univariate measure, both type I error rate and statistical power are examined under diverse conditions. The results indicate that tests based on linear regression or rank correlation cannot be recommended in diagnostic meta-analysis, because type I error rates are either inflated or power is too low, irrespective of the applied univariate measure. In contrast, the combination of trim and fill and $\ln\omega$ has non-inflated or only slightly inflated type I error rates and medium to high power, even under extreme circumstances (at least when the number of studies per meta-analysis is large enough). Therefore, we recommend the application of trim and fill combined with $\ln\omega$ to detect funnel plot asymmetry in diagnostic meta-analysis. Please cite this paper as published in Statistics in Medicine (https://doi.org/10.1002/sim.6177).Comment: arXiv admin note: text overlap with arXiv:2002.04775 by other author

Prevalence and risk factors of malnutrition in patients with pulmonary tuberculosis: a systematic review and meta-analysis

BackgroundMalnutrition is prevalent in patients with pulmonary tuberculosis (PTB) and is associated with a poor prognosis.ObjectiveThis study aims to assess the prevalence and risk factors of malnutrition in patients with PTB.MethodsStudies related to the prevalence and risk factors of malnutrition in patients with PTB were searched through PubMed, Embase, Web of Science, and Cochrane Library databases from January 1990 to August 2022, and two researchers screened the literature, evaluated the quality, and extracted data independently. A random-effects model was used to pool the effect sizes and 95% confidence intervals. Subgroup analysis, meta-regression analysis, and sensitivity analysis were further performed to identify sources of heterogeneity and evaluate the stability of the results. Publication bias was assessed by Doi plot, Luis Furuya-Kanamori (LFK) asymmetry index, funnel plot, and Egger's tests.ResultsA total of 53 studies involving 48, 598 participants were identified in this study. The prevalence of malnutrition was 48.0% (95% CI, 40.9–55.2%). Subgroup analysis revealed that malnutrition was more common among male gender (52.3%), bacterial positivity (55.9%), family size over 4 (54.5%), drug resistance (44.1%), residing in rural areas (51.2%), HIV infection (51.5%), Asian (51.5%), and African (54.5%) background. The prevalence of mild, moderate, and severe malnutrition was 21.4%, 14.0%, and 29.4%, respectively. Bacterial positivity (OR = 2.08, 95% CI 1.26–3.41), low income (OR = 1.44, 95% CI 1.11–1.86), and residing in rural areas (OR = 1.51, 95% CI 1.20–1.89) were risk factors of malnutrition in patients with PTB. However, male (OR = 1.04, 95% CI 0.85–1.26) and drinking (OR = 1.17, 95% CI 0.81–1.69) were not risk factors for malnutrition in patients with PTB. Due to the instability of sensitivity analysis, HIV infection, age, family size, smoking, and pulmonary cavity need to be reevaluated. Meta-regression suggested that sample size was a source of heterogeneity of prevalence. The Doi plot and LFK asymmetry index (LFK = 3.87) indicated the presence of publication bias for prevalence, and the funnel plot and Egger's test showed no publication bias for risk factors.ConclusionThis meta-analysis indicated that malnutrition was prevalent in patients with PTB, and bacterial positivity, low income, and those residing in rural areas were risk factors for malnutrition. Therefore, clinical workers should pay attention to screening the nutritional status of patients with PTB and identifying the risk factors to reduce the incidence of malnutrition and provide nutritional interventions early to improve the prognosis in patients with PTB

Aid and Growth What Meta-Analysis Reveals

Some recent literature in the meta-analysis category where results from a range of studies are brought together throws doubt on the ability of foreign aid to foster economic growth and development. This paper assesses what meta-analysis has to say about the effectiveness of foreign aid in terms of the growth impact. We re-examine key hypotheses, and find that the effect of aid on growth is positive and statistically significant. This significant effect is genuine, and not an artefact of publication selection. We also show why our results differ from those published elsewhere.aid and growth, meta-analysis, heterogeneity and publication bias