75 research outputs found

    The fragility of meta-regression models in observational research

    Full text link
    Many meta-regression analyses that synthesize estimates from primary studies have now been published in economics. Meta-regression models attempt to infer the presence of genuine empirical effects even if the authors of primary studies select statistically significant and theory-confirming estimates for publication. Meta-regression models were originally developed for the synthesis of experimental research where randomization ensures unbiased and consistent estimation of the effect of interest. Most economics research is, however, observational and authors of primary studies can search across different regression specifications for statistically significant and theory-confirming estimates. Each regression specification may possibly suffer from biases such as omitted-variable biases that result in biased and inconsistent estimation of the effect of interest. We show that if the authors of primary studies search for statistically significant and theory-confirming estimates, meta-regression models tend to systematically make false-positive findings of genuine empirical effects. The ubiquity of such search processes for specific results may limit the applicability of meta-regression models in identifying genuine empirical effects in economics

    Identifying genuine effects in observational research by means of meta-regressions

    Full text link
    Meta-regression models are increasingly utilized to integrate empirical results across studies while controlling for the potential threats of data-mining and publication bias. We propose extended meta-regression models and evaluate their performance in identifying genuine empirical effects by means of a comprehensive simulation study for various scenarios that are prevalent in empirical economics. We can show that the meta-regression models here proposed systematically outperform the prior gold standard of meta-regression analysis of regression coefficients. Most meta-regression models are robust to the presence of publication bias, but data-mining bias leads to seriously inflated type I errors and has to be addressed explicitly

    Can declining energy intensity mitigate climate change? Decomposition and meta-regression results

    Full text link
    Drawing on the Kaya identity, we assess the role of the main driver of the decline in carbon intensity, namely the (economic) energy intensity. Using meta-significance testing for a sample of 44 studies, dealing with the causality between energy and GDP, we find that both variables are strongly coupled. Hence, after having exhausted energy savings from nonrecurring structural changes, the economic energy intensity may soon converge than being arbitrarily reducible. We suggest, therefore, not to rely on further reductions of economic energy intensity but rather to invest in the reduction of the carbon intensity of energy to mitigate climate change

    Lag length selection and p-hacking in Granger causality testing: prevalence and performance of meta-regression models

    Get PDF
    The academic system incentivizes p-hacking, where researchers select estimates and statistics with statistically significant p-values for publication. We analyze the complete process of Granger causality testing including p-hacking using Monte Carlo simulations. If the degrees of freedom of the underlying vector autoregressive model are small to moderate, information criteria tend to overfit the lag length and overfitted vector autoregressive models tend to result in false-positive findings of Granger causality. Researchers may p-hack Granger causality tests by estimating multiple vector autoregressive models with different lag lengths and then selecting only those models that reject the null of Granger non-causality for presentation in the final publication. We show that overfitted lag lengths and the corresponding false-positive findings of Granger causality can frequently occur in research designs that are prevalent in empirical macroeconomics. We demonstrate that meta-regression models can control for spuriously significant Granger causality tests due to overfitted lag lengths. Finally, we find evidence that false-positive findings of Granger causality may be prevalent in the large literature that tests for Granger causality between energy use and economic output, while we do not find evidence for a genuine relation between these variables as tested in the literature

    Is There Really Granger Causality Between Energy Use and Output?

    Get PDF
    We carry out a meta-analysis of the very large literature on Granger causality tests between energy use and economic output to determine if there is a genuine effect in this literature or whether the large number of apparently significant results is due to publication and misspecification bias. Our model extends the standard meta-regression model for detecting genuine effects using the statistical power trace in the presence of publication biases by controlling for the tendency to over-fit vector auto regression models in small samples. These over-fitted models have inflated type 1 errors. We find that models that include energy prices as a control variable find a genuine effect from output to energy use in the long-run. A genuine causal effect also seems apparent from energy to output when employment is controlled for and the Johansen procedure is used

    Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics

    Get PDF
    Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine

    Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics

    Get PDF
    Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine

    Practitioner’s Section: Integrated Resource Efficiency Analysis for Reducing Climate Impacts in the Chemical Industry

    Full text link
    Reducing greenhouse gas emissions of the material-intensive chemical industry requires an integrated analysis and optimization of the complex production systems including raw material and energy use, resulting costs and environmental and climate impacts. To meet this challenge, the research project InReff (Integrated Resource Efficiency Analysis for Reducing Climate Impacts in the Chemical Industry) has been established. It aims at the development of an IT-supported modeling and evaluation framework which is able to comprehensively address issues of resource efficiency and climate change within the chemical industry, e.g. the minimization of material and energy intensity and consequently greenhouse gas emissions, without compromising on production performance. The paper presents background information on resource efficiency and the research project, an ideal-typical decision model for resource efficiency analysis, the conceptual approach for an IT-based integration platform as well as the case study design at the industrial project partners’ sites. These first results are linked to future activities and further research questions are highlighted in the concluding section

    Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics

    Get PDF
    Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine

    Prolactinomas, Cushing's disease and acromegaly: debating the role of medical therapy for secretory pituitary adenomas

    Get PDF
    Pituitary adenomas are associated with a variety of clinical manifestations resulting from excessive hormone secretion and tumor mass effects, and require a multidisciplinary management approach. This article discusses the treatment modalities for the management of patients with a prolactinoma, Cushing's disease and acromegaly, and summarizes the options for medical therapy in these patients
    • …
    corecore