75 research outputs found
The fragility of meta-regression models in observational research
Many meta-regression analyses that synthesize estimates from primary studies have now been published in economics. Meta-regression models attempt to infer the presence of genuine empirical effects even if the authors of primary studies select statistically significant and theory-confirming estimates for publication. Meta-regression models were originally developed for the synthesis of experimental research where randomization ensures unbiased and consistent estimation of the effect of interest. Most economics research is, however, observational and authors of primary studies can search across different regression specifications for statistically significant and theory-confirming estimates. Each regression specification may possibly suffer from biases such as omitted-variable biases that result in biased and inconsistent estimation of the effect of interest. We show that if the authors of primary studies search for statistically significant and theory-confirming estimates, meta-regression models tend to systematically make false-positive findings of genuine empirical effects. The ubiquity of such search processes for specific results may limit the applicability of meta-regression models in identifying genuine empirical effects in economics
Identifying genuine effects in observational research by means of meta-regressions
Meta-regression models are increasingly utilized to integrate empirical results across studies while controlling for the potential threats of data-mining and publication bias. We propose extended meta-regression models and evaluate their performance in identifying genuine empirical effects by means of a comprehensive simulation study for various scenarios that are prevalent in empirical economics. We can show that the meta-regression models here proposed systematically outperform the prior gold standard of meta-regression analysis of regression coefficients. Most meta-regression models are robust to the presence of publication bias, but data-mining bias leads to seriously inflated type I errors and has to be addressed explicitly
Can declining energy intensity mitigate climate change? Decomposition and meta-regression results
Drawing on the Kaya identity, we assess the role of the main driver of the decline in carbon intensity, namely the (economic) energy intensity. Using meta-significance testing for a sample of 44 studies, dealing with the causality between energy and GDP, we find that both variables are strongly coupled. Hence, after having exhausted energy savings from nonrecurring structural changes, the economic energy intensity may soon converge than being arbitrarily reducible. We suggest, therefore, not to rely on further reductions of economic energy intensity but rather to invest in the reduction of the carbon intensity of energy to mitigate climate change
Lag length selection and p-hacking in Granger causality testing: prevalence and performance of meta-regression models
The academic system incentivizes p-hacking, where researchers select estimates and statistics with statistically significant p-values for publication. We analyze the complete process of Granger causality testing including p-hacking using Monte Carlo simulations. If the degrees of freedom of the underlying vector autoregressive model are small to moderate, information criteria tend to overfit the lag length and overfitted vector autoregressive models tend to result in false-positive findings of Granger causality. Researchers may p-hack Granger causality tests by estimating multiple vector autoregressive models with different lag lengths and then selecting only those models that reject the null of Granger non-causality for presentation in the final publication. We show that overfitted lag lengths and the corresponding false-positive findings of Granger causality can frequently occur in research designs that are prevalent in empirical macroeconomics. We demonstrate that meta-regression models can control for spuriously significant Granger causality tests due to overfitted lag lengths. Finally, we find evidence that false-positive findings of Granger causality may be prevalent in the large literature that tests for Granger causality between energy use and economic output, while we do not find evidence for a genuine relation between these variables as tested in the literature
Is There Really Granger Causality Between Energy Use and Output?
We carry out a meta-analysis of the very large literature on Granger causality tests between energy
use and economic output to determine if there is a genuine effect in this literature or whether the
large number of apparently significant results is due to publication and misspecification bias. Our
model extends the standard meta-regression model for detecting genuine effects using the statistical
power trace in the presence of publication biases by controlling for the tendency to over-fit vector
auto regression models in small samples. These over-fitted models have inflated type 1 errors. We find that models that include energy prices as a control variable find a genuine effect from output to energy use in the long-run. A genuine causal effect also seems apparent from energy to output when employment is controlled for and the Johansen procedure is used
Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics
Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine
Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics
Publication selection bias undermines the systematic accumulation of
evidence. To assess the extent of this problem, we survey over 68,000
meta-analyses containing over 700,000 effect size estimates from medicine
(67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563),
and economics (327/91,421). Our results indicate that meta-analyses in
economics are the most severely contaminated by publication selection bias,
closely followed by meta-analyses in environmental sciences and psychology,
whereas meta-analyses in medicine are contaminated the least. After adjusting
for publication selection bias, the median probability of the presence of an
effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in
psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to
29.7% in medicine. The median absolute effect sizes (in terms of standardized
mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d =
0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental
sciences, and from d = 0.24 to d = 0.13 in medicine
Practitioner’s Section: Integrated Resource Efficiency Analysis for Reducing Climate Impacts in the Chemical Industry
Reducing greenhouse gas emissions of the material-intensive chemical industry requires an integrated analysis and optimization of the complex production systems including raw material and energy use, resulting costs and environmental and climate impacts. To meet this challenge, the research project InReff (Integrated Resource Efficiency Analysis for Reducing Climate Impacts in the Chemical Industry) has been established. It aims at the development of an IT-supported modeling and evaluation framework which is able to comprehensively address issues of resource efficiency and climate change within the chemical industry, e.g. the minimization of material and energy intensity and consequently greenhouse gas emissions, without compromising on production performance. The paper presents background information on resource efficiency and the research project, an ideal-typical decision model for resource efficiency analysis, the conceptual approach for an IT-based integration platform as well as the case study design at the industrial project partners’ sites. These first results are linked to future activities and further research questions are highlighted in the concluding section
Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics
Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine
Prolactinomas, Cushing's disease and acromegaly: debating the role of medical therapy for secretory pituitary adenomas
Pituitary adenomas are associated with a variety of clinical manifestations resulting from excessive hormone secretion and tumor mass effects, and require a multidisciplinary management approach. This article discusses the treatment modalities for the management of patients with a prolactinoma, Cushing's disease and acromegaly, and summarizes the options for medical therapy in these patients
- …