Comparison of Variable Selection Methods


Use of classic variable selection methods in public health research is quite common. Many criteria, and various strategies for applying them, now exist including forward selection, backward elimination, stepwise selection, best-subset selection and so on, but all suffer from similar drawbacks. Chief among them is a failure to account for the uncertainty contained in the model selection process. Ignoring model uncertainty can cause several serious problems. Variance estimates are generally underestimated, p-values are generally inflated, prediction ability is overestimated, and results are not reproducible in another dataset. Modern variable selection methods have become increasingly popular, especially in applications of high-dimensional or sparse data. Some of these methods were developed to address the short-comings of classic variable selection methods, such as backward elimination and stepwise selection methods. However, it remains unclear how modern variable selection methods behave in a classical, meaning non-high-dimensional, setting. A simulation study investigates the estimation, predictive performance and variable selection capabilities of three representative modern variable selection methods: Bayesian model averaging (BMA), stochastic search variable selection (SSVS), and the adaptive lasso. These three methods are considered in the setting of linear regression with a single variable of interest which is always included in the model. A second simulation study compares BMA to classical variable selection methods, including backward elimination, two-stage method, and change-in-effect method in the setting of logistic regression. Additionally, the data generated in both simulation studies closely mimic a real study and reflect a realistic correlation structure between potential covariates. Sample sizes ranging from 150 to 20000 are investigated. BMA is demonstrated in an example building a predictive model using data from the China Health and Nutrition Survey.Doctor of Public Healt

Similar works

Full text


Carolina Digital Repository

Provided a free PDF
Last time updated on 9/3/2019View original full text link

This paper was published in Carolina Digital Repository.

Having an issue?

Is data on this page outdated, violates copyrights or anything else? Report the problem now and we will take corresponding actions after reviewing your request.