Skip to main content
Article thumbnail
Location of Repository

DRAFT COPY – NOT FOR CITATION OR QUOTATION WITHOUT THE AUTHORS’

By Lawrence E. Raffalovich, Glenn D. Deane, David Armstrong and Hui-shien Tsao

Abstract

Model Selection Procedures in Social Research: Monte-Carlo Simulation Results Model selection strategies play an important, if not explicit, role in quantitative research. The inferential properties of these strategies are largely unknown, so there is little basis for recommending (or avoiding) any particular set of strategies. In this paper we evaluate several commonly used model selection procedures (BIC, adjusted R 2, Mallow’s C p, and stepwise regression) using monte-carlo simulation of model selection when the true data-generating processes are known. We find that the ability of these selection procedures to include important variables and exclude irrelevant variables increases with the size of the sample and decreases with the amount of noise in the model. None of the model selection procedures do well in small samples unless the true DGP is almost entirely deterministic; so data mining in small samples should be avoided entirely. In large samples, BIC is better than the other procedures at correctly identifying most of the generating processes we simulated, and Stepwise does even better with some models and almost as well with others. In the absence of strong theory, both BIC and Stepwise appear to be reasonable model selection strategies i

Topics: Model selection strategies – selecting the set of variables to be included
Year: 2001
OAI identifier: oai:CiteSeerX.psu:10.1.1.133.9832
Provided by: CiteSeerX
Download PDF:
Sorry, we are unable to provide the full text but you may find it at the following location(s):
  • http://citeseerx.ist.psu.edu/v... (external link)
  • http://www.albany.edu/csda/200... (external link)
  • Suggested articles


    To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.