Quadratic regression (QR) models naturally extend linear models by
considering interaction effects between the covariates. To conduct model
selection in QR, it is important to maintain the hierarchical model structure
between main effects and interaction effects. Existing regularization methods
generally achieve this goal by solving complex optimization problems, which
usually demands high computational cost and hence are not feasible for high
dimensional data. This paper focuses on scalable regularization methods for
model selection in high dimensional QR. We first consider two-stage
regularization methods and establish theoretical properties of the two-stage
LASSO. Then, a new regularization method, called Regularization Algorithm under
Marginality Principle (RAMP), is proposed to compute a hierarchy-preserving
regularization solution path efficiently. Both methods are further extended to
solve generalized QR models. Numerical results are also shown to demonstrate
performance of the methods.Comment: 37 pages, 1 figure with supplementary materia