To find efficient screening methods for high dimensional linear regression
models, this paper studies the relationship between model fitting and screening
performance. Under a sparsity assumption, we show that a subset that includes
the true submodel always yields smaller residual sum of squares (i.e., has
better model fitting) than all that do not in a general asymptotic setting.
This indicates that, for screening important variables, we could follow a
"better fitting, better screening" rule, i.e., pick a "better" subset that has
better model fitting. To seek such a better subset, we consider the
optimization problem associated with best subset regression. An EM algorithm,
called orthogonalizing subset screening, and its accelerating version are
proposed for searching for the best subset. Although the two algorithms cannot
guarantee that a subset they yield is the best, their monotonicity property
makes the subset have better model fitting than initial subsets generated by
popular screening methods, and thus the subset can have better screening
performance asymptotically. Simulation results show that our methods are very
competitive in high dimensional variable screening even for finite sample
sizes.Comment: 24 pages, 1 figur