Many high dimensional classification techniques have been proposed in the
literature based on sparse linear discriminant analysis (LDA). To efficiently
use them, sparsity of linear classifiers is a prerequisite. However, this might
not be readily available in many applications, and rotations of data are
required to create the needed sparsity. In this paper, we propose a family of
rotations to create the required sparsity. The basic idea is to use the
principal components of the sample covariance matrix of the pooled samples and
its variants to rotate the data first and to then apply an existing high
dimensional classifier. This rotate-and-solve procedure can be combined with
any existing classifiers, and is robust against the sparsity level of the true
model. We show that these rotations do create the sparsity needed for high
dimensional classifications and provide theoretical understanding why such a
rotation works empirically. The effectiveness of the proposed method is
demonstrated by a number of simulated and real data examples, and the
improvements of our method over some popular high dimensional classification
rules are clearly shown.Comment: 30 pages and 9 figures. This paper has been accepted by Journal of
the Royal Statistical Society: Series B (Statistical Methodology). The first
two versions of this paper were uploaded to Bin Dong's web site under the
title "A Rotate-and-Solve Procedure for Classification" in 2013 May and 2014
January. This version may be slightly different from the published versio
