Professors Candès and Tao are to be congratulated for their innovative and valuable contribution to high-dimensional sparse recovery and model selection. The analysis of vast data sets now commonly arising in scientific investigations poses many statistical challenges not present in smaller scale studies. Many of these data sets exhibit sparsity where most of the data corresponds to noise and only a small fraction is of interest. The needs of this research have excited much interest in the statistical community. In particular, high-dimensional model selection has attracted much recent attention and has become a central topic in statistics. The main difficulty of such a problem comes from collinearity between the predictor variables. It is clear from the geometric point of view that the collinearity increases as the dimensionality grows

Cai, T. Tony

Lv, Jinchi

English

ScholarlyCommons@Penn

University of PennsylvaniaScholarlyCommonsStatistics Papers Wharton Faculty Research2007Discussion: The Dantzig Selector: StatisticalEstimation When p is Much Larger Than nT. Tony CaiUniversity of PennsylvaniaJinchi LvPrinceton UniversityFollow this and additional works at: http://repository.upenn.edu/statistics_papersPart of the Statistics and Probability CommonsThis paper is posted at ScholarlyCommons. http://repository.upenn.edu/statistics_papers/305For more information, please contact repository@pobox.upenn.edu.Recommended CitationCai, T., & Lv, J. (2007). Discussion: The Dantzig Selector: Statistical Estimation When p is Much Larger Than n. The Annals ofStatistics, 35 (6), 2365-2369. http://dx.doi.org/10.1214/009053607000000442Discussion: The Dantzig Selector: Statistical Estimation When p is MuchLarger Than nAbstractProfessors Candès and Tao are to be congratulated for their innovative and valuable contribution to high-dimensional sparse recovery and model selection. The analysis of vast data sets now commonly arising inscientific investigations poses many statistical challenges not present in smaller scale studies. Many of thesedata sets exhibit sparsity where most of the data corresponds to noise and only a small fraction is of interest.The needs of this research have excited much interest in the statistical community. In particular, high-dimensional model selection has attracted much recent attention and has become a central topic in statistics.The main difficulty of such a problem comes from collinearity between the predictor variables. It is clear fromthe geometric point of view that the collinearity increases as the dimensionality grows.DisciplinesStatistics and ProbabilityThis journal article is available at ScholarlyCommons: http://repository.upenn.edu/statistics_papers/305The Annals of Statistics2007, Vol. 35, No. 6, 2365–2369DOI: 10.1214/009053607000000442Main article DOI: 10.1214/009053606000001523© Institute of Mathematical Statistics, 2007DISCUSSION: THE DANTZIG SELECTOR: STATISTICALESTIMATION WHEN p IS MUCH LARGER THAN nBY T. TONY CAI1 AND JINCHI LVUniversity of Pennsylvania and Princeton UniversityProfessors Candès and Tao are to be congratulated for their innovative and valu-able contribution to high-dimensional sparse recovery and model selection. Theanalysis of vast data sets now commonly arising in scientific investigations posesmany statistical challenges not present in smaller scale studies. Many of these datasets exhibit sparsity where most of the data corresponds to noise and only a smallfraction is of interest. The needs of this research have excited much interest inthe statistical community. In particular, high-dimensional model selection has at-tracted much recent attention and has become a central topic in statistics. The maindifficulty of such a problem comes from collinearity between the predictor vari-ables. It is clear from the geometric point of view that the collinearity increases asthe dimensionality grows.A common approach taken in the statistics literature is the penalized likeli-hood, for example, Lasso (Tibshirani [11]) and adaptive Lasso (Zou [12]), SCAD(Fan and Li [7] and Fan and Peng [9]) and nonnegative garrote (Breiman [1]).Commonly used algorithms include LARS (Efron, Hastie, Johnstone and Tibshi-rani [6]), LQA (Fan and Li [7]) and MM (Hunter and Li [10]). In the presentpaper, Candès and Tao take a new approach, called the Dantzig selector, whichuses 1-minimization with regularization on the residuals. One promising fact isthat the Dantzig selector solves a linear program, usually faster than the existingmethods. In addition, the authors establish that, under the Uniform UncertaintyPrinciple (UUP), with large probability the Dantzig selector mimics the risk of theoracle estimator up to a logarithmic factor logp, where p denotes the number ofvariables.We appreciate the opportunity to comment on several aspects of this article.Our discussion here will focus on four issues: (1) connection to sparse signal re-covery in the noiseless case; (2) the UUP condition and identifiability of the model;(3) computation and model selection; (4) minimax rate.1. Sparse signal recovery. The “large p, small n” regression problem consid-ered in this paper can be viewed as a generalization of the classical linear algebraproblem in which one wishes to solve the linear equationy = Xβ,(1)Received January 2007.1Supported in part by NSF Grant DMS-06-04954.23652366 T. T. CAI AND J. LVwhere X is a given n×p matrix and y is a vector in Rn. Because p > n, the linearequation (1) is underdetermined and there are an infinite number of solutions to theequation. The goal is to find the “sparsest” solution under certain regularity condi-tions. In this noiseless setting, the Dantzig selector reduces to an 1-minimizationover the space of all representations of the signal:Minimize ‖β‖1 subject to y = Xβ.(2)This idea of finding a sparse representation using 1-minimization has been usedin Donoho and Elad [4]. The authors have also used this approach in their earlierwork (Candès and Tao [2, 3]) on recovering sparse signals in the noiseless case.When adding a Gaussian noise term ε to (1), the linear algebra problem becomesa nonstandard linear regression problem because p  n. In the classical linearregression problem when p ≤ n the least squares estimator is the solution to thenormal equationXT y = XT Xβ.(3)The constraint ‖XT r‖∞ ≤ λpσ (which is the same as ‖XT y − XT Xβ‖∞ ≤ λpσ )in the convex program (DS) in this paper can be viewed as a relaxation of thenormal equation (3). And similarly to the noiseless case 1-minimization leads tothe “sparsest” solution over the space of all feasible solutions.The authors suggest using λp = √2 logp, which is equal to √2 logn in theorthogonal design setting. In this setting, the oracle properties of the Dantzig se-lector are in line with those shrinkage results in Donoho and Johnstone [5] whichare shown to be optimal in the minimax sense. When p  n, it might be possiblethat the regularization factor λp = √2 logp in the Dantzig selector overshrinks thep-vector β and underestimates the nonzero coordinates. It would be interesting tofind the “optimal” regularization factor.2. The UUP condition and identifiability of the model. A nice idea in thispaper is the use of the UUP condition. The UUP condition has also been usedby the authors in their earlier work (Candès and Tao [2, 3]) in the noiseless set-ting. The UUP condition roughly says that for any small set of predictors, thesen-vectors are nearly orthogonal to each other. The authors give an interpretation ofthe UUP condition in terms of the model identifiability and have established ora-cle inequalities for the Dantzig selector under the UUP using geometric arguments.However, we still have some concerns about this condition.First, it is computationally unrealistic to verify whether the UUP condition holdsfor a given design matrix X when p is large and the number of signals S is nottoo small. Note that computing the S-restricted isometry constant δS of the designmatrix X is over the space of all S-subsets of {1, . . . , p}, which is of cardinality (pS).This combinatorial complexity makes it infeasible to check the UUP condition forreasonable values of p and S, say p = 1000 and S = 5. So it is interesting to lookDISCUSSION 2367FIG. 1. Distribution of the maximum absolute sample correlation when n = 60, pn = 1000 (solidcurve) and n = 60, pn = 5000 (dashed curve).for other checkable conditions that are compatible with the model identifiabilitycondition in a similar or new setup.Second, as discussed in Fan and Lv [8], the UUP is hard to satisfy when the di-mension p grows rapidly with the sample size n. This is essentially due to signifi-cant sample correlation, that is, strong collinearity, between the predictor variablesin the high-dimensional setting of p  n. For instance, we take p independentpredictors X1, . . . ,Xp from the standard Gaussian distribution and compute themaximum of the pairwise absolute sample correlations from an n × p design ma-trix X. Figure 1, which is extracted from Fan and Lv [8], shows the distributionsof the maximum correlation with n = 60, p = 1000 and n = 60, p = 5000, re-spectively. The maximum sample correlation between predictors can be very largeand close to 1. Moreover, the maximum of the first canonical correlations betweentwo groups of predictors, for example, three predictors in one group and five inanother, can be much larger since there are(p3)(p−35) = O(p8) ( n) choices inthis example.3. Computation and model selection. Due to its nature of involving lin-ear programming, the Dantzig selector can be solved quickly and efficiently bya primal–dual interior point algorithm when the dimension is not ultrahigh, forexample, in the thousands. It is usually faster to implement than other existingmethods such as Lasso. However, in problems of large or ultralarge scale the com-putational cost of implementing linear programs is still a potential hurdle. For ex-ample, in the analysis of microarray gene expression or proteomics data, it is com-mon to have dimension of tens of thousands and implementing linear programs insuch settings can still be computationally challenging. Therefore, it is interestingand necessary to study ultrahigh-dimensional model selection. Recently, Fan andLv [8] introduced a procedure for screening variables via iteratively thresholded2368 T. T. CAI AND J. LVridge regression and proposed a new method of dimension reduction called SureIndependence Screening, for ultrahigh-dimensional feature space. The method canimprove the estimation accuracy significantly while speeding up variable selectiondrastically.In our experience, we have found that the algorithm solving the Dantzig selec-tor is sensitive to the initial value. Trivial initial values such as constant vectorsusually do not work well. The generalized least squares estimator (XT X)−XT ycan be used as an initial value. However, the solution is usually nonsparse in ourexperience.As mentioned earlier, the regularization factor λp = √2 logp in the Dantzigselector leads to relatively large bias in estimating the sparse regression coefficientvector β . To reduce the bias, the authors suggest a two-stage procedure calledthe Gauss–Dantzig selector which uses the original Dantzig selector for variableselection and then runs ordinary least squares on the selected variables. It wouldbe interesting to know the theoretical and numerical properties of this and othervariations.4. Minimax rate. It is appealing that the Dantzig selector achieves within alogarithmic factor logp of the ideal risk. We are curious about the optimality ofthis factor. It is unclear at this point whether the minimax factor should be logp orlogn or some other quantity. When p is polynomial in the sample size n, logp andlogn are of the same order and the difference between the two is not significant.However, when p is exponential in n, say p = ena for some a > 0, then logp = nabecomes large, much larger than logn. It is of theoretical and practical interest tostudy the minimax behavior of the problem. If the minimax rate is logp, then theDantzig selector is a rate optimal minimax procedure. Otherwise, it is interestingto construct a procedure that can attain the minimax rate.5. Concluding remarks. 1-regularization in terms of linear programmingprovides a new framework for model selection and is proven effective in solvinghigh-dimensional problems. The Dantzig selector provides us new insight on high-dimensional model selection. Clearly, there is much work ahead of us. This paperopens a new area and will definitely stimulate new ideas in the future. We thankthe authors for their clear and imaginative work.REFERENCES[1] BREIMAN, L. (1995). Better subset regression using the nonnegative garrote. Technometrics37 373–384. MR1365720[2] CANDÈS, E. J. and TAO, T. (2005). Decoding by linear programming. IEEE Trans. Inform.Theory 51 4203–4215. MR2243152[3] CANDÈS, E. J. and TAO, T. (2006). Near-optimal signal recovery from random projections:Universal encoding strategies? IEEE Trans. Inform. Theory 52 5406–5425. MR2300700DISCUSSION 2369[4] DONOHO, D. L. and ELAD, M. (2003). Optimally sparse representation in general (nonorthog-onal) dictionaries via l1 minimization. Proc. Natl. Acad. Sci. U.S.A. 100 2197–2202.MR1963681[5] DONOHO, D. L. and JOHNSTONE, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage.Biometrika 81 425–455. MR1311089[6] EFRON, B., HASTIE, T., JOHNSTONE, I. and TIBSHIRANI, R. (2004). Least angle regression(with discussion). Ann. Statist. 32 407–499. MR2060166[7] FAN, J. and LI, R. (2001). Variable selection via nonconcave penalized likelihood and its oracleproperties. J. Amer. Statist. Assoc. 96 1348–1360. MR1946581[8] FAN, J. and LV, J. (2006). Sure independence screening for ultra-high dimensional featurespace. Unpublished manuscript. Available at arxiv.org/abs/math/0612857.[9] FAN, J. and PENG, H. (2004). Nonconcave penalized likelihood with diverging number ofparameters. Ann. Statist. 32 928–961. MR2065194[10] HUNTER, D. and LI, R. (2005). Variable selection using MM algorithms. Ann. Statist. 331617–1642. MR2166557[11] TIBSHIRANI, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc.Ser. B 58 267–288. MR1379242[12] ZOU, H. (2006). The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418–1429. MR2279469DEPARTMENT OF STATISTICSTHE WHARTON SCHOOLUNIVERSITY OF PENNSYLVANIAPHILADELPHIA, PENNSYLVANIA 19104USAE-MAIL: tcai@wharton.upenn.eduDEPARTMENT OF MATHEMATICSPRINCETON UNIVERSITYPRINCETON, NEW JERSEY 08544USAE-MAIL: jlv@princeton.edu

Discussion: The Dantzig Selector: Statistical Estimation When \u3cem\u3ep\u3c/em\u3e is Much Larger Than \u3cem\u3en\u3c/em\u3e

Kosmopolis

University of Pennsylvania
ScholarlyCommons
Statistics Papers Wharton Faculty Research
2007
Discussion: The Dantzig Selector: Statistical
Estimation When p is Much Larger Than n
T. Tony Cai
University of Pennsylvania
Jinchi Lv
Princeton University
Follow this and additional works at: http://repository.upenn.edu/statistics_papers
Part of the Statistics and Probability Commons
This paper is posted at ScholarlyCommons. http://repository.upenn.edu/statistics_papers/305
For more information, please contact repository@pobox.upenn.edu.
Recommended Citation
Cai, T., & Lv, J. (2007). Discussion: The Dantzig Selector: Statistical Estimation When p is Much Larger Than n. The Annals of
Statistics, 35 (6), 2365-2369. http://dx.doi.org/10.1214/009053607000000442
Discussion: The Dantzig Selector: Statistical Estimation When p is Much
Larger Than n
Abstract
Professors Candès and Tao are to be congratulated for their innovative and valuable contribution to high-
dimensional sparse recovery and model selection. The analysis of vast data sets now commonly arising in
scientific investigations poses many statistical challenges not present in smaller scale studies. Many of these
data sets exhibit sparsity where most of the data corresponds to noise and only a small fraction is of interest.
The needs of this research have excited much interest in the statistical community. In particular, high-
dimensional model selection has attracted much recent attention and has become a central topic in statistics.
The main difficulty of such a problem comes from collinearity between the predictor variables. It is clear from
the geometric point of view that the collinearity increases as the dimensionality grows.
Disciplines
Statistics and Probability
This journal article is available at ScholarlyCommons: http://repository.upenn.edu/statistics_papers/305
The Annals of Statistics
2007, Vol. 35, No. 6, 2365–2369
DOI: 10.1214/009053607000000442
Main article DOI: 10.1214/009053606000001523
© Institute of Mathematical Statistics, 2007
DISCUSSION: THE DANTZIG SELECTOR: STATISTICAL
ESTIMATION WHEN p IS MUCH LARGER THAN n
BY T. TONY CAI1 AND JINCHI LV
University of Pennsylvania and Princeton University
Professors Candès and Tao are to be congratulated for their innovative and valu-
able contribution to high-dimensional sparse recovery and model selection. The
analysis of vast data sets now commonly arising in scientific investigations poses
many statistical challenges not present in smaller scale studies. Many of these data
sets exhibit sparsity where most of the data corresponds to noise and only a small
fraction is of interest. The needs of this research have excited much interest in
the statistical community. In particular, high-dimensional model selection has at-
tracted much recent attention and has become a central topic in statistics. The main
difficulty of such a problem comes from collinearity between the predictor vari-
ables. It is clear from the geometric point of view that the collinearity increases as
the dimensionality grows.
A common approach taken in the statistics literature is the penalized likeli-
hood, for example, Lasso (Tibshirani [11]) and adaptive Lasso (Zou [12]), SCAD
(Fan and Li [7] and Fan and Peng [9]) and nonnegative garrote (Breiman [1]).
Commonly used algorithms include LARS (Efron, Hastie, Johnstone and Tibshi-
rani [6]), LQA (Fan and Li [7]) and MM (Hunter and Li [10]). In the present
paper, Candès and Tao take a new approach, called the Dantzig selector, which
uses 1-minimization with regularization on the residuals. One promising fact is
that the Dantzig selector solves a linear program, usually faster than the existing
methods. In addition, the authors establish that, under the Uniform Uncertainty
Principle (UUP), with large probability the Dantzig selector mimics the risk of the
oracle estimator up to a logarithmic factor logp, where p denotes the number of
variables.
We appreciate the opportunity to comment on several aspects of this article.
Our discussion here will focus on four issues: (1) connection to sparse signal re-
covery in the noiseless case; (2) the UUP condition and identifiability of the model;
(3) computation and model selection; (4) minimax rate.
1. Sparse signal recovery. The “large p, small n” regression problem consid-
ered in this paper can be viewed as a generalization of the classical linear algebra
problem in which one wishes to solve the linear equation
y = Xβ,(1)
Received January 2007.
1Supported in part by NSF Grant DMS-06-04954.
2365
2366 T. T. CAI AND J. LV
where X is a given n×p matrix and y is a vector in Rn. Because p > n, the linear
equation (1) is underdetermined and there are an infinite number of solutions to the
equation. The goal is to find the “sparsest” solution under certain regularity condi-
tions. In this noiseless setting, the Dantzig selector reduces to an 1-minimization
over the space of all representations of the signal:
Minimize ‖β‖1 subject to y = Xβ.(2)
This idea of finding a sparse representation using 1-minimization has been used
in Donoho and Elad [4]. The authors have also used this approach in their earlier
work (Candès and Tao [2, 3]) on recovering sparse signals in the noiseless case.
When adding a Gaussian noise term ε to (1), the linear algebra problem becomes
a nonstandard linear regression problem because p  n. In the classical linear
regression problem when p ≤ n the least squares estimator is the solution to the
normal equation
XT y = XT Xβ.(3)
The constraint ‖XT r‖∞ ≤ λpσ (which is the same as ‖XT y − XT Xβ‖∞ ≤ λpσ )
in the convex program (DS) in this paper can be viewed as a relaxation of the
normal equation (3). And similarly to the noiseless case 1-minimization leads to
the “sparsest” solution over the space of all feasible solutions.
The authors suggest using λp = √2 logp, which is equal to √2 logn in the
orthogonal design setting. In this setting, the oracle properties of the Dantzig se-
lector are in line with those shrinkage results in Donoho and Johnstone [5] which
are shown to be optimal in the minimax sense. When p  n, it might be possible
that the regularization factor λp = √2 logp in the Dantzig selector overshrinks the
p-vector β and underestimates the nonzero coordinates. It would be interesting to
find the “optimal” regularization factor.
2. The UUP condition and identifiability of the model. A nice idea in this
paper is the use of the UUP condition. The UUP condition has also been used
by the authors in their earlier work (Candès and Tao [2, 3]) in the noiseless set-
ting. The UUP condition roughly says that for any small set of predictors, these
n-vectors are nearly orthogonal to each other. The authors give an interpretation of
the UUP condition in terms of the model identifiability and have established ora-
cle inequalities for the Dantzig selector under the UUP using geometric arguments.
However, we still have some concerns about this condition.
First, it is computationally unrealistic to verify whether the UUP condition holds
for a given design matrix X when p is large and the number of signals S is not
too small. Note that computing the S-restricted isometry constant δS of the design
matrix X is over the space of all S-subsets of {1, . . . , p}, which is of cardinality (p
S
)
.
This combinatorial complexity makes it infeasible to check the UUP condition for
reasonable values of p and S, say p = 1000 and S = 5. So it is interesting to look
DISCUSSION 2367
FIG. 1. Distribution of the maximum absolute sample correlation when n = 60, pn = 1000 (solid
curve) and n = 60, pn = 5000 (dashed curve).
for other checkable conditions that are compatible with the model identifiability
condition in a similar or new setup.
Second, as discussed in Fan and Lv [8], the UUP is hard to satisfy when the di-
mension p grows rapidly with the sample size n. This is essentially due to signifi-
cant sample correlation, that is, strong collinearity, between the predictor variables
in the high-dimensional setting of p  n. For instance, we take p independent
predictors X1, . . . ,Xp from the standard Gaussian distribution and compute the
maximum of the pairwise absolute sample correlations from an n × p design ma-
trix X. Figure 1, which is extracted from Fan and Lv [8], shows the distributions
of the maximum correlation with n = 60, p = 1000 and n = 60, p = 5000, re-
spectively. The maximum sample correlation between predictors can be very large
and close to 1. Moreover, the maximum of the first canonical correlations between
two groups of predictors, for example, three predictors in one group and five in
another, can be much larger since there are
(p
3
)(p−3
5
) = O(p8) ( n) choices in
this example.
3. Computation and model selection. Due to its nature of involving lin-
ear programming, the Dantzig selector can be solved quickly and efficiently by
a primal–dual interior point algorithm when the dimension is not ultrahigh, for
example, in the thousands. It is usually faster to implement than other existing
methods such as Lasso. However, in problems of large or ultralarge scale the com-
putational cost of implementing linear programs is still a potential hurdle. For ex-
ample, in the analysis of microarray gene expression or proteomics data, it is com-
mon to have dimension of tens of thousands and implementing linear programs in
such settings can still be computationally challenging. Therefore, it is interesting
and necessary to study ultrahigh-dimensional model selection. Recently, Fan and
Lv [8] introduced a procedure for screening variables via iteratively thresholded
2368 T. T. CAI AND J. LV
ridge regression and proposed a new method of dimension reduction called Sure
Independence Screening, for ultrahigh-dimensional feature space. The method can
improve the estimation accuracy significantly while speeding up variable selection
drastically.
In our experience, we have found that the algorithm solving the Dantzig selec-
tor is sensitive to the initial value. Trivial initial values such as constant vectors
usually do not work well. The generalized least squares estimator (XT X)−XT y
can be used as an initial value. However, the solution is usually nonsparse in our
experience.
As mentioned earlier, the regularization factor λp = √2 logp in the Dantzig
selector leads to relatively large bias in estimating the sparse regression coefficient
vector β . To reduce the bias, the authors suggest a two-stage procedure called
the Gauss–Dantzig selector which uses the original Dantzig selector for variable
selection and then runs ordinary least squares on the selected variables. It would
be interesting to know the theoretical and numerical properties of this and other
variations.
4. Minimax rate. It is appealing that the Dantzig selector achieves within a
logarithmic factor logp of the ideal risk. We are curious about the optimality of
this factor. It is unclear at this point whether the minimax factor should be logp or
logn or some other quantity. When p is polynomial in the sample size n, logp and
logn are of the same order and the difference between the two is not significant.
However, when p is exponential in n, say p = ena for some a > 0, then logp = na
becomes large, much larger than logn. It is of theoretical and practical interest to
study the minimax behavior of the problem. If the minimax rate is logp, then the
Dantzig selector is a rate optimal minimax procedure. Otherwise, it is interesting
to construct a procedure that can attain the minimax rate.
5. Concluding remarks. 1-regularization in terms of linear programming
provides a new framework for model selection and is proven effective in solving
high-dimensional problems. The Dantzig selector provides us new insight on high-
dimensional model selection. Clearly, there is much work ahead of us. This paper
opens a new area and will definitely stimulate new ideas in the future. We thank
the authors for their clear and imaginative work.
REFERENCES
[1] BREIMAN, L. (1995). Better subset regression using the nonnegative garrote. Technometrics
37 373–384. MR1365720
[2] CANDÈS, E. J. and TAO, T. (2005). Decoding by linear programming. IEEE Trans. Inform.
Theory 51 4203–4215. MR2243152
[3] CANDÈS, E. J. and TAO, T. (2006). Near-optimal signal recovery from random projections:
Universal encoding strategies? IEEE Trans. Inform. Theory 52 5406–5425. MR2300700
DISCUSSION 2369
[4] DONOHO, D. L. and ELAD, M. (2003). Optimally sparse representation in general (nonorthog-
onal) dictionaries via l1 minimization. Proc. Natl. Acad. Sci. U.S.A. 100 2197–2202.
MR1963681
[5] DONOHO, D. L. and JOHNSTONE, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage.
Biometrika 81 425–455. MR1311089
[6] EFRON, B., HASTIE, T., JOHNSTONE, I. and TIBSHIRANI, R. (2004). Least angle regression
(with discussion). Ann. Statist. 32 407–499. MR2060166
[7] FAN, J. and LI, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle
properties. J. Amer. Statist. Assoc. 96 1348–1360. MR1946581
[8] FAN, J. and LV, J. (2006). Sure independence screening for ultra-high dimensional feature
space. Unpublished manuscript. Available at arxiv.org/abs/math/0612857.
[9] FAN, J. and PENG, H. (2004). Nonconcave penalized likelihood with diverging number of
parameters. Ann. Statist. 32 928–961. MR2065194
[10] HUNTER, D. and LI, R. (2005). Variable selection using MM algorithms. Ann. Statist. 33
1617–1642. MR2166557
[11] TIBSHIRANI, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc.
Ser. B 58 267–288. MR1379242
[12] ZOU, H. (2006). The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418–
1429. MR2279469
DEPARTMENT OF STATISTICS
THE WHARTON SCHOOL
UNIVERSITY OF PENNSYLVANIA
PHILADELPHIA, PENNSYLVANIA 19104
USA
E-MAIL: tcai@wharton.upenn.edu
DEPARTMENT OF MATHEMATICS
PRINCETON UNIVERSITY
PRINCETON, NEW JERSEY 08544
USA
E-MAIL: jlv@princeton.edu


https://repository.upenn.edu/cgi/viewcontent.cgi?article=1296&amp;context=statistics_papers

Discussion: The Dantzig Selector: Statistical Estimation When \u3cem\u3ep\u3c/em\u3e is Much Larger Than \u3cem\u3en\u3c/em\u3e

Abstract

Similar works

Full text

Available Versions

ScholarlyCommons@Penn

Kosmopolis