Comment: Classifier Technology and the Illusion of Progress

Comment on Classifier Technology and the Illusion of Progress [math.ST/0606441]Comment: Published at http://dx.doi.org/10.1214/088342306000000024 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Comment: Classifier Technology and the Illusion of Progress

Comment on Classifier Technology and the Illusion of Progress [math.ST/0606441]Comment: Published at http://dx.doi.org/10.1214/088342306000000042 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Sparse inverse covariance estimation with the lasso

We consider the problem of estimating sparse graphs by a lasso penalty applied to the inverse covariance matrix. Using a coordinate descent procedure for the lasso, we develop a simple algorithm that is remarkably fast: in the worst cases, it solves a 1000 node problem (~500,000 parameters) in about a minute, and is 50 to 2000 times faster than competing methods. It also provides a conceptual link between the exact problem and the approximation suggested by Meinhausen and Buhlmann (2006). We illustrate the method on some cell-signaling data from proteomics.Comment: submitte

Pathwise coordinate optimization

We consider ``one-at-a-time'' coordinate-wise descent algorithms for a class of convex optimization problems. An algorithm of this kind has been proposed for the $L_1$-penalized regression (lasso) in the literature, but it seems to have been largely ignored. Indeed, it seems that coordinate-wise algorithms are not often used in convex optimization. We show that this algorithm is very competitive with the well-known LARS (or homotopy) procedure in large lasso problems, and that it can be applied to related methods such as the garotte and elastic net. It turns out that coordinate-wise descent does not work in the ``fused lasso,'' however, so we derive a generalized algorithm that yields the solution in much less time that a standard convex optimizer. Finally, we generalize the procedure to the two-dimensional fused lasso, and demonstrate its performance on some image smoothing problems.Comment: Published in at http://dx.doi.org/10.1214/07-AOAS131 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Regularization Paths for Generalized Linear Models via Coordinate Descent

We develop fast algorithms for estimation of generalized linear models with convex penalties. The models include linear regression, two-class logistic regression, and multi- nomial regression problems while the penalties include âÂÂ_1 (the lasso), âÂÂ_2 (ridge regression) and mixtures of the two (the elastic net). The algorithms use cyclical coordinate descent, computed along a regularization path. The methods can handle large problems and can also deal efficiently with sparse features. In comparative timings we find that the new algorithms are considerably faster than competing methods.

Will Innovation Flourish in the Future?

Este artículo apuesta por la investigación básica como clave para la futura innovación, diferenciándola de otros tipos de investigación que normalmente están subvencionadas.Estudios recientes muestran que de la mitad a dos tercios del crecimiento económico de los países desarrollados está basado en el conocimiento, lo que pone de manifiesto la necesidad de mantener a las universidades que se dedican a la investigación con un alto nivel de innovación. Pero la cuestión que se plantea aquí es cómo favorecer la creatividad en aquellos jóvenes que la poseen. El punto de vista que se defiende en este artículo es que debería haber escuelas preuniversitarias de excelencia que reúnan a las mejores mentes para introducirlas tempranamente a la ciencia y darles oportunidades para el trabajo creativo. También, libertad en la investigación con el fin de que produzcan nuevas ideas y no estén condicionados por la ortodoxia científica imperante.De todos los tipos de investigación, la investigación básica es la más vulnerable porque lleva implícita una inversión a largo plazo, busca el conocimiento científico por sí mismo, y ni sus resultados ni su aplicación pueden predecirse. De todo ello se desprende, que es de crucial importancia proporcionar a los jóvenes una educación que propicie la investigación básica como forma de asegurar la existencia de cerebros innovadores para el futuro