Lasso type classifiers with a reject option

We consider the problem of binary classification where one can, for a particular cost, choose not to classify an observation. We present a simple proof for the oracle inequality for the excess risk of structural risk minimizers using a lasso type penalty.Comment: Published at http://dx.doi.org/10.1214/07-EJS058 in the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of Mathematical Statistics (http://www.imstat.org

Support vector machines with a reject option

This paper studies $\ell_1$ regularization with high-dimensional features for support vector machines with a built-in reject option (meaning that the decision of classifying an observation can be withheld at a cost lower than that of misclassification). The procedure can be conveniently implemented as a linear program and computed using standard software. We prove that the minimizer of the penalized population risk favors sparse solutions and show that the behavior of the empirical risk minimizer mimics that of the population risk minimizer. We also introduce a notion of classification complexity and prove that our minimizers adapt to the unknown complexity. Using a novel oracle inequality for the excess risk, we identify situations where fast rates of convergence occur.Comment: Published in at http://dx.doi.org/10.3150/10-BEJ320 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

Complexity regularization via localized random penalties

In this article, model selection via penalized empirical loss minimization in nonparametric classification problems is studied. Data-dependent penalties are constructed, which are based on estimates of the complexity of a small subclass of each model class, containing only those functions with small empirical loss. The penalties are novel since those considered in the literature are typically based on the entire model class. Oracle inequalities using these penalties are established, and the advantage of the new penalties over those based on the complexity of the whole model class is demonstrated.Comment: Published by the Institute of Mathematical Statistics (http://www.imstat.org) in the Annals of Statistics (http://www.imstat.org/aos/) at http://dx.doi.org/10.1214/00905360400000046

Sparsity oracle inequalities for the Lasso

This paper studies oracle properties of $\ell_1$-penalized least squares in nonparametric regression setting with random design. We show that the penalized least squares estimator satisfies sparsity oracle inequalities, i.e., bounds in terms of the number of non-zero components of the oracle vector. The results are valid even when the dimension of the model is (much) larger than the sample size and the regression matrix is not positive definite. They can be applied to high-dimensional linear regression, to nonparametric adaptive regression estimation and to the problem of aggregation of arbitrary estimators.Comment: Published at http://dx.doi.org/10.1214/07-EJS008 in the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of Mathematical Statistics (http://www.imstat.org

Optimal selection of reduced rank estimators of high-dimensional matrices

We introduce a new criterion, the Rank Selection Criterion (RSC), for selecting the optimal reduced rank estimator of the coefficient matrix in multivariate response regression models. The corresponding RSC estimator minimizes the Frobenius norm of the fit plus a regularization term proportional to the number of parameters in the reduced rank model. The rank of the RSC estimator provides a consistent estimator of the rank of the coefficient matrix; in general, the rank of our estimator is a consistent estimate of the effective rank, which we define to be the number of singular values of the target matrix that are appropriately large. The consistency results are valid not only in the classic asymptotic regime, when $n$, the number of responses, and $p$, the number of predictors, stay bounded, and $m$, the number of observations, grows, but also when either, or both, $n$ and $p$ grow, possibly much faster than $m$. We establish minimax optimal bounds on the mean squared errors of our estimators. Our finite sample performance bounds for the RSC estimator show that it achieves the optimal balance between the approximation error and the penalty term. Furthermore, our procedure has very low computational complexity, linear in the number of candidate models, making it particularly appealing for large scale problems. We contrast our estimator with the nuclear norm penalized least squares (NNP) estimator, which has an inherently higher computational complexity than RSC, for multivariate regression models. We show that NNP has estimation properties similar to those of RSC, albeit under stronger conditions. However, it is not as parsimonious as RSC. We offer a simple correction of the NNP estimator which leads to consistent rank estimation.Comment: Published in at http://dx.doi.org/10.1214/11-AOS876 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org) (some typos corrected

Asymptotics in Minimum Distance from Independence Estimation

In this paper we introduce a family of minimum distance from independence estimators, suggested by Manski's minimum mean square from independence estimator. We establish strong consistency, asymptotic normality and consistency of resampling estimates of the distribution and variance of these estimators. For Manski's estimator we derive both strong consistency and asymptotic normality.Donsker class, empirical processes, extremum estimator, nonlinear simultaneous equations models, resampling estimators

Ultrafast electron dynamics and the role of screening

This thesis focuses on the ultrafast dynamics of electronic excitations in solids and how they are influenced by the screening of the Coulomb interaction between charged particles. The impact of screening on electron dynamics is manifold, ranging from modifications of electron-electron scattering rates over trapping of excess charges to massive renormalisation of electronic band structures. The timescales of these dynamical processes are directly accessible by femtosecond time-resolved photoemission and optical spectroscopy. Three exemplary systems are investigated to shed light onto these fundamental processes: Vanadiumdioxide undergoes a phase transition from a monoclinic insulator to a rutile metal. Apart from temperature, doping and other influences, the insulator-to-metal transition can also be driven by photoexcitation. This, in the past, gave rise to a controversy about the timescales of structural and electronic transition and raised the question which of them constitutes the driving mechanism. Using time-resolved photoelectron spectroscopy, it is shown that the electronic band gap of the insulator collapses instantaneously with photoexcitation and without any structural involvement. The reason is a change of screening due to the generation of photoholes. At the same time, the symmetry of the lattice potential changes, as seen by coherent phonon spectroscopy. This potential change is likely to initiate the structural phase transition from monoclinic to rutile structure. However, the initial non-equilibrium situation can be described by a metallic electronic structure with the atoms still in the monoclinic lattice positions. The SrTiO3/vacuum interface exhibits a two-dimensional electron gas (2DEG), which is delocalised within the surface plane, but localised perpendicular to it. The lower dimensionality changes the form of the screened Coulomb interaction and the phase space within the 2DEG, leading to modified hot carrier lifetimes. These are investigated by time-resolved photoemission spectroscopy: The predicted 2D behaviour is confirmed and two distinct final states within the unoccupied electronic band structure are discovered. Furthermore, the population of the 2DEG is transiently increased by photoexcitation from localised in-gap states into the 2DEG. A different type of screening by dipole moments in amorphous ice layers, is exploited to stabilise and trap electrons within the polar medium in front of a metal surface. Thereby, the mean free path of low energy electrons in amorphous ice is estimated. Moreover, the trapped electrons are used to drive a chemical reaction: A persistent modification of the surface electronic structure of the ice layer is explained via the `dielectron hydrogen evolution reaction'. Understanding the role of screening in these systems allows to explain seemingly unrelated effects, like trapping of excess electrons in ice and the insulator-to-metal transition in VO2, within the same concept