214 research outputs found
Lasso type classifiers with a reject option
We consider the problem of binary classification where one can, for a
particular cost, choose not to classify an observation. We present a simple
proof for the oracle inequality for the excess risk of structural risk
minimizers using a lasso type penalty.Comment: Published at http://dx.doi.org/10.1214/07-EJS058 in the Electronic
Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Support vector machines with a reject option
This paper studies regularization with high-dimensional features for
support vector machines with a built-in reject option (meaning that the
decision of classifying an observation can be withheld at a cost lower than
that of misclassification). The procedure can be conveniently implemented as a
linear program and computed using standard software. We prove that the
minimizer of the penalized population risk favors sparse solutions and show
that the behavior of the empirical risk minimizer mimics that of the population
risk minimizer. We also introduce a notion of classification complexity and
prove that our minimizers adapt to the unknown complexity. Using a novel oracle
inequality for the excess risk, we identify situations where fast rates of
convergence occur.Comment: Published in at http://dx.doi.org/10.3150/10-BEJ320 the Bernoulli
(http://isi.cbs.nl/bernoulli/) by the International Statistical
Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm
Complexity regularization via localized random penalties
In this article, model selection via penalized empirical loss minimization in
nonparametric classification problems is studied. Data-dependent penalties are
constructed, which are based on estimates of the complexity of a small subclass
of each model class, containing only those functions with small empirical loss.
The penalties are novel since those considered in the literature are typically
based on the entire model class. Oracle inequalities using these penalties are
established, and the advantage of the new penalties over those based on the
complexity of the whole model class is demonstrated.Comment: Published by the Institute of Mathematical Statistics
(http://www.imstat.org) in the Annals of Statistics
(http://www.imstat.org/aos/) at http://dx.doi.org/10.1214/00905360400000046
Sparsity oracle inequalities for the Lasso
This paper studies oracle properties of -penalized least squares in
nonparametric regression setting with random design. We show that the penalized
least squares estimator satisfies sparsity oracle inequalities, i.e., bounds in
terms of the number of non-zero components of the oracle vector. The results
are valid even when the dimension of the model is (much) larger than the sample
size and the regression matrix is not positive definite. They can be applied to
high-dimensional linear regression, to nonparametric adaptive regression
estimation and to the problem of aggregation of arbitrary estimators.Comment: Published at http://dx.doi.org/10.1214/07-EJS008 in the Electronic
Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Optimal selection of reduced rank estimators of high-dimensional matrices
We introduce a new criterion, the Rank Selection Criterion (RSC), for
selecting the optimal reduced rank estimator of the coefficient matrix in
multivariate response regression models. The corresponding RSC estimator
minimizes the Frobenius norm of the fit plus a regularization term proportional
to the number of parameters in the reduced rank model. The rank of the RSC
estimator provides a consistent estimator of the rank of the coefficient
matrix; in general, the rank of our estimator is a consistent estimate of the
effective rank, which we define to be the number of singular values of the
target matrix that are appropriately large. The consistency results are valid
not only in the classic asymptotic regime, when , the number of responses,
and , the number of predictors, stay bounded, and , the number of
observations, grows, but also when either, or both, and grow, possibly
much faster than . We establish minimax optimal bounds on the mean squared
errors of our estimators. Our finite sample performance bounds for the RSC
estimator show that it achieves the optimal balance between the approximation
error and the penalty term. Furthermore, our procedure has very low
computational complexity, linear in the number of candidate models, making it
particularly appealing for large scale problems. We contrast our estimator with
the nuclear norm penalized least squares (NNP) estimator, which has an
inherently higher computational complexity than RSC, for multivariate
regression models. We show that NNP has estimation properties similar to those
of RSC, albeit under stronger conditions. However, it is not as parsimonious as
RSC. We offer a simple correction of the NNP estimator which leads to
consistent rank estimation.Comment: Published in at http://dx.doi.org/10.1214/11-AOS876 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org) (some typos corrected
Asymptotics in Minimum Distance from Independence Estimation
In this paper we introduce a family of minimum distance from independence estimators, suggested by Manski's minimum mean square from independence estimator. We establish strong consistency, asymptotic normality and consistency of resampling estimates of the distribution and variance of these estimators. For Manski's estimator we derive both strong consistency and asymptotic normality.Donsker class, empirical processes, extremum estimator, nonlinear simultaneous equations models, resampling estimators
Ultrafast electron dynamics and the role of screening
This thesis focuses on the ultrafast dynamics of electronic excitations in solids and how they are influenced by the screening of the Coulomb interaction between charged particles. The impact of screening on electron dynamics is manifold, ranging from modifications of electron-electron scattering rates over trapping of excess charges to massive renormalisation of electronic band structures. The timescales of these dynamical processes are directly accessible by femtosecond time-resolved photoemission and optical spectroscopy. Three exemplary systems are investigated to shed light onto these fundamental processes: Vanadiumdioxide undergoes a phase transition from a monoclinic insulator to a rutile metal. Apart from temperature, doping and other influences, the insulator-to-metal transition can also be driven by photoexcitation. This, in the past, gave rise to a controversy about the timescales of structural and electronic transition and raised the question which of them constitutes the driving mechanism. Using time-resolved photoelectron spectroscopy, it is shown that the electronic band gap of the insulator collapses instantaneously with photoexcitation and without any structural involvement. The reason is a change of screening due to the generation of photoholes. At the same time, the symmetry of the lattice potential changes, as seen by coherent phonon spectroscopy. This potential change is likely to initiate the structural phase transition from monoclinic to rutile structure. However, the initial non-equilibrium situation can be described by a metallic electronic structure with the atoms still in the monoclinic lattice positions. The SrTiO3/vacuum interface exhibits a two-dimensional electron gas (2DEG), which is delocalised within the surface plane, but localised perpendicular to it. The lower dimensionality changes the form of the screened Coulomb interaction and the phase space within the 2DEG, leading to modified hot carrier lifetimes. These are investigated by time-resolved photoemission spectroscopy: The predicted 2D behaviour is confirmed and two distinct final states within the unoccupied electronic band structure are discovered. Furthermore, the population of the 2DEG is transiently increased by photoexcitation from localised in-gap states into the 2DEG. A different type of screening by dipole moments in amorphous ice layers, is exploited to stabilise and trap electrons within the polar medium in front of a metal surface. Thereby, the mean free path of low energy electrons in amorphous ice is estimated. Moreover, the trapped electrons are used to drive a chemical reaction: A persistent modification of the surface electronic structure of the ice layer is explained via the `dielectron hydrogen evolution reaction'. Understanding the role of screening in these systems allows to explain seemingly unrelated effects, like trapping of excess electrons in ice and the insulator-to-metal transition in VO2, within the same concept
- …