160,628 research outputs found
A Posteriori Probabilistic Bounds of Convex Scenario Programs with Validation Tests
Scenario programs have established themselves as efficient tools towards
decision-making under uncertainty. To assess the quality of scenario-based
solutions a posteriori, validation tests based on Bernoulli trials have been
widely adopted in practice. However, to reach a theoretically reliable
judgement of risk, one typically needs to collect massive validation samples.
In this work, we propose new a posteriori bounds for convex scenario programs
with validation tests, which are dependent on both realizations of support
constraints and performance on out-of-sample validation data. The proposed
bounds enjoy wide generality in that many existing theoretical results can be
incorporated as particular cases. To facilitate practical use, a systematic
approach for parameterizing a posteriori probability bounds is also developed,
which is shown to possess a variety of desirable properties allowing for easy
implementations and clear interpretations. By synthesizing comprehensive
information about support constraints and validation tests, improved risk
evaluation can be achieved for randomized solutions in comparison with existing
a posteriori bounds. Case studies on controller design of aircraft lateral
motion are presented to validate the effectiveness of the proposed a posteriori
bounds
Certified Roundoff Error Bounds using Bernstein Expansions and Sparse Krivine-Stengle Representations
Floating point error is an inevitable drawback of embedded systems
implementation. Computing rigorous upper bounds of roundoff errors is
absolutely necessary to the validation of critical software. This problem is
even more challenging when addressing non-linear programs. In this paper, we
propose and compare two new methods based on Bernstein expansions and sparse
Krivine-Stengle representations, adapted from the field of the global
optimization to compute upper bounds of roundoff errors for programs
implementing polynomial functions. We release two related software package
FPBern and FPKiSten, and compare them with state of the art tools. We show that
these two methods achieve competitive performance, while computing accurate
upper bounds by comparison with other tools.Comment: 20 pages, 2 table
Temporal evolution of generalization during learning in linear networks
We study generalization in a simple framework of feedforward linear networks with n inputs and n outputs, trained from examples by gradient descent on the usual quadratic error function. We derive analytical results on the behavior of the validation function corresponding to the LMS error function calculated on a set of validation patterns. We show that the behavior of the validation function depends critically on the initial conditions and on the characteristics of the noise. Under certain simple assumptions, if the initial weights are sufficiently small, the validation function has a unique minimum corresponding to an optimal stopping time for training for which simple bounds can be calculated. There exists also situations where the validation function can have more complicated and somewhat unexpected behavior such as multiple local minima (at most n) of variable depth and long but finite plateau effects. Additional results and possible extensions are briefly discussed
An Algorithmic Framework for Computing Validation Performance Bounds by Using Suboptimal Models
Practical model building processes are often time-consuming because many
different models must be trained and validated. In this paper, we introduce a
novel algorithm that can be used for computing the lower and the upper bounds
of model validation errors without actually training the model itself. A key
idea behind our algorithm is using a side information available from a
suboptimal model. If a reasonably good suboptimal model is available, our
algorithm can compute lower and upper bounds of many useful quantities for
making inferences on the unknown target model. We demonstrate the advantage of
our algorithm in the context of model selection for regularized learning
problems
Improved Uniform Test Error Bounds
We derive distribution-free uniform test error bounds that improve on VC-type bounds for validation. We show how to use knowledge of test inputs to improve the bounds. The bounds are sharp, but they require intense computation. We introduce a method to trade sharpness for speed of computation. Also, we compute the bounds for several test cases
On PAC-Bayesian Bounds for Random Forests
Existing guarantees in terms of rigorous upper bounds on the generalization
error for the original random forest algorithm, one of the most frequently used
machine learning methods, are unsatisfying. We discuss and evaluate various
PAC-Bayesian approaches to derive such bounds. The bounds do not require
additional hold-out data, because the out-of-bag samples from the bagging in
the training process can be exploited. A random forest predicts by taking a
majority vote of an ensemble of decision trees. The first approach is to bound
the error of the vote by twice the error of the corresponding Gibbs classifier
(classifying with a single member of the ensemble selected at random). However,
this approach does not take into account the effect of averaging out of errors
of individual classifiers when taking the majority vote. This effect provides a
significant boost in performance when the errors are independent or negatively
correlated, but when the correlations are strong the advantage from taking the
majority vote is small. The second approach based on PAC-Bayesian C-bounds
takes dependencies between ensemble members into account, but it requires
estimating correlations between the errors of the individual classifiers. When
the correlations are high or the estimation is poor, the bounds degrade. In our
experiments, we compute generalization bounds for random forests on various
benchmark data sets. Because the individual decision trees already perform
well, their predictions are highly correlated and the C-bounds do not lead to
satisfactory results. For the same reason, the bounds based on the analysis of
Gibbs classifiers are typically superior and often reasonably tight. Bounds
based on a validation set coming at the cost of a smaller training set gave
better performance guarantees, but worse performance in most experiments
Geometric proof for normally hyperbolic invariant manifolds
We present a new proof of the existence of normally hyperbolic manifolds and
their whiskers for maps. Our result is not perturbative. Based on the bounds on
the map and its derivative, we establish the existence of the manifold within a
given neighbourhood. Our proof follows from a graph transform type method and
is performed in the state space of the system. We do not require the map to be
invertible. From our method follows also the smoothness of the established
manifolds, which depends on the smoothness of the map, as well as rate
conditions, which follow from bounds on the derivative of the map. Our method
is tailor made for rigorous, interval arithmetic based, computer assisted
validation of the needed assumptions.Comment: 64 pages, 4 figure
A Statistical Learning Theory Approach for Uncertain Linear and Bilinear Matrix Inequalities
In this paper, we consider the problem of minimizing a linear functional
subject to uncertain linear and bilinear matrix inequalities, which depend in a
possibly nonlinear way on a vector of uncertain parameters. Motivated by recent
results in statistical learning theory, we show that probabilistic guaranteed
solutions can be obtained by means of randomized algorithms. In particular, we
show that the Vapnik-Chervonenkis dimension (VC-dimension) of the two problems
is finite, and we compute upper bounds on it. In turn, these bounds allow us to
derive explicitly the sample complexity of these problems. Using these bounds,
in the second part of the paper, we derive a sequential scheme, based on a
sequence of optimization and validation steps. The algorithm is on the same
lines of recent schemes proposed for similar problems, but improves both in
terms of complexity and generality. The effectiveness of this approach is shown
using a linear model of a robot manipulator subject to uncertain parameters.Comment: 19 pages, 2 figures, Accepted for Publication in Automatic
Theoretical analysis of cross-validation for estimating the risk of the k-Nearest Neighbor classifier
The present work aims at deriving theoretical guaranties on the behavior of
some cross-validation procedures applied to the -nearest neighbors (NN)
rule in the context of binary classification. Here we focus on the
leave--out cross-validation (LO) used to assess the performance of the
NN classifier. Remarkably this LO estimator can be efficiently computed
in this context using closed-form formulas derived by
\cite{CelisseMaryHuard11}. We describe a general strategy to derive moment and
exponential concentration inequalities for the LO estimator applied to the
NN classifier. Such results are obtained first by exploiting the connection
between the LO estimator and U-statistics, and second by making an intensive
use of the generalized Efron-Stein inequality applied to the LO estimator.
One other important contribution is made by deriving new quantifications of the
discrepancy between the LO estimator and the classification error/risk of
the NN classifier. The optimality of these bounds is discussed by means of
several lower bounds as well as simulation experiments
- …