54 research outputs found
Robust Online Convex Optimization in the Presence of Outliers
We consider online convex optimization when a number k of data points are
outliers that may be corrupted. We model this by introducing the notion of
robust regret, which measures the regret only on rounds that are not outliers.
The aim for the learner is to achieve small robust regret, without knowing
where the outliers are. If the outliers are chosen adversarially, we show that
a simple filtering strategy on extreme gradients incurs O(k) additive overhead
compared to the usual regret bounds, and that this is unimprovable, which means
that k needs to be sublinear in the number of rounds. We further ask which
additional assumptions would allow for a linear number of outliers. It turns
out that the usual benign cases of independently, identically distributed
(i.i.d.) observations or strongly convex losses are not sufficient. However,
combining i.i.d. observations with the assumption that outliers are those
observations that are in an extreme quantile of the distribution, does lead to
sublinear robust regret, even though the expected number of outliers is linear
All-In-One Robust Estimator of the Gaussian Mean
The goal of this paper is to show that a single robust estimator of the mean
of a multivariate Gaussian distribution can enjoy five desirable properties.
First, it is computationally tractable in the sense that it can be computed in
a time which is at most polynomial in dimension, sample size and the logarithm
of the inverse of the contamination rate. Second, it is equivariant by
translations, uniform scaling and orthogonal transformations. Third, it has a
high breakdown point equal to , and a nearly-minimax-rate-breakdown point
approximately equal to . Fourth, it is minimax rate optimal, up to a
logarithmic factor, when data consists of independent observations corrupted by
adversarially chosen outliers. Fifth, it is asymptotically efficient when the
rate of contamination tends to zero. The estimator is obtained by an iterative
reweighting approach. Each sample point is assigned a weight that is
iteratively updated by solving a convex optimization problem. We also establish
a dimension-free non-asymptotic risk bound for the expected error of the
proposed estimator. It is the first result of this kind in the literature and
involves only the effective rank of the covariance matrix. Finally, we show
that the obtained results can be extended to sub-Gaussian distributions, as
well as to the cases of unknown rate of contamination or unknown covariance
matrix.Comment: 41 pages, 5 figures; added sub-Gaussian case with unknown Sigma or
ep
The Geometric Median and Applications to Robust Mean Estimation
This paper is devoted to the statistical and numerical properties of the
geometric median, and its applications to the problem of robust mean estimation
via the median of means principle. Our main theoretical results include (a) an
upper bound for the distance between the mean and the median for general
absolutely continuous distributions in R^d, and examples of specific classes of
distributions for which these bounds do not depend on the ambient dimension
; (b) exponential deviation inequalities for the distance between the sample
and the population versions of the geometric median, which again depend only on
the trace-type quantities and not on the ambient dimension. As a corollary, we
deduce improved bounds for the (geometric) median of means estimator that hold
for large classes of heavy-tailed distributions. Finally, we address the error
of numerical approximation, which is an important practical aspect of any
statistical estimation procedure. We demonstrate that the objective function
minimized by the geometric median satisfies a "local quadratic growth"
condition that allows one to translate suboptimality bounds for the objective
function to the corresponding bounds for the numerical approximation to the
median itself. As a corollary, we propose a simple stopping rule (applicable to
any optimization method) which yields explicit error guarantees. We conclude
with the numerical experiments including the application to estimation of mean
values of log-returns for S&P 500 data.Comment: 28 pages, 2 figure
- …