8,643 research outputs found
Learning with Square Loss: Localization through Offset Rademacher Complexity
We consider regression with square loss and general classes of functions
without the boundedness assumption. We introduce a notion of offset Rademacher
complexity that provides a transparent way to study localization both in
expectation and in high probability. For any (possibly non-convex) class, the
excess loss of a two-step estimator is shown to be upper bounded by this offset
complexity through a novel geometric inequality. In the convex case, the
estimator reduces to an empirical risk minimizer. The method recovers the
results of \citep{RakSriTsy15} for the bounded case while also providing
guarantees without the boundedness assumption.Comment: 21 pages, 1 figur
Escaping the Local Minima via Simulated Annealing: Optimization of Approximately Convex Functions
We consider the problem of optimizing an approximately convex function over a
bounded convex set in using only function evaluations. The
problem is reduced to sampling from an \emph{approximately} log-concave
distribution using the Hit-and-Run method, which is shown to have the same
complexity as sampling from log-concave distributions. In
addition to extend the analysis for log-concave distributions to approximate
log-concave distributions, the implementation of the 1-dimensional sampler of
the Hit-and-Run walk requires new methods and analysis. The algorithm then is
based on simulated annealing which does not relies on first order conditions
which makes it essentially immune to local minima.
We then apply the method to different motivating problems. In the context of
zeroth order stochastic convex optimization, the proposed method produces an
-minimizer after noisy function
evaluations by inducing a -approximately log concave
distribution. We also consider in detail the case when the "amount of
non-convexity" decays towards the optimum of the function. Other applications
of the method discussed in this work include private computation of empirical
risk minimizers, two-stage stochastic programming, and approximate dynamic
programming for online learning.Comment: 27 page
- …