3 research outputs found
Black-box optimization with a politician
We propose a new framework for black-box convex optimization which is
well-suited for situations where gradient computations are expensive. We derive
a new method for this framework which leverages several concepts from convex
optimization, from standard first-order methods (e.g. gradient descent or
quasi-Newton methods) to analytical centers (i.e. minimizers of self-concordant
barriers). We demonstrate empirically that our new technique compares favorably
with state of the art algorithms (such as BFGS).Comment: 19 page
An optimal first order method based on optimal quadratic averaging
In a recent paper, Bubeck, Lee, and Singh introduced a new first order method
for minimizing smooth strongly convex functions. Their geometric descent
algorithm, largely inspired by the ellipsoid method, enjoys the optimal linear
rate of convergence. We show that the same iterate sequence is generated by a
scheme that in each iteration computes an optimal average of quadratic
lower-models of the function. Indeed, the minimum of the averaged quadratic
approaches the true minimum at an optimal rate. This intuitive viewpoint
reveals clear connections to the original fast-gradient methods and cutting
plane ideas, and leads to limited-memory extensions with improved performance.Comment: 23 page
Subgradient Ellipsoid Method for Nonsmooth Convex Problems
In this paper, we present a new ellipsoid-type algorithm for solving
nonsmooth problems with convex structure. Examples of such problems include
nonsmooth convex minimization problems, convex-concave saddle-point problems
and variational inequalities with monotone operator. Our algorithm can be seen
as a combination of the standard Subgradient and Ellipsoid methods. However, in
contrast to the latter one, the proposed method has a reasonable convergence
rate even when the dimensionality of the problem is sufficiently large. For
generating accuracy certificates in our algorithm, we propose an efficient
technique, which ameliorates the previously known recipes