541 research outputs found
An optimal subgradient algorithm for large-scale convex optimization in simple domains
This paper shows that the optimal subgradient algorithm, OSGA, proposed in
\cite{NeuO} can be used for solving structured large-scale convex constrained
optimization problems. Only first-order information is required, and the
optimal complexity bounds for both smooth and nonsmooth problems are attained.
More specifically, we consider two classes of problems: (i) a convex objective
with a simple closed convex domain, where the orthogonal projection on this
feasible domain is efficiently available; (ii) a convex objective with a simple
convex functional constraint. If we equip OSGA with an appropriate
prox-function, the OSGA subproblem can be solved either in a closed form or by
a simple iterative scheme, which is especially important for large-scale
problems. We report numerical results for some applications to show the
efficiency of the proposed scheme. A software package implementing OSGA for
above domains is available
Constructing a subgradient from directional derivatives for functions of two variables
For any scalar-valued bivariate function that is locally Lipschitz continuous
and directionally differentiable, it is shown that a subgradient may always be
constructed from the function's directional derivatives in the four compass
directions, arranged in a so-called "compass difference". When the original
function is nonconvex, the obtained subgradient is an element of Clarke's
generalized gradient, but the result appears to be novel even for convex
functions. The function is not required to be represented in any particular
form, and no further assumptions are required, though the result is
strengthened when the function is additionally L-smooth in the sense of
Nesterov. For certain optimal-value functions and certain parametric solutions
of differential equation systems, these new results appear to provide the only
known way to compute a subgradient. These results also imply that centered
finite differences will converge to a subgradient for bivariate nonsmooth
functions. As a dual result, we find that any compact convex set in two
dimensions contains the midpoint of its interval hull. Examples are included
for illustration, and it is demonstrated that these results do not extend
directly to functions of more than two variables or sets in higher dimensions.Comment: 16 pages, 2 figure
Structure-Aware Methods for Expensive Derivative-Free Nonsmooth Composite Optimization
We present new methods for solving a broad class of bound-constrained
nonsmooth composite minimization problems. These methods are specially designed
for objectives that are some known mapping of outputs from a computationally
expensive function. We provide accompanying implementations of these methods:
in particular, a novel manifold sampling algorithm (\mspshortref) with
subproblems that are in a sense primal versions of the dual problems solved by
previous manifold sampling methods and a method (\goombahref) that employs more
difficult optimization subproblems. For these two methods, we provide rigorous
convergence analysis and guarantees. We demonstrate extensive testing of these
methods. Open-source implementations of the methods developed in this
manuscript can be found at \url{github.com/POptUS/IBCDFO/}
- …