26,717 research outputs found
Convergence to Second-Order Stationarity for Constrained Non-Convex Optimization
We consider the problem of finding an approximate second-order stationary
point of a constrained non-convex optimization problem. We first show that,
unlike the gradient descent method for unconstrained optimization, the vanilla
projected gradient descent algorithm may converge to a strict saddle point even
when there is only a single linear constraint. We then provide a hardness
result by showing that checking -second order
stationarity is NP-hard even in the presence of linear constraints. Despite our
hardness result, we identify instances of the problem for which checking second
order stationarity can be done efficiently. For such instances, we propose a
dynamic second order Frank--Wolfe algorithm which converges to ()-second order stationary points in
iterations. The
proposed algorithm can be used in general constrained non-convex optimization
as long as the constrained quadratic sub-problem can be solved efficiently
Distributed saddle-point subgradient algorithms with Laplacian averaging
We present distributed subgradient methods for min-max problems with
agreement constraints on a subset of the arguments of both the convex and
concave parts. Applications include constrained minimization problems where
each constraint is a sum of convex functions in the local variables of the
agents. In the latter case, the proposed algorithm reduces to primal-dual
updates using local subgradients and Laplacian averaging on local copies of the
multipliers associated to the global constraints. For the case of general
convex-concave saddle-point problems, our analysis establishes the convergence
of the running time-averages of the local estimates to a saddle point under
periodic connectivity of the communication digraphs. Specifically, choosing the
gradient step-sizes in a suitable way, we show that the evaluation error is
proportional to , where is the iteration step. We illustrate
our results in simulation for an optimization scenario with nonlinear
constraints coupling the decisions of agents that cannot communicate directly.Comment: 15 pages, 4 figures, Proceedings of the IEEE Conference on Decision
and Control, Osaka, Japan, 201
Variational Policy Gradient Method for Reinforcement Learning with General Utilities
In recent years, reinforcement learning (RL) systems with general goals
beyond a cumulative sum of rewards have gained traction, such as in constrained
problems, exploration, and acting upon prior experiences. In this paper, we
consider policy optimization in Markov Decision Problems, where the objective
is a general concave utility function of the state-action occupancy measure,
which subsumes several of the aforementioned examples as special cases. Such
generality invalidates the Bellman equation. As this means that dynamic
programming no longer works, we focus on direct policy search. Analogously to
the Policy Gradient Theorem \cite{sutton2000policy} available for RL with
cumulative rewards, we derive a new Variational Policy Gradient Theorem for RL
with general utilities, which establishes that the parametrized policy gradient
may be obtained as the solution of a stochastic saddle point problem involving
the Fenchel dual of the utility function. We develop a variational Monte Carlo
gradient estimation algorithm to compute the policy gradient based on sample
paths. We prove that the variational policy gradient scheme converges globally
to the optimal policy for the general objective, though the optimization
problem is nonconvex. We also establish its rate of convergence of the order
by exploiting the hidden convexity of the problem, and proves that it
converges exponentially when the problem admits hidden strong convexity. Our
analysis applies to the standard RL problem with cumulative rewards as a
special case, in which case our result improves the available convergence rate
Escaping Saddle Points in Constrained Optimization
In this paper, we study the problem of escaping from saddle points in smooth
nonconvex optimization problems subject to a convex set . We
propose a generic framework that yields convergence to a second-order
stationary point of the problem, if the convex set is simple for
a quadratic objective function. Specifically, our results hold if one can find
a -approximate solution of a quadratic program subject to
in polynomial time, where is a positive constant that depends on the
structure of the set . Under this condition, we show that the
sequence of iterates generated by the proposed framework reaches an
-second order stationary point (SOSP) in at most
iterations. We
further characterize the overall complexity of reaching an SOSP when the convex
set can be written as a set of quadratic constraints and the
objective function Hessian has a specific structure over the convex set
. Finally, we extend our results to the stochastic setting and
characterize the number of stochastic gradient and Hessian evaluations to reach
an -SOSP
Augmented Lagrangian Functions for Cone Constrained Optimization: the Existence of Global Saddle Points and Exact Penalty Property
In the article we present a general theory of augmented Lagrangian functions
for cone constrained optimization problems that allows one to study almost all
known augmented Lagrangians for cone constrained programs within a unified
framework. We develop a new general method for proving the existence of global
saddle points of augmented Lagrangian functions, called the localization
principle. The localization principle unifies, generalizes and sharpens most of
the known results on existence of global saddle points, and, in essence,
reduces the problem of the existence of saddle points to a local analysis of
optimality conditions. With the use of the localization principle we obtain
first necessary and sufficient conditions for the existence of a global saddle
point of an augmented Lagrangian for cone constrained minimax problems via both
second and first order optimality conditions. In the second part of the paper,
we present a general approach to the construction of globally exact augmented
Lagrangian functions. The general approach developed in this paper allowed us
not only to sharpen most of the existing results on globally exact augmented
Lagrangians, but also to construct first globally exact augmented Lagrangian
functions for equality constrained optimization problems, for nonlinear second
order cone programs and for nonlinear semidefinite programs. These globally
exact augmented Lagrangians can be utilized in order to design new
superlinearly (or even quadratically) convergent optimization methods for cone
constrained optimization problems.Comment: This is a preprint of an article published by Springer in Journal of
Global Optimization (2018). The final authenticated version is available
online at: http://dx.doi.org/10.1007/s10898-017-0603-
Saddle-point dynamics: conditions for asymptotic stability of saddle points
This paper considers continuously differentiable functions of two vector
variables that have (possibly a continuum of) min-max saddle points. We study
the asymptotic convergence properties of the associated saddle-point dynamics
(gradient-descent in the first variable and gradient-ascent in the second one).
We identify a suite of complementary conditions under which the set of saddle
points is asymptotically stable under the saddle-point dynamics. Our first set
of results is based on the convexity-concavity of the function defining the
saddle-point dynamics to establish the convergence guarantees. For functions
that do not enjoy this feature, our second set of results relies on properties
of the linearization of the dynamics, the function along the proximal normals
to the saddle set, and the linearity of the function in one variable. We also
provide global versions of the asymptotic convergence results. Various examples
illustrate our discussion.Comment: 26 pages, To appear in SIAM Journal on Control and Optimizatio
Cooperative data-driven distributionally robust optimization
This paper studies a class of multiagent stochastic optimization problems
where the objective is to minimize the expected value of a function which
depends on a random variable. The probability distribution of the random
variable is unknown to the agents, so each one gathers samples of it. The
agents aim to cooperatively find, using their data, a solution to the
optimization problem with guaranteed out-of-sample performance. The approach is
to formulate a data-driven distributionally robust optimization problem using
Wasserstein ambiguity sets, which turns out to be equivalent to a convex
program. We reformulate the latter as a distributed optimization problem and
identify a convex-concave augmented Lagrangian function whose saddle points are
in correspondence with the optimizers provided a min-max interchangeability
criteria is met. Our distributed algorithm design then consists of the
saddle-point dynamics associated to the augmented Lagrangian. We formally
establish that the trajectories of the dynamics converge asymptotically to a
saddle point and hence an optimizer of the problem. Finally, we provide a class
of functions that meet the min-max interchangeability criteria. Simulations
illustrate our results.Comment: 14 page
Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition
We analyze stochastic gradient descent for optimizing non-convex functions.
In many cases for non-convex functions the goal is to find a reasonable local
minimum, and the main concern is that gradient updates are trapped in saddle
points. In this paper we identify strict saddle property for non-convex problem
that allows for efficient optimization. Using this property we show that
stochastic gradient descent converges to a local minimum in a polynomial number
of iterations. To the best of our knowledge this is the first work that gives
global convergence guarantees for stochastic gradient descent on non-convex
functions with exponentially many local minima and saddle points. Our analysis
can be applied to orthogonal tensor decomposition, which is widely used in
learning a rich class of latent variable models. We propose a new optimization
formulation for the tensor decomposition problem that has strict saddle
property. As a result we get the first online algorithm for orthogonal tensor
decomposition with global convergence guarantee
Fixed-Time Stable Gradient Flows: Applications to Continuous-Time Optimization
This paper proposes novel gradient-flow schemes that yield convergence to the
optimal point of a convex optimization problem within a \textit{fixed} time
from any given initial condition for unconstrained optimization, constrained
optimization, and min-max problems. The application of the modified gradient
flow to unconstrained optimization problems is studied under the assumption of
gradient-dominance. Then, a modified Newton's method is presented that exhibits
fixed-time convergence under some mild conditions on the objective function.
Building upon this method, a novel technique for solving convex optimization
problems with linear equality constraints that yields convergence to the
optimal point in fixed time is developed. More specifically, constrained
optimization problems formulated as min-max problems are considered, and a
novel method for computing the optimal solution in fixed-time is proposed using
the Lagrangian dual. Finally, the general min-max problem is considered, and a
modified scheme to obtain the optimal solution of saddle-point dynamics in
fixed time is developed. Numerical illustrations that compare the performance
of the proposed method against Newton's method, rescaled-gradient method, and
Nesterov's accelerated method are included to corroborate the efficacy and
applicability of the modified gradient flows in constrained and unconstrained
optimization problems.Comment: 15 pages, 11 figure
A Unified Approach to the Global Exactness of Penalty and Augmented Lagrangian Functions I: Parametric Exactness
In this two-part study we develop a unified approach to the analysis of the
global exactness of various penalty and augmented Lagrangian functions for
finite-dimensional constrained optimization problems. This approach allows one
to verify in a simple and straightforward manner whether a given
penalty/augmented Lagrangian function is exact, i.e. whether the problem of
unconstrained minimization of this function is equivalent (in some sense) to
the original constrained problem, provided the penalty parameter is
sufficiently large. Our approach is based on the so-called localization
principle that reduces the study of global exactness to a local analysis of a
chosen merit function near globally optimal solutions. In turn, such local
analysis can usually be performed with the use of sufficient optimality
conditions and constraint qualifications.
In the first paper we introduce the concept of global parametric exactness
and derive the localization principle in the parametric form. With the use of
this version of the localization principle we recover existing simple necessary
and sufficient conditions for the global exactness of linear penalty functions,
and for the existence of augmented Lagrange multipliers of Rockafellar-Wets'
augmented Lagrangian. Also, we obtain completely new necessary and sufficient
conditions for the global exactness of general nonlinear penalty functions, and
for the global exactness of a continuously differentiable penalty function for
nonlinear second-order cone programming problems. We briefly discuss how one
can construct a continuously differentiable exact penalty function for
nonlinear semidefinite programming problems, as well.Comment: 34 pages. arXiv admin note: text overlap with arXiv:1710.0196
- …