99 research outputs found
Approximate Bregman Proximal Gradient Algorithm for Relatively Smooth Nonconvex Optimization
In this paper, we propose the approximate Bregman proximal gradient algorithm
(ABPG) for solving composite nonconvex optimization problems. ABPG employs a
new distance that approximates the Bregman distance, making the subproblem of
ABPG simpler to solve compared to existing Bregman-type algorithms. The
subproblem of ABPG is often expressed in a closed form. Similarly to existing
Bregman-type algorithms, ABPG does not require the global Lipschitz continuity
for the gradient of the smooth part. Instead, assuming the smooth adaptable
property, we establish the global subsequential convergence under standard
assumptions. Additionally, assuming that the Kurdyka--{\L}ojasiewicz property
holds, we prove the global convergence for a special case. Our numerical
experiments on the regularized least squares problem, the
loss problem, and the nonnegative linear system show that ABPG outperforms
existing algorithms especially when the gradient of the smooth part is not
globally Lipschitz or even local Lipschitz continuous.Comment: 26 pages, 10 figure
Fast and Provably Convergent Algorithms for Gromov-Wasserstein in Graph Data
In this paper, we study the design and analysis of a class of efficient
algorithms for computing the Gromov-Wasserstein (GW) distance tailored to
large-scale graph learning tasks. Armed with the Luo-Tseng error bound
condition~\citep{luo1992error}, two proposed algorithms, called Bregman
Alternating Projected Gradient (BAPG) and hybrid Bregman Proximal Gradient
(hBPG) enjoy the convergence guarantees. Upon task-specific properties, our
analysis further provides novel theoretical insights to guide how to select the
best-fit method. As a result, we are able to provide comprehensive experiments
to validate the effectiveness of our methods on a host of tasks, including
graph alignment, graph partition, and shape matching. In terms of both
wall-clock time and modeling performance, the proposed methods achieve
state-of-the-art results
A Variational Perspective on Accelerated Methods in Optimization
Accelerated gradient methods play a central role in optimization, achieving
optimal rates in many settings. While many generalizations and extensions of
Nesterov's original acceleration method have been proposed, it is not yet clear
what is the natural scope of the acceleration concept. In this paper, we study
accelerated methods from a continuous-time perspective. We show that there is a
Lagrangian functional that we call the \emph{Bregman Lagrangian} which
generates a large class of accelerated methods in continuous time, including
(but not limited to) accelerated gradient descent, its non-Euclidean extension,
and accelerated higher-order gradient methods. We show that the continuous-time
limit of all of these methods correspond to traveling the same curve in
spacetime at different speeds. From this perspective, Nesterov's technique and
many of its generalizations can be viewed as a systematic way to go from the
continuous-time curves generated by the Bregman Lagrangian to a family of
discrete-time accelerated algorithms.Comment: 38 pages. Subsumes an earlier working draft arXiv:1509.0361
- …