1 research outputs found
Dissipativity Theory for Nesterov's Accelerated Method
In this paper, we adapt the control theoretic concept of dissipativity theory
to provide a natural understanding of Nesterov's accelerated method. Our theory
ties rigorous convergence rate analysis to the physically intuitive notion of
energy dissipation. Moreover, dissipativity allows one to efficiently construct
Lyapunov functions (either numerically or analytically) by solving a small
semidefinite program. Using novel supply rate functions, we show how to recover
known rate bounds for Nesterov's method and we generalize the approach to
certify both linear and sublinear rates in a variety of settings. Finally, we
link the continuous-time version of dissipativity to recent works on algorithm
analysis that use discretizations of ordinary differential equations.Comment: to appear in the International Conference on Machine Learning 201