3 research outputs found
The Computational Complexity of Finding Stationary Points in Non-Convex Optimization
Finding approximate stationary points, i.e., points where the gradient is
approximately zero, of non-convex but smooth objective functions over
unrestricted -dimensional domains is one of the most fundamental problems in
classical non-convex optimization. Nevertheless, the computational and query
complexity of this problem are still not well understood when the dimension
of the problem is independent of the approximation error. In this paper, we
show the following computational and query complexity results:
1. The problem of finding approximate stationary points over unrestricted
domains is PLS-complete.
2. For , we provide a zero-order algorithm for finding
-approximate stationary points that requires at most
value queries to the objective function.
3. We show that any algorithm needs at least queries
to the objective function and/or its gradient to find -approximate
stationary points when . Combined with the above, this characterizes the
query complexity of this problem to be .
4. For , we provide a zero-order algorithm for finding
-KKT points in constrained optimization problems that requires at
most value queries to the objective function. This
closes the gap between the works of Bubeck and Mikulincer [2020] and Vavasis
[1993] and characterizes the query complexity of this problem to be
.
5. Combining our results with the recent result of Fearnley et al. [2022], we
show that finding approximate KKT points in constrained optimization is
reducible to finding approximate stationary points in unconstrained
optimization but the converse is impossible.Comment: Full version of COLT 2023 extended abstrac