11,832 research outputs found
Global convergence rate analysis of unconstrained optimization methods based on probabilistic models
We present global convergence rates for a line-search method which is based
on random first-order models and directions whose quality is ensured only with
certain probability. We show that in terms of the order of the accuracy, the
evaluation complexity of such a method is the same as its counterparts that use
deterministic accurate models; the use of probabilistic models only increases
the complexity by a constant, which depends on the probability of the models
being good. We particularize and improve these results in the convex and
strongly convex case.
We also analyze a probabilistic cubic regularization variant that allows
approximate probabilistic second-order models and show improved complexity
bounds compared to probabilistic first-order methods; again, as a function of
the accuracy, the probabilistic cubic regularization bounds are of the same
(optimal) order as for the deterministic case
- …