Faster Lagrangian-Based Methods in Convex Optimization

In this paper, we aim at unifying, simplifying, and improving the convergence rate analysis of Lagrangian-based methods for convex optimization problems. We first introduce the notion of nice primal algorithmic map, which plays a central role in the unification and in the simplification of the analysis of all Lagrangian-based methods. Equipped with a nice primal algorithmic map, we then introduce a versatile generic scheme, which allows for the design and analysis of Faster LAGrangian (FLAG) methods with a new provably sublinear rate of convergence expressed in terms of functions values and feasibility violation of the original (non-ergodic) generated sequence. To demonstrate the power and versatility of our approach and results, we show that all well-known iconic Lagrangian-based schemes admit a nice primal algorithmic map, and hence share the new faster rate of convergence results within their corresponding FLAG.Comment: 21 pages, misprints have been fixe

Resetting the Optimizer in Deep RL: An Empirical Study

We focus on the task of approximating the optimal value function in deep reinforcement learning. This iterative process is comprised of solving a sequence of optimization problems where the loss function changes per iteration. The common approach to solving this sequence of problems is to employ modern variants of the stochastic gradient descent algorithm such as Adam. These optimizers maintain their own internal parameters such as estimates of the first-order and the second-order moments of the gradient, and update them over time. Therefore, information obtained in previous iterations is used to solve the optimization problem in the current iteration. We demonstrate that this can contaminate the moment estimates because the optimization landscape can change arbitrarily from one iteration to the next one. To hedge against this negative effect, a simple idea is to reset the internal parameters of the optimizer when starting a new iteration. We empirically investigate this resetting idea by employing various optimizers in conjunction with the Rainbow algorithm. We demonstrate that this simple modification significantly improves the performance of deep RL on the Atari benchmark.Comment: Accepted at Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023

Bregman strongly nonexpansive operators in reflexive Banach spaces

We present a detailed study of right and left Bregman strongly nonexpansive operators in reflexive Banach spaces. We analyze, in particular, compositions and convex combinations of such operators, and prove the convergence of the Picard iterative method for operators of these types. Finally, we use our results to approximate common zeroes of maximal monotone mappings and solutions to convex feasibility problems.Ministerio de Educación y CienciaJunta de AndalucíaIsrael Science Foundatio

Right Bregman nonexpansive operators in Banach spaces

We introduce and study new classes of Bregman nonexpansive operators in reflexive Banach spaces. These classes of operators are associated with the Bregman distance induced by a convex function. In particular, we characterize sunny right quasi-Bregman nonexpansive retractions, and as a consequence we show that the fixed point set of any right quasi-Bregman nonexpansive operator is a sunny right quasi-Bregman nonexpansive retract of the ambient Banach space.Dirección General de Enseñanza SuperiorJunta de AndalucíaIsrael Science FoundationGraduate School of the TechnionFund for the Promotion of Research at the TechnionTechnion VPR Fun

Iterative methods for approximating fixed points of Bregman nonexpansive operators

Diverse notions of nonexpansive type operators have been extended to the more general framework of Bregman distances in reflexive Banach spaces. We study these classes of operators, mainly with respect to the existence and approximation of their (asymptotic) fixed points. In particular, the asymptotic behavior of Picard and Mann type iterations is discussed for quasi-Bregman nonexpansive operators. We also present parallel algorithms for approximating common fixed points of a finite family of Bregman strongly nonexpansive operators by means of a block operator which preserves the Bregman strong nonexpansivity. All the results hold, in particular, for the smaller class of Bregman firmly nonexpansive operators, a class which contains the generalized resolvents of monotone mappings with respect to the Bregman distance.Dirección General de Enseñanza SuperiorJunta de AndalucíaIsrael Science FoundationGraduate School of the TechnionFund for the Promotion of Research at the TechnionTechnion President’s Research Fun