TOINT, Philippe

English

Repository of the University of Namur

RESEARCH OUTPUTS / RÉSULTATS DE RECHERCHEAuthor(s) - Auteur(s) :Publication date - Date de publication :Permanent link - Permalien :Rights / License - Licence de droit d’auteur :Bibliothèque Universitaire Moretus PlantinInstitutional Repository - Research PortalDépôt Institutionnel - Portail de la Rechercheresearchportal.unamur.beUniversity of NamurDivergence of the ADAM algorithm with fixed-stepsizeTOINT, PhilippePublication date:2023Document VersionOther versionLink to publicationCitation for pulished version (HARVARD):TOINT, P 2023 'Divergence of the ADAM algorithm with fixed-stepsize: a (very) simple example' Arxiv.General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.            • Users may download and print one copy of any publication from the public portal for the purpose of private study or research.            • You may not further distribute the material or use it for any profit-making activity or commercial gain            • You may freely distribute the URL identifying the publication in the public portal ?Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Download date: 08. Sep. 2023Divergence of the ADAM algorithm with fixed-stepsize:a (very) simple examplePhilippe L. Toint∗19 VI 2023AbstractA very simple unidimensional function with Lipschitz continuous gradient is constructedsuch that the ADAM algorithm with constant stepsize, started from the origin, diverges whenapplied to minimize this function in the absence of noise on the gradient. Divergence occursirrespective of the choice of the method parameters.Keywords: ADAM algorithm, machine learning, deterministic nonconvex optimization.1 IntroductionThis short note provides a new explicit example of failure of the ADAM algorithm [3], one of themost popular training methods in machine learning. Given the problemminx∈IRnf(x) (1)where f is continuously differentiable function from IRn into IR with Lipschitz continuous gradient,and a starting iterate x0, the ADAM sequence of iterates is defined (see [4]), for i ∈ {1, . . . , n}and k ≥ 0, by the recurrences[mk]i = β1[mk−1]i + (1− β1)[gk]i, (2)[vk]i = β2[vk−1]i + (1− β2)[gk]2i , (3)[xk+1]i = [xk]i − α[mk]i√[vk]i, (4)where [v]i is the i-th component of the vector v ∈ IRn, mk is the k-th “momentum”, xk is the k-thiterate, gk = ∇1xf(xk), β1 ∈ [0, 1) is the momentum parameter and β2 ∈ [0, 1) is the “forgetting”parameter, and α > 0 is a (fixed) steplength/learning-rate parameter. The recurrences (2) and(3) are initialized by setting, for i ∈ {1, . . . , n}, [m−1]i = [g0]i and [v−1]i = [g0]2i , respectively.ADAM is intended to converge to find first-order points for problem (1), in the sense that, foreach i ∈ {1, . . . , n}, |[gk]i| should converge to zero when k tends to infinity. In practice, thisalgorithm is most often used in a stochastic context where the gradient gk is contaminated bynoise (typically resulting from sampling) and has generated a considerable interest in the machinelearning community.Despite its widespread use, difficulties with this algorithm are not new. In the noiseless (de-terministic/full batch) case, obstacles for proving convergence were in particular mentioned in[2], essentially pointing out the possibility that second-order terms in the Taylor’s expansion ofthe objective function could not vanish quickly enough. In [4, Theorem 1] an example of non∗Namur Center for Complex Systems (naXys), University of Namur, Namur, Belgium. Email:philippe.toint@unamur.be.1Toint: ADAM may fail to converge 2convergence on a convex function was produced in the online-learning stochastic context, but thisexample crucially depends on the nonzero variance of the noise. In a recent discussion at theJune 2023 Thematic Einstein Semester on Optimization and Machine Learning in Berlin, it wassuggested that, although likely, no explicit example of failure of Adam with fixed stepsize wasavailable for the deterministic case (where the variance is zero). This prompted the author toproduce the (very simple) one which is, for the record, detailed in the next section. We notethat an again convex but deterministic example had already been provided in the comprehensiveanalysis of ADAM’s behaviour (with decreasing stepsize) detailed in [5] (see Propositions 3.3 andE1). This analysis describes conditions which delineate a region strictly included in [0, 1)2 suchthat ADAM with parameters β1 and β2 chosen in this region generates a diverging sequence onthis example. In contrast, the simple example we are about to discuss is nonconvex and appliesto the entire [0, 1)2, but requires constant stepsize. It can therefore be seen as complementing theanalysis of [5].2 The exampleTo show that the ADAM algorithm may fail to converge on nonconvex functions with Lipschitzgradient, we will exhibit an example in dimension one, which we construct in two stages. We firstdefine sequences of iterates, together with associated function and gradient values which remainconstant throughout the iterations. We next verify that these sequences may be considered asgenerated by applying the ADAM algorithm to a nonconvex objective function with Lipschitzgradient. (Since the example is unidimensional, we omit the component indices (i) if what follows.)For k ≥ 0, let the sequence of function values and gradients be defined byfk = 0 and gk = −1, (5)and the sequence of (potential) iterates be defined (from (2)-(4)) bymk = β1mk−1 + (1− β1)gk = −1, (6)vk = β2vk−1 + (1− β2)g2k = 1, (7)xk+1 = xk − αmk√vk= xk + α, (8)where we used (5) to derive the last equality in (6) and (7). Thus∑kj=0 βk−j ≤ 1/(1 − β) forβ ∈ (0, 1) imply thatsk = xk+1 − xk = α, (9)for k ≥ 0 and xk tends tp infinity. We now show that there exists a (nonconvex) univariatefunction f1 defined on IR+ with Lipschitz continuous gradient such that fk = f(xk) = 0 andgk = ∇1xf1(xk) = −1 for all k ≥ 0. Indeed, a simple Hermite interpolation calculation based ofthese conditions yields that, for all t ≥ 0,f1(t) = −(t− xk(t)) +3sk(t)(t− xk(t))2 −2s2k(t)(t− xk(t))3, (10)where k(t) is such that t ∈ [xk, xk+1]. We may then definef(t) ={f1(t) if t ≥ 0,−t if t < 0,so that f(t) is well-defined on the whole of IR, has Lipschitz continuous gradient and is such thatthe ADAM algorithm (6)-(8) applied on f starting from x0 = 0 generates iterates with |gk| = 1for all k ≥ 0. We thus conclude that the ADAM algorithm fails to converge on this particularinstance of problem (1). A graph of f(t) for t ∈ [−1, 10], β1 = β2 = 0.9 is shown in Figure 1. OneToint: ADAM may fail to converge 3-1 0 1 2 3 4 5 6 7 8 9 10-0.500.511.5Figure 1: The shape of f(t) for small values of |t|also verifies that the Lipschitz constant on the interval [xk, xk+1] is given byLk = supt∈(xl,xk+1)|∇2tf(t)| =6skso that, using (9),L = maxk≥0Lk =6αMoreover, defining Tk(s) = fk+gks, it results from (5), [1, Theorem A.9.2], (9) and the inequalities|fk+1 − Tk(sk)| = sk ≤1αs2k and |gk+1 −∇1sTk(s)| = | − 1 + 1| ≤1αskthat f(t) is bounded below by a constant only depending on α. As a consequence, we see that,for any fixed (β1, β2) ∈ [0, 1)2 and α > 0, there exist unidimensional functions with Lipschitzcontinuous gradient whose gradient’s Lipschitz constant is as small as 6/α, which is boundedbelow by a constant only depending on α and for which the ADAM algorithm (6)-(8) starting fromx0 = 0 generates iterates with constant nonzero gradients (therefore failing to converge).Since our example is unidimensional and since ADAM is defined componentwise, the sameconclusion obviously applies irrespective of n, the problem dimension. Indeed divergence in asingle component implies divergence on the whole space.Our result thus extends that of [5] in that it includes methods for arbitrary (β1, β2) ∈ (0, 1)2but fixed stepsize. Note that |∇1tf(t)| is bounded by L for all t ∈ IR, again at variance with theexample of this reference.Observe that our conclusions would also hold if we had fixed gk to another negative constant(we can multiply f by this constant) or if, instead of (4), we had considered[xk+1]i = [xk]i −α [mk]i√ε+ [vk]2i, or [xk+1]i = [xk]i −α [mk]iε+√[vk]2i,where ε is a small positive constant, but they do not apply in the more realistic situation wherestepsizes αk → 0 are used (as is for instance the case in [5, Proposition 1.1], where αk is a multipleof 1/√k). We finally note that we have chosen a constant zero value for fk in order to simplify ourbounds, but that it is also possible to choose fk+1 > fk (leading to an monotonically increasingsequence of function values) without qualitatively affecting our conclusion, although this leads toa larger value of L.AcknowledgementThanks to Satyen Kale, Omri Weinstein, Alena Kopaničáková and Serge Gratton for interesting exchanges.Toint: ADAM may fail to converge 4References[1] C. Cartis, N. I. M. Gould, and Ph. L. Toint. Evaluation complexity of algorithms for nonconvex optimization.Number 30 in MOS-SIAM Series on Optimization. SIAM, Philadelphia, USA, June 2022.[2] A. Défossez, L. Bottou, F. Bach, and N. Usunier. A simple convergence proof for Adam and Adagrad.Transactions on Machine Learning Research, October 2022.[3] D. Kingma and J. Ba. Adam: A method for stochastic optimization. In Proceedings in the InternationalConference on Learning Representations (ICLR), 2015.[4] S. Reddi, S. Kale, and S. Kumar. On the convergence of Adam and beyond. In Proceedings in the InternationalConference on Learning Representations (ICLR), 2018.[5] Y. Zhang, C. Chen, N. Shi, R. Sun, and Z.-Q. Luo. Adam can converge without any modification on updaterules. arXiv:2208.09632v5, 2022.

Divergence of the ADAM algorithm with fixed-stepsize:a (very) simple example

https://pure.unamur.be/ws/files/85430408/arXiv-2308.00720.pdf

Divergence of the ADAM algorithm with fixed-stepsize:a (very) simple example

Abstract

Similar works

Full text

Available Versions

Repository of the University of Namur