Directional k-Step Newton Methods in n Variables and its Semilocal Convergence Analysis

[EN] The directional k-step Newton methods (k a positive integer) is developed for solving a single nonlinear equation in n variables. Its semilocal convergence analysis is established by using two different approaches (recurrent relations and recurrent functions) under the assumption that the first derivative satisfies a combination of the Lipschitz and the center-Lipschitz continuity conditions instead of only Lipschitz condition. The convergence theorems for the existence and uniqueness of the solution for each of them are established. Numerical examples including nonlinear Hammerstein-type integral equations are worked out and significantly improved results are obtained. It is shown that the second approach based on recurrent functions solves problems failed to be solved by first one using recurrent relations. This demonstrates the efficacy and applicability of these approaches. This work extends the directional one and two-step Newton methods for solving a single nonlinear equation in n variables. Their semilocal convergence analysis using majorizing sequences are studied in Levin (Math Comput 71(237): 251-262, 2002) and Ioannis (Num Algorithms 55(4): 503-528, 2010) under the assumption that the first derivative satisfies the Lipschitz and the combination of the Lipschitz and the center-Lipschitz continuity conditions, respectively. Finally, the computational order of convergence and the computational efficiency of developed method are studied.The authors thank the referees for their fruitful suggestions which have uncovered several weaknesses leading to the improvement in the paper. A. Kumar wishes to thank UGC-CSIR(Grant no. 2061441001), New Delhi and IIT Kharagpur, India, for their financial assistance during this work.Kumar, A.; Gupta, D.; Martínez Molada, E.; Singh, S. (2018). Directional k-Step Newton Methods in n Variables and its Semilocal Convergence Analysis. Mediterranean Journal of Mathematics. 15(2):15-34. https://doi.org/10.1007/s00009-018-1077-0S1534152Levin, Y., Ben-Israel, A.: Directional Newton methods in n variables. Math. Comput. 71(237), 251–262 (2002)Argyros, I.K., Hilout, S.: A convergence analysis for directional two-step Newton methods. Num. Algorithms 55(4), 503–528 (2010)Lukács, G.: The generalized inverse matrix and the surface-surface intersection problem. In: Theory and Practice of Geometric Modeling, pp. 167–185. Springer (1989)Argyros, I.K., Magreñán, Á.A.: Extending the applicability of Gauss–Newton method for convex composite optimization on Riemannian manifolds. Appl. Math. Comput. 249, 453–467 (2014)Argyros, I.K.: A semilocal convergence analysis for directional Newton methods. Math. Comput. 80(273), 327–343 (2011)Ortega, J.M., Rheinboldt, W.C.: Iterative solution of nonlinear equations in several variables. SIAM (2000)Argyros, I.K., Hilout, S.: Weaker conditions for the convergence of Newton’s method. J. Complex. 28(3), 364–387 (2012)Argyros, I.K., Hilout, S.: On an improved convergence analysis of Newton’s method. Appl. Math. Comput. 225, 372–386 (2013)Tapia, R.A.: The Kantorovich theorem for Newton’s method. Am. Math. Mon. 78(4), 389–392 (1971)Argyros, I.K., George, S.: Local convergence for some high convergence order Newton-like methods with frozen derivatives. SeMA J. 70(1), 47–59 (2015)Martínez, E., Singh, S., Hueso, J.L., Gupta, D.K.: Enlarging the convergence domain in local convergence studies for iterative methods in Banach spaces. Appl. Math. Comput. 281, 252–265 (2016)Argyros, I.K., Behl, R. Motsa,S.S.: Ball convergence for a family of quadrature-based methods for solving equations in banach Space. Int. J. Comput. Methods, pp. 1750017 (2016)Parhi, S.K., Gupta, D.K.: Convergence of Stirling’s method under weak differentiability condition. Math. Methods Appl. Sci. 34(2), 168–175 (2011)Prashanth, M., Gupta, D.K.: A continuation method and its convergence for solving nonlinear equations in Banach spaces. Int. J. Comput. Methods 10(04), 1350021 (2013)Parida, P.K., Gupta, D.K.: Recurrence relations for semilocal convergence of a Newton-like method in banach spaces. J. Math. Anal. Appl. 345(1), 350–361 (2008)Argyros, I.K., Hilout, S.: Convergence of Directional Methods under mild differentiability and applications. Appl. Math. Comput. 217(21), 8731–8746 (2011)Amat, S, Bermúdez, C., Hernández-Verón, M.A., Martínez, E.: On an efficient k-step iterative method for nonlinear equations. J. Comput. Appl. Math. 302, 258–271 (2016)Hernández-Verón, M.A., Martínez, E., Teruel, C.: Semilocal convergence of a k-step iterative process and its application for solving a special kind of conservative problems. Num. Algorithms, pp. 1–23Argyros, M., Hernández, I.K., Hilout, S., Romero, N.: Directional Chebyshev-type methods for solving equations. Math. Comput. 84(292), 815–830 (2015)Davis, P.J., Rabinowitz, P.: Methods of numerical integration. Courier Corporation (2007)Cordero, A, Torregrosa, J.R.: Variants of Newton’s method using fifth-order quadrature formulas. Appl. Math. Computation . 190(1), 686–698 (2007)Weerakoon, S., Fernando, T.G.I.: A variant of Newton’s method with accelerated third-order convergence. Appl. Math. Lett. 13(8), 87–93 (2000

Local Convergence of Newton-type Methods for Nonsmooth Constrained Equations and Applications

In this thesis we consider constrained systems of equations. The focus is on local Newton-type methods for the solution of constrained systems which converge locally quadratically under mild assumptions implying neither local uniqueness of solutions nor differentiability of the equation function at solutions. The first aim of this thesis is to improve existing local convergence results of the constrained Levenberg-Marquardt method. To this end, we describe a general Newton-type algorithm. Then we prove local quadratic convergence of this general algorithm under the same four assumptions which were recently used for the local convergence analysis of the LP-Newton method. Afterwards, we show that, besides the LP-Newton method, the constrained Levenberg-Marquardt method can be regarded as a special realization of the general Newton-type algorithm and therefore enjoys the same local convergence properties. Thus, local quadratic convergence of a nonsmooth constrained Levenberg-Marquardt method is proved without requiring conditions implying the local uniqueness of solutions. As already mentioned, we use four assumptions for the local convergence analysis of the general Newton-type algorithm. The second aim of this thesis is a detailed discussion of these convergence assumptions for the case that the equation function of the constrained system is piecewise continuously differentiable. Some of the convergence assumptions seem quite technical and difficult to check. Therefore, we look for sufficient conditions which are still mild but which seem to be more familiar. We will particularly prove that the whole set of the convergence assumptions holds if some set of local error bound conditions is satisfied and in addition the feasible set of the constrained system excludes those zeros of the selection functions which are not zeros of the equation function itself, at least in a sufficiently small neighborhood of some fixed solution. We apply our results to constrained systems arising from complementarity systems, i.e., systems of equations and inequalities which contain complementarity constraints. Our new conditions are discussed for a suitable reformulation of the complementarity system as constrained system of equations by means of the minimum function. In particular, it will turn out that the whole set of the convergence assumptions is actually implied by some set of local error bound conditions. In addition, we provide a new constant rank condition implying the whole set of the convergence assumptions. Particularly, we provide adapted formulations of our new conditions for special classes of complementarity systems. We consider Karush-Kuhn-Tucker (KKT) systems arising from optimization problems, variational inequalities, or generalized Nash equilibrium problems (GNEPs) and Fritz-John (FJ) systems arising from GNEPs. Thus, we obtain for each problem class conditions which guarantee local quadratic convergence of the general Newton-type algorithm and its special realizations to a solution of the particular problem. Moreover, we prove for FJ systems of GNEPs that generically some full row rank condition is satisfied at any solution of the FJ system of a GNEP. The latter condition implies the whole set of the convergence assumptions if the functions which characterize the GNEP are sufficiently smooth. Finally, we describe an idea for a possible globalization of our Newton-type methods, at least for the case that the constrained system arises from a certain smooth reformulation of the KKT system of a GNEP. More precisely, a hybrid method is presented whose local part is the LP-Newton method. The hybrid method turns out to be, under appropriate conditions, both globally and locally quadratically convergent

A Bregman forward-backward linesearch algorithm for nonconvex composite optimization: superlinear convergence to nonisolated local minima

We introduce Bella, a locally superlinearly convergent Bregman forward backward splitting method for minimizing the sum of two nonconvex functions, one of which satisfying a relative smoothness condition and the other one possibly nonsmooth. A key tool of our methodology is the Bregman forward-backward envelope (BFBE), an exact and continuous penalty function with favorable first- and second-order properties, and enjoying a nonlinear error bound when the objective function satisfies a Lojasiewicz-type property. The proposed algorithm is of linesearch type over the BFBE along candidate update directions, and converges subsequentially to stationary points, globally under a KL condition, and owing to the given nonlinear error bound can attain superlinear convergence rates even when the limit point is a nonisolated minimum, provided the directions are suitably selected

A Simple and Efficient Algorithm for Nonlinear Model Predictive Control

We present PANOC, a new algorithm for solving optimal control problems arising in nonlinear model predictive control (NMPC). A usual approach to this type of problems is sequential quadratic programming (SQP), which requires the solution of a quadratic program at every iteration and, consequently, inner iterative procedures. As a result, when the problem is ill-conditioned or the prediction horizon is large, each outer iteration becomes computationally very expensive. We propose a line-search algorithm that combines forward-backward iterations (FB) and Newton-type steps over the recently introduced forward-backward envelope (FBE), a continuous, real-valued, exact merit function for the original problem. The curvature information of Newton-type methods enables asymptotic superlinear rates under mild assumptions at the limit point, and the proposed algorithm is based on very simple operations: access to first-order information of the cost and dynamics and low-cost direct linear algebra. No inner iterative procedure nor Hessian evaluation is required, making our approach computationally simpler than SQP methods. The low-memory requirements and simple implementation make our method particularly suited for embedded NMPC applications

Generalized Newton's Method based on Graphical Derivatives

This paper concerns developing a numerical method of the Newton type to solve systems of nonlinear equations described by nonsmooth continuous functions. We propose and justify a new generalized Newton algorithm based on graphical derivatives, which have never been used to derive a Newton-type method for solving nonsmooth equations. Based on advanced techniques of variational analysis and generalized differentiation, we establish the well-posedness of the algorithm, its local superlinear convergence, and its global convergence of the Kantorovich type. Our convergence results hold with no semismoothness assumption, which is illustrated by examples. The algorithm and main results obtained in the paper are compared with well-recognized semismooth and $B$-differentiable versions of Newton's method for nonsmooth Lipschitzian equations

Optimal bilinear control of Gross-Pitaevskii equations

A mathematical framework for optimal bilinear control of nonlinear Schr\"odinger equations of Gross-Pitaevskii type arising in the description of Bose-Einstein condensates is presented. The obtained results generalize earlier efforts found in the literature in several aspects. In particular, the cost induced by the physical work load over the control process is taken into account rather then often used $L^2$- or $H^1$-norms for the cost of the control action. Well-posedness of the problem and existence of an optimal control is proven. In addition, the first order optimality system is rigorously derived. Also a numerical solution method is proposed, which is based on a Newton type iteration, and used to solve several coherent quantum control problems.Comment: 30 pages, 14 figure

A Multigrid Optimization Algorithm for the Numerical Solution of Quasilinear Variational Inequalities Involving the pp-Laplacian

In this paper we propose a multigrid optimization algorithm (MG/OPT) for the numerical solution of a class of quasilinear variational inequalities of the second kind. This approach is enabled by the fact that the solution of the variational inequality is given by the minimizer of a nonsmooth energy functional, involving the $p$-Laplace operator. We propose a Huber regularization of the functional and a finite element discretization for the problem. Further, we analyze the regularity of the discretized energy functional, and we are able to prove that its Jacobian is slantly differentiable. This regularity property is useful to analyze the convergence of the MG/OPT algorithm. In fact, we demostrate that the algorithm is globally convergent by using a mean value theorem for semismooth functions. Finally, we apply the MG/OPT algorithm to the numerical simulation of the viscoplastic flow of Bingham, Casson and Herschel-Bulkley fluids in a pipe. Several experiments are carried out to show the efficiency of the proposed algorithm when solving this kind of fluid mechanics problems