A multidimensional dynamical approach to iterative methods with memory

[EN] A dynamical approach on the dynamics of iterative methods with memory for solving nonlinear equations is made. We have designed new methods with memory from Steffensen’ or Traub’s schemes, as well as from a parametric family of iterative procedures of third- and fourth-order of convergence. We study the local order of convergence of the new iterative methods with memory. We define each iterative method with memory as a discrete dynamical system and we analyze the stability of the fixed points of its rational operator associated on quadratic polynomials. As far as we know, there is no dynamical study on iterative methods with memory and the techniques of complex dynamics used in schemes without memory are not useful in this context. So, we adapt real multidimensional dynamical tools to afford this task. The dynamical behavior of Secant method and the versions of Steffensen’ and Traub’s schemes with memory, applied on quadratic polynomials, are analyzed. Different kinds of behavior occur, being in general very stable but pathologic cases as attracting strange fixed points are also found. Finally, a modified parametric family of order four, applied on quadratic polynomials, is also studied, showing the bifurcations diagrams and the appearance of chaos.This research was partially supported by Ministerio de Economia y Competitividad MTM2014-52016-C02-2-P.Campos, B.; Cordero Barbero, A.; Torregrosa Sánchez, JR.; Vindel, P. (2015). A multidimensional dynamical approach to iterative methods with memory. Applied Mathematics and Computation. 271:701-715. https://doi.org/10.1016/j.amc.2015.09.05670171527

A dynamical comparison between iterative methods with memory: Are the derivatives good for the memory?

[EN] The role of the derivatives at the iterative expression of methods with memory for solving nonlinear equations is analyzed in this manuscript. To get this aim, a known class of methods without memory is transformed into different families involving or not derivatives with an only accelerating parameter, then they are defined as discrete dynamical systems and the stability of the fixed points of their rational operators on quadratic polynomials are studied by means of real multidimensional dynamical tools, showing in all cases similar results. Finally, a different approach holding the derivatives, and by using different accelerating parameters, in the iterative methods involved present the most stable results, showing that the role of the appropriated accelerating factors is the most relevant fact in the design of this kind of iterative methods. (C) 2016 Elsevier B.V. All rights reserved.This research was partially supported by Ministerio de Economia y Competitividad MTM2014-52016-C02-2-P and Generalitat Valenciana PROME-TEO/2016/089.Cordero Barbero, A.; Jordan-Lluch, C.; Torregrosa Sánchez, JR. (2017). A dynamical comparison between iterative methods with memory: Are the derivatives good for the memory?. Journal of Computational and Applied Mathematics. 318:335-347. https://doi.org/10.1016/j.cam.2016.08.049S33534731

Iterative methods with memory for solving systems of nonlinear equations using a second order approximation

[EN] Iterative methods for solving nonlinear equations are said to have memory when the calculation of the next iterate requires the use of more than one previous iteration. Methods with memory usually have a very stable behavior in the sense of the wideness of the set of convergent initial estimations. With the right choice of parameters, iterative methods without memory can increase their order of convergence significantly, becoming schemes with memory. In this work, starting from a simple method without memory, we increase its order of convergence without adding new functional evaluations by approximating the accelerating parameter with Newton interpolation polynomials of degree one and two. Using this technique in the multidimensional case, we extend the proposed method to systems of nonlinear equations. Numerical tests are presented to verify the theoretical results and a study of the dynamics of the method is applied to different problems to show its stability.This research was supported by PGC2018-095896-B-C22 (MCIU/AEI/FEDER, UE), Generalitat Valenciana PROMETEO/2016/089, and FONDOCYT 2016-2017-212 Republica Dominicana.Cordero Barbero, A.; Maimó, JG.; Torregrosa Sánchez, JR.; Vassileva, MP. (2019). Iterative methods with memory for solving systems of nonlinear equations using a second order approximation. Mathematics. 7(11):1-12. https://doi.org/10.3390/math7111069S112711Soleymani, F., Lotfi, T., Tavakoli, E., & Khaksar Haghani, F. (2015). Several iterative methods with memory using self-accelerators. Applied Mathematics and Computation, 254, 452-458. doi:10.1016/j.amc.2015.01.045Petković, M. S., & Sharma, J. R. (2015). On some efficient derivative-free iterative methods with memory for solving systems of nonlinear equations. Numerical Algorithms, 71(2), 457-474. doi:10.1007/s11075-015-0003-9Narang, M., Bhatia, S., Alshomrani, A. S., & Kanwar, V. (2019). General efficient class of Steffensen type methods with memory for solving systems of nonlinear equations. Journal of Computational and Applied Mathematics, 352, 23-39. doi:10.1016/j.cam.2018.10.048Potra, F. A. (1982). An error analysis for the secant method. Numerische Mathematik, 38(3), 427-445. doi:10.1007/bf01396443Fatou, P. (1919). Sur les équations fonctionnelles. Bulletin de la Société mathématique de France, 2, 161-271. doi:10.24033/bsmf.998Cordero, A., & Torregrosa, J. R. (2007). Variants of Newton’s Method using fifth-order quadrature formulas. Applied Mathematics and Computation, 190(1), 686-698. doi:10.1016/j.amc.2007.01.062Campos, B., Cordero, A., Torregrosa, J. R., & Vindel, P. (2015). A multidimensional dynamical approach to iterative methods with memory. Applied Mathematics and Computation, 271, 701-715. doi:10.1016/j.amc.2015.09.056Chicharro, F. I., Cordero, A., & Torregrosa, J. R. (2013). Drawing Dynamical and Parameters Planes of Iterative Families and Methods. The Scientific World Journal, 2013, 1-11. doi:10.1155/2013/78015

Multidimensional approximation of nonlinear dynamical systems

A key task in the field of modeling and analyzing nonlinear dynamical systems is the recovery of unknown governing equations from measurement data only. There is a wide range of application areas for this important instance of system identification, ranging from industrial engineering and acoustic signal processing to stock market models. In order to find appropriate representations of underlying dynamical systems, various data-driven methods have been proposed by different communities. However, if the given data sets are high-dimensional, then these methods typically suffer from the curse of dimensionality. To significantly reduce the computational costs and storage consumption, we propose the method multidimensional approximation of nonlinear dynamical systems (MANDy) which combines data-driven methods with tensor network decompositions. The efficiency of the introduced approach will be illustrated with the aid of several high-dimensional nonlinear dynamical systems

Impact on stability by the use of memory in Traub-type schemes

[EN] In this work, two Traub-type methods with memory are introduced using accelerating parameters. To obtain schemes with memory, after the inclusion of these parameters in Traub's method, they have been designed using linear approximations or the Newton's interpolation polynomials. In both cases, the parameters use information from the current and the previous iterations, so they define a method with memory. Moreover, they achieve higher order of convergence than Traub's scheme without any additional functional evaluations. The real dynamical analysis verifies that the proposed methods with memory not only converge faster, but they are also more stable than the original scheme. The methods selected by means of this analysis can be applied for solving nonlinear problems with a wider set of initial estimations than their original partners. This fact also involves a lower number of iterations in the process.This research was partially supported by Ministerio de Ciencia, Innovacion y Universidades under grants PGC2018-095896-B-C22 (MCIU/AEI/FEDER/UE).Chicharro, FI.; Cordero Barbero, A.; Garrido, N.; Torregrosa Sánchez, JR. (2020). Impact on stability by the use of memory in Traub-type schemes. Mathematics. 8(2):1-16. https://doi.org/10.3390/math8020274S11682Shacham, M. (1989). An improved memory method for the solution of a nonlinear equation. Chemical Engineering Science, 44(7), 1495-1501. doi:10.1016/0009-2509(89)80026-0Balaji, G. V., & Seader, J. D. (1995). Application of interval Newton’s method to chemical engineering problems. Reliable Computing, 1(3), 215-223. doi:10.1007/bf02385253Shacham, M. (1986). Numerical solution of constrained non-linear algebraic equations. International Journal for Numerical Methods in Engineering, 23(8), 1455-1481. doi:10.1002/nme.1620230805Shacham, M., & Kehat, E. (1973). Converging interval methods for the iterative solution of a non-linear equation. Chemical Engineering Science, 28(12), 2187-2193. doi:10.1016/0009-2509(73)85008-0Amat, S., Busquier, S., & Plaza, S. (2010). Chaotic dynamics of a third-order Newton-type method. Journal of Mathematical Analysis and Applications, 366(1), 24-32. doi:10.1016/j.jmaa.2010.01.047Argyros, I. K., Cordero, A., Magreñán, Á. A., & Torregrosa, J. R. (2017). Third-degree anomalies of Traub’s method. Journal of Computational and Applied Mathematics, 309, 511-521. doi:10.1016/j.cam.2016.01.060Chicharro, F., Cordero, A., Gutiérrez, J. M., & Torregrosa, J. R. (2013). Complex dynamics of derivative-free methods for nonlinear equations. Applied Mathematics and Computation, 219(12), 7023-7035. doi:10.1016/j.amc.2012.12.075Chicharro, F., Cordero, A., & Torregrosa, J. (2015). Dynamics and Fractal Dimension of Steffensen-Type Methods. Algorithms, 8(2), 271-279. doi:10.3390/a8020271Scott, M., Neta, B., & Chun, C. (2011). Basin attractors for various methods. Applied Mathematics and Computation, 218(6), 2584-2599. doi:10.1016/j.amc.2011.07.076Steffensen, J. F. (1933). Remarks on iteration. Scandinavian Actuarial Journal, 1933(1), 64-72. doi:10.1080/03461238.1933.10419209Wang, X., & Zhang, T. (2012). A new family of Newton-type iterative methods with and without memory for solving nonlinear equations. Calcolo, 51(1), 1-15. doi:10.1007/s10092-012-0072-2Džunić, J., & Petković, M. S. (2014). On generalized biparametric multipoint root finding methods with memory. Journal of Computational and Applied Mathematics, 255, 362-375. doi:10.1016/j.cam.2013.05.013Petković, M. S., Neta, B., Petković, L. D., & Džunić, J. (2014). Multipoint methods for solving nonlinear equations: A survey. Applied Mathematics and Computation, 226, 635-660. doi:10.1016/j.amc.2013.10.072Campos, B., Cordero, A., Torregrosa, J. R., & Vindel, P. (2015). A multidimensional dynamical approach to iterative methods with memory. Applied Mathematics and Computation, 271, 701-715. doi:10.1016/j.amc.2015.09.056Chicharro, F. I., Cordero, A., & Torregrosa, J. R. (2019). Dynamics of iterative families with memory based on weight functions procedure. Journal of Computational and Applied Mathematics, 354, 286-298. doi:10.1016/j.cam.2018.01.019Chicharro, F. I., Cordero, A., Torregrosa, J. R., & Vassileva, M. P. (2017). King-Type Derivative-Free Iterative Families: Real and Memory Dynamics. Complexity, 2017, 1-15. doi:10.1155/2017/2713145Magreñán, Á. A., Cordero, A., Gutiérrez, J. M., & Torregrosa, J. R. (2014). Real qualitative behavior of a fourth-order family of iterative methods by using the convergence plane. Mathematics and Computers in Simulation, 105, 49-61. doi:10.1016/j.matcom.2014.04.006Blanchard, P. (1984). Complex analytic dynamics on the Riemann sphere. Bulletin of the American Mathematical Society, 11(1), 85-141. doi:10.1090/s0273-0979-1984-15240-6Magreñán, Á. A. (2014). A new tool to study real dynamics: The convergence plane. Applied Mathematics and Computation, 248, 215-224. doi:10.1016/j.amc.2014.09.061Chicharro, F. I., Cordero, A., & Torregrosa, J. R. (2013). Drawing Dynamical and Parameters Planes of Iterative Families and Methods. The Scientific World Journal, 2013, 1-11. doi:10.1155/2013/78015

Equation-Free Dynamic Renormalization: Self-Similarity in Multidimensional Particle System Dynamics

We present an equation-free dynamic renormalization approach to the computational study of coarse-grained, self-similar dynamic behavior in multidimensional particle systems. The approach is aimed at problems for which evolution equations for coarse-scale observables (e.g. particle density) are not explicitly available. Our illustrative example involves Brownian particles in a 2D Couette flow; marginal and conditional Inverse Cumulative Distribution Functions (ICDFs) constitute the macroscopic observables of the evolving particle distributions.Comment: 7 pages, 5 figure

Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks

We introduce a data-driven forecasting method for high-dimensional chaotic systems using long short-term memory (LSTM) recurrent neural networks. The proposed LSTM neural networks perform inference of high-dimensional dynamical systems in their reduced order space and are shown to be an effective set of nonlinear approximators of their attractor. We demonstrate the forecasting performance of the LSTM and compare it with Gaussian processes (GPs) in time series obtained from the Lorenz 96 system, the Kuramoto-Sivashinsky equation and a prototype climate model. The LSTM networks outperform the GPs in short-term forecasting accuracy in all applications considered. A hybrid architecture, extending the LSTM with a mean stochastic model (MSM-LSTM), is proposed to ensure convergence to the invariant measure. This novel hybrid method is fully data-driven and extends the forecasting capabilities of LSTM networks.Comment: 31 page

The Kernel Polynomial Method

Efficient and stable algorithms for the calculation of spectral quantities and correlation functions are some of the key tools in computational condensed matter physics. In this article we review basic properties and recent developments of Chebyshev expansion based algorithms and the Kernel Polynomial Method. Characterized by a resource consumption that scales linearly with the problem dimension these methods enjoyed growing popularity over the last decade and found broad application not only in physics. Representative examples from the fields of disordered systems, strongly correlated electrons, electron-phonon interaction, and quantum spin systems we discuss in detail. In addition, we illustrate how the Kernel Polynomial Method is successfully embedded into other numerical techniques, such as Cluster Perturbation Theory or Monte Carlo simulation.Comment: 32 pages, 17 figs; revised versio