Parallel and sequential Kaczmarz methods for solving underdetermined nonlinear equations

AbstractWe analyze the convergence of iterative process in Rn, of the type xk + 1 =Φ(xk, wk). Using this theory we prove the local convergence of a sequential Kaczmarz type method and a parallel Kaczmarz type method for solving underdetermined systems of nonlinear equations. Some numerical experiences are presented

Novos resultados sobre formulas secantes e aplicações

Orientador: Jose Mario MartinezTese (doutorado) - Universidade Estadual de Campinas, Instituto de Matematica, Estatistica e Computação CientíficaResumo: Não informado.Abstract: Not informed.DoutoradoDoutor em Matemática Aplicad

Novos resultados sobre formulas secantes e aplicações

Tese (doutorado) - Universidade Estadual de Campinas. Instituto de Matematica Estatistica e Ciencias da Computaçã

A Quasi-newton Method With Modification Of One Column Per Iteration

In this paper we introduce a new Quasi-Newton method for solving nonlinear simultaneous equations. At each iteration only one column of Bk is changed to obtain Bk+1. This permits to use the well-known techniques of Linear Programming for modifying the factorization of Bk. We present a local convergence theorem for a restarted version of the method. The new algorithm is compared numerically with some other methods which were introduced for solving the same kind of problems. © 1984 Springer-Verlag.333-435336

Analytical study of the Least Squares Quasi-Newton method for interaction problems

Often in nature different systems interact, like fluids and structures, heat and electricity, populations of species, etc. It is our aim in this thesis to find, describe and analyze solution methods to solve the equations resulting from the mathematical models describing those interacting systems. Even if powerful solvers often already exist for problems in a single physical domain (e.g. structural or fluid problems), the development of similar tools for multi-physics problems is still ongoing. When the interaction (or coupling) between the two systems is strong, many methods still fail or are computationally very expensive. Approaches for solving these multi-physics problems can be broadly put in two categories: monolithic or partitioned. While we are not claiming that the partitioned approach is panacea for all coupled problems, we will only focus our attention in this thesis on studying methods to solve (strongly) coupled problems with a partitioned approach in which each of the physical problems is solved with a specialized code that we consider to be a black box solver and of which the Jacobian is unknown. We also assume that calling these black boxes is the most expensive part of any algorithm, so that performance is judged by the number of times these are called. In 2005 Vierendeels presented a new coupling procedure for this partitioned approach in a fluid-structure interaction context, based on sensitivity analysis of the important displacement and pressure modes which are detected during the iteration process. This approach only uses input-output couples of the solvers (one for the fluid problem and one for the structural problem). In this thesis we will focus on establishing the properties of this method and show that it can be interpreted as a block quasi-Newton method with approximate Jacobians based on a least squares formulation. We also establish and investigate other algorithms that exploit the original idea but use a single approximate Jacobian. The main focus in this thesis lies on establishing the algebraic properties of the methods under investigation and not so much on the best implementation form

A study about solving non-linear systems : theoretical perspectives and applications

Orientador: José Mario Martínez PérezTese (doutorado) - Universidade Estadual de Campinas, Instituto de Matemática Estatística e Computação CientíficaResumo: O objetivo deste trabalho é estudar e analisar diferentes abordagens para resolver sistemas não lineares. Em primeiro lugar, uma versão esparsa do método de Newton é aplicada para encontrar uma solução do problema de complementaridade horizontal não linear (HNCP) associado a uma solução viável do problema de programação matemática com restrições de complementaridade (MPCC). O algoritmo combina direções do tipo Newton e Gradientes Projetados com um procedimento de busca linear que garante convergência global a um ponto estacionário da função de mérito associada a este problema. Convergência local quadrática é estabelecida sob hipóteses razoáveis. Experiência numérica em problemas teste de uma coleção bem conhecida ilustra a eficiência do algoritmo para encontrar soluções viáveis de MPCC na prática. Em seguida, uma estratégia quase-Newton para acelerar a convergência de iterações de ponto fixo é analisada. Para isso, atualizações secantes clássicas são consideradas. Experimentos numéricos em um conjunto treino são desenvolvidos, a fim de validar esta estratégia. Posteriormente, a estratégia quase-Newton é aplicada ao problema prático de representar o comportamento cinético de um marcador PET (Tomografia por Emissão de Pósitrons) durante a perfusão cardí­aca. O desempenho do método quando aplicado a problemas com dados reais é ilustrado numericamente. Finalmente, um método hí­brido que combina direções de Newton e Homotopia é introduzido para resolver problemas onde o método de Newton apresenta dificuldades. Experimentos iniciais constituem uma base para validação da técnica apresentadaAbstract: The aim of this work is to study and analyse different approaches for solving nonlinear systems. First of all, a sparse version of Newton's method is applied for finding a solution of a horizontal nonlinear complementarity problem (HNCP) associated to a feasible solution of a mathematical programming problem with complementarity constraints (MPCC). The algorithm combines Newton-like and Projected-Gradient directions with a line-search procedure that guarantees global convergence to a stationary point of the merit function associated to this problem. Local quadratic convergence is stated under reasonable hypothesis. Numerical experience on test problems from a well-known collection illustrates the efficiency of the algorithm to find feasible solutions of MPCC in practice. Next, a quasi-Newton strategy for accelerating the convergence of fixed-point iterations is analysed. For that, classical secant updates are considered. Numerical experiments on a training set are developed in order to validate this strategy. After that, the quasi-Newton strategy is applied on the practical problem of represent the kinetic behavior of a PET (Positron Emission Tomography) tracer during cardiac perfusion. The performance of the method when applied to real data problems is illustrated numerically. Finally, a hybrid method combining Newton and Homotopy directions is introduced for solving problems where Newton's method presented difficulties. Initial experiments provide a basis for the presented technic validationDoutoradoMatematica AplicadaDoutora em Matemática Aplicada2012/10444-0FAPESPCAPE

A Globally Convergent Inexact Newton Method With A New Choice For The Forcing Term

In inexact Newton methods for solving nonlinear systems of equations, an approximation to the step s k of the Newton's system J(x k )s=-F(x k ) is found. This means that s k must satisfy a condition like F(x k )+J(x k )s k η k F(x k ) for a forcing term η k [0,1). Possible choices for η k have already been presented. In this work, a new choice for η k is proposed. The method is globalized using a robust backtracking strategy proposed by Birgin et al. (Numerical Algorithms 32:249-260, 2003), and its convergence properties are proved. Several numerical experiments with boundary value problems are presented. The numerical performance of the proposed algorithm is analyzed by the performance profile tool proposed by Dolan and Moré (Mathematical Programming Series A 91:201-213, 2002). The results obtained show a competitive inexact Newton method for solving academic and applied problems in several areas. © 2007 Springer Science+Business Media, LLC.1571193205Averick, B.M., Carter, R.G., Moré, J.J., Xue, G.-L., (1992) The minpack-2 Test Problem Collection, , Preprint MCS-P153-0692, Mathematics and Computer Science Division, Argonne National LaboratoryBirgin, E.G., Krejić, N., Martínez, J.M., Globally convergent inexact quasi-Newton methods for solving nonlinear systems (2003) Numerical Algorithms, 32, pp. 249-260Bleistein, N., (1984) Mathematical Methods for Wave Phenomena, , Academic San DiegoBriggs, W.L., Henson, V.E., McCormick, S.F., (2000) A Multigrid Tutorial, , 2 SIAM PhiladelphiaBrown, P.N., Saad, Y., Hybrid Krylov methods for nonlinear systems of equations (1990) SIAM Journal on Scientific and Statistical Computing, 11, pp. 450-481. , 3Broyden, C.G., A class of methods for solving sparse nonlinear systems (1965) Mathematics of Computation, 25, pp. 285-294Broyden, C.G., Dennis Jr., J.E., Moré, J.J., On the local and superlinear convergence of quasi-Newton methods (1973) Journal of the Institute of Mathematical Applications, 12, pp. 223-245Dembo, R.S., Eisenstat, S.C., Steihaug, T., Inexact Newton methods (1982) SIAM Journal on Numerical Analysis, 19, pp. 401-408. , 2Dennis Jr., J.E., Schnabel, R.B., Numerical methods for unconstrained optimization and nonlinear equations (1996) SIAM Classics in Applied MathematicsDiniz-Ehrhardt, M.A., Gomes-Ruggiero, M.A., Lopes, V.L.R., Martínez, J.M., Discrete Newton's method with local variations for solving large-scale nonlinear systems (2003) Optimization, 52, pp. 417-440. , 4-5Dolan, E.D., Moré, J.J., Benchmarking optimization software with performance profiles (2002) Mathematical Programming Series A, 91, pp. 201-213Eisenstat, S.C., Walker, H.F., Globally convergent inexact newton methods (1994) SIAM Journal Optimization, 4, pp. 393-422. , 2Eisenstat, S.C., Walker, H.F., Choosing the forcing terms in inexact newton method (1996) SIAM Journal on Scientific Computing, 17, pp. 16-32. , 1Gomes-Ruggiero, M.A., Martínez, J.M., The column-updating method for solving nonlinear equations in Hilbert space (1992) Mathematical Modeling and Numerical Analysis, 26, pp. 309-330. , 2Gomes-Ruggiero, M.A., Martínez, J.M., Moretti, A.C., Comparing algorithms for solving sparse nonlinear systems of equations (1992) SIAM Journal Scientific and Statistical Computing, 13, pp. 459-483. , 2Gomes-Ruggiero, M.A., Kozakevich, D.N., Martínez, J.M., A numerical study on large-scale nonlinear solver (1996) Computers and Mathematics with Applications, 32, pp. 1-13. , 3Kelley, C.T., (1995) Iterative Methods for Linear and Nonlinear Equations, , SIAM PhiladelphiaLi, D.-H., Fukushima, M., Derivative-free line search and global convergence of Broyden-like method for nonlinear equations (2000) Optimization Methods and Software, 13, pp. 181-201Lopes, V.L.R., Martínez, J.M., Convergence properties of the inverse column-updating method (1995) Optimization Methods and Software, 6, pp. 127-144Martínez, J.M., A quasi-Newton method with modification of one column per iteration (1984) Computing, 33, pp. 353-362Martínez, J.M., Zambaldi, M.C., An inverse column-updating method for solving large-scale nonlinear systems of equations (1992) Optimization Methods and Software, 1, pp. 129-140Pernice, M., Walker, H.F., NITSOL: A Newton iterative solver for nonlinear systems (1998) SIAM Journal on Scientific Computing, 19, pp. 302-318. , 1Saad, Y., Schultz, M.H., GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems (1986) SIAM Journal on Scientific and Statistical Computing, 7, pp. 856-869. , 3Toledo-Benavides, J.V., (2005) Um Método Newton-GMRES Globalmente Convergente Com um Nova Escolha Para o Termo Forçante e Algumas Estratégias Para Melhorar o Desempenho de GMRES( M ), , PhD Thesis, Department of Applied Mathematics, State University of Campinas Imecc-Unicamp, T/Unicamp T575

An Extension Of The Theory Of Secant Preconditioners

A theory of inexact Newton methods with secant preconditioners for solving large nonlinear systems of equations has been developed recently by Martínez (Math. Comput., 1993). According to this theory, local and superlinear convergence with bounded work per iteration of the inexact Newton method is obtained if the first trial increment at each iteration is a suitable quasi-Newton step computed using least-change secant-update procedures. The Jacobian approximation is interpreted as a preconditioner of the iterative linear method. In this paper, we extend the theory in two ways. On the one hand, since in many iterative methods the true residual is not computed but the preconditioned residual is, we show how to stop the linear iteration using the preconditioned residual instead of the original one. On the other hand, we introduce damping parameters that modify the usual unitary secant step. Two natural damping parameters are introduced, one of them tries to reduce the true residual and the other one tries to reduce the preconditioned residual. We prove that the main results of the theory of secant preconditioners hold under these modifications. © 1995.601-2115125Broyden, A class of methods for solving nonlinear simultaneous equations (1965) Mathematics of Computation, 19, pp. 577-593Broyden, Dennis, Jr., Moré, On the local and superlinear convergence of quasi-Newton methods (1973) J. Inst. Math. Appl., 12, pp. 223-245Dembo, Eisenstat, Steihaug, Inexact Newton Methods (1982) SIAM Journal on Numerical Analysis, 19, pp. 400-408Dennis, Jr., Moré, A characterization of superlinear convergence and its application to quasi-Newton methods (1974) Mathematics of Computation, 28, pp. 549-560Dennis, Jr., Schnabel, Least Change Secant Updates for Quasi-Newton Methods (1979) SIAM Review, 21, pp. 443-459Dennis, Jr., Schnabel, (1983) Numerical Methods for Unconstrained Optimization and Nonlinear Equations, , Prentice-Hall, Englewood Cliffs, NJDennis, Jr., Walker, Convergence Theorems for Least-Change Secant Update Methods (1981) SIAM Journal on Numerical Analysis, 18, pp. 949-987Deuflhard, Global inexact Newton methods for very large scale nonlinear problems (1991) Impact Comput. Sci. Eng., 3, pp. 366-393Deuflhard, Freund, Walter, Fast secant methods for the iterative solution of large nonsymmetric linear systems (1990) Impact Comput. Sci. Eng., 2, pp. 244-276Duff, Erisman, Reid, (1986) Direct Methods for Sparse Matrices, , Clarendon Press, OxfordS.C. Eisenstat and H.F. Walker, Globally convergent inexact Newton methods, SIAM J. Optim., to appearGeorge, Ng, Symbolic Factorization for Sparse Gaussian Elimination with Partial Pivoting (1987) SIAM Journal on Scientific and Statistical Computing, 8, pp. 877-898Golub, Van Loan, (1989) Matrix Computations, , The Johns Hopkins Univ. Press, Baltimore and LondonGomes-Ruggiero, Martínez, The column-updating method for solving nonlinear equations in Hilbert space (1992) RAIRO Math. Modelling Numer. Anal., 26, pp. 309-330Gomes-Ruggiero, Martínez, Moretti, Comparing Algorithms for Solving Sparse Nonlinear Systems of Equations (1992) SIAM Journal on Scientific and Statistical Computing, 13, pp. 459-483Hestenes, Stiefel, Methods of conjugate gradients for solving linear systems (1952) Journal of Research of the National Bureau of Standards, 49 B, pp. 409-436D.N. Kozakevich, J.M. Martínez and M.C. Zambaldi, Experiments using secant preconditioners, in preparationMartínez, A quasi-Newton method with modification of one column per iteration (1984) Computing, 33, pp. 353-362Martínez, Local Convergence Theory of Inexact Newton Methods Based on Structured Least Change Updates (1990) Mathematics of Computation, 55, pp. 143-168Martínez, On the Relation between Two Local Convergence Theories of Least-Change Secant Update Methods (1992) Mathematics of Computation, 59, pp. 457-481Martínez, A Theory of Secant Preconditioners (1993) Mathematics of Computation, 60, pp. 681-698Martínez, On the convergence of the column-updating method (1993) Mat. Apl. Comput., 12, pp. 83-94Martínez, Qi, Inexact Newton methods for solving nonsmooth equations (1993) Relatório de Pesquisa 67/93, , Instituto de Matemática, Universidade Estadual de Campinas, BrazilMartínez, Zambaldi, An inverse column-updating method for solving large-scale nonlinear systems of equations (1991) Optimization Methods and Software, 1, pp. 129-140Nash, Newton-type minimization via the Lanczos method (1984) SIAM J. Numer. Anal., 21, pp. 770-787Nash, Preconditioning of Truncated-Newton Methods (1985) SIAM Journal on Scientific and Statistical Computing, 6, pp. 599-616Nazareth, Nocedal, A study of conjugate-gradient methods (1978) Tech. Report SOL 78-29, , Department of Operations Research, Stanford UniversityOrtega, Rheinboldt, (1970) Interative Solution of Nonlinear Equations in Several Variables, , Academic Press, New YorkOstrowski, (1973) Solution of Equations in Euclidean and Banach Spaces, , Academic Press, New YorkSaad, Schultz, GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems (1986) SIAM Journal on Scientific and Statistical Computing, 7, pp. 856-869Schwetlick, (1978) Numerische Lösung Nichtlinearer Gleichungen, , Deutscher Verlag der Wissenschaften, BerlinYpma, Local convergence of inexact Newton methods (1984) SIAM J. Numer. Anal., 21, pp. 583-59