In this article, an iterative method - Conjugate Gradient Method is used to deal 4 by 4 Wilkinson matrix and 10 by 10 Hilbert matrix during solving ill- conditioned linear systems. The work has been carried out in FORTRAN on new computational facilities. All calculations have been done in double precision. It is found that, this method is very efficient, fast converging and less time consuming

Awasthi, Arun K

Saxena, Vinay

English

Update Publishing (E-Journals)

International Multidisciplinary Research Journal 2012, 2(1):16-17 ISSN: 2231-6302 Available Online: http://irjs.info/    Conjugate gradient method for ill – conditioned linear system  Vinay Saxena* and Arun K Awasthi#  *Department of Mathematics, Kisan PG College, Bahraich, (UP) 271801, India # Sanjivni Institute of Technology & Management, Bahraich, (UP) 271801, India  Abstract  In this article, an iterative method - Conjugate Gradient Method is used to deal 4 by 4 Wilkinson matrix and 10 by 10 Hilbert matrix during solving ill- conditioned linear systems. The work has been carried out in FORTRAN on new computational facilities. All calculations have been done in double precision. It is found that, this method is very efficient, fast converging and less time consuming.  Keywords: Conjugate Gradient Method, Wilkinson Matrix, Hilbert Matrix  INTRODUCTION  Consider the following system of linear equations                                        AX = B            -----(1.1)      where A is n by n matrix, X and B are n-dimensional column vectors.       Our concern is here to yield the solution of (1.1) if A is either 4 by 4 Wilkinson matrix or 10 by 10 Hilbert matrix. That is the coefficient matrix of the system (1.1) is restricted to be only 4 by 4 Wilkinson matrix or 10 by 10 Hilbert matrix. The various direct/ iterative methods for solving linear equations get failed or give solutions which are not good. It is investigated that various factorization methods are failed to solve this problem. Among the iterative methods, Conjugate Gradient Method has been found to be most efficient, fast converging and less time consuming.  METHOD AND ALGORITHM       Before conjugate gradient method, it is necessary to discuss ‘Steepest Descent Method’.  Steepest Descent Method       Using the fact that gradient vector represents the direction of ‘steepest descent’, where direction for minimization will be taken as negative of gradient vector. In this method, starting with an initial trail point ‘X’ and iteratively move towards the optimum point according to the rule-   Xk+1 = Xk + λk Sk = Xk – λk ∇fk    -----(2.1)      where λk is the optimal step length along the search direction           Si = -∇fi      -----(2.2)      The phenomenon of function for solution of linear simultaneous equation is as follows- To minimize the function f(x) defined by           f(X) = ½ XTAX - XTB         -----(2.3)      where B∈Rn and A∈Rn×n is assumed to be positive definite and symmetric. The minimum value of ‘f’ is -(BT A-1B)/2  achieved by setting  X = A-1B. Thus minimizing ‘f’ and solving AX = B are equivalent problems. In the direction of negative gradient           -∇f(Xc) = B – AXc        -----(2.4) The residual at X = Xc is defined as            rc = B – AXc          -----(2.5) If the residual is non-zero, then there exists a positive ‘α’ such that            f(Xc + αrc)<f(Xc)        -----(2.6) The optimal step length λk is the scalar α. At each point Xi, find               Sk = -∇fk          -----(2.7)      To find Xk+1, find the optimal step length λk. For this minimize f(Xk + λk Sk) with respect to λk. Then using (2.1) calculate Xk+1.      This process has to be continued until the optimum point X* is reached. Any one of convergence criteria can be used to terminate the process-      (a) |( f(Xk+1) – f(Xk))/f(Xk)|≤ε1      (b) |Xk+1 - Xk|≤ε2      (c) |Xk+1 - Xk|2  / |Xk+1|2 ≤ε3              The Conjugate Gradient method       The convergence characteristics of ‘steepest descent method’ that can be greatly improved by modifying it into conjugate gradient method. Any minimization method that makes use of conjugate directions is quadratically convergent; this property ensures that the method will minimize a function in ‘n’ steps or less. The following result is important in developing the conjugate gradient method.      If the search directions used in minimization process, S1,S2,…………Sk, are mutually conjugate with respect to A then       ЅkT ∇fi+1 = 0 ,for  k = 1,2, ……,i     -----(2.8)  Procedure       Starting with an initial point X1,calculate the first search   Received: Oct 3, 2011; Revised: Dec 18, 2011; Accepted: Dec 26, 2011  *Corresponding Author  Vinay Saxena  Department of Mathematics, Kisan PG College, Bahraich, (UP) 271801, India   Tel: +91-9415178901 ; Fax:  E-mail: dr.vinaysaxena@gmail.com       International Multidisciplinary Research Journal 2012, 2(1):16-17  17direction     S1 = -∇f(X1) = -∇f1 (say) The point X2 is calculated according as-     X2 =X1+λ1S1 where λ1 is the optimal step length in the direction S1. Now, calculate S2 by formula     Sk = ∇fk + {|∇fk|2 / |∇fk-1|2}Sk-1   -----(2.9)       Now, to find λ2, minimize  f(X2 + λ2 S2)  after getting λ2, calculate     X3 = X2 + λ2 S2 and repeat the above process till the optimum is reached as given by desired accuracy.      The convergence of the conjugate gradient iterates {Xk} depends upon the fact that- “ If  A = I + B  is an  n×n  symmetric positive matrix and  rank(B) = r, then method converges in at most ‘r’ steps”.      The method performs well when A is near the identity either in the sense of a low rank perturbation matrix or in the sense of norm.  Algorithm If A∈Rn×n is symmetric positive definite and B∈Rn, then the following algorithm computes X∈Rn  so AX = B.   (i) start k = 0; X(0) = 0; r(0) = B  (ii) if (r(k) ≠ 0)  (iii) k = k+1  (iv) if k =1   p(1) = r(0)         else   β(k) = r(k-1)T r(k-1) / r(k-2)T r(k-2)   p(k) = r(k-1) + β(k) p(k-1)          (end of if statement)  (v) α(k) = r(k-1)T r(k-1) / p(k)T A p(k)   (vi) x(k) = x(k-1) + α(k) p(k)  (vi) r(k) = r(k-1) – α(k) A p(k)  (vii) for i = 1,n   calculate   eps = {∑(x(i)(k+1) – x(i)(k))2}/ ∑(x(i)(k+1))2  (ix) if  eps  is less than given accuracy   goto(x)         else   goto(ii)  (x) X = X(k)  RESULTS AND DISCUSSION       The equation systems containing 4 by 4 Wilkinson matrix and 10 by 10 Hilbert matrix as coefficient matrix have been experimented in order to estimate the potential of Conjugate Gradient method.  The 4 by 4 Wilkinson matrix       The following matrix is deceptively simple, as it is already in lower triangular form. However, it posses severe computational problems.  This make true solution of AX = B, the vector [1,1,1,1]T.      When this problem is solved by Direct methods, it is observe that factorization methods do not give a good result. This is due to, in the step of traingulization the diagonal element of last row tends to zero. Therefore one of the factors goes to singular. Thus, perhaps all factorization methods are failed to solve this problem.  For this matrix, we have-   A = 2.13   A-1 = 1.15 × 1016   r = 1.46 × 10-14   X - X = 0.00177   (X - X)/X = 0.0017  The 10 by 10 Hilbert matrix  It is defined by as follows- h(i,j) = 1/((i + j) – 1);   i,j = 1,2,……,10. Taking  B = [1,0,0,………,0]T ,       When we try to solve this problem by Direct methods, we observe that elimination process with S.P.P. gives close result while factorization method failed. For this matrix, we have-   A = 2.93   A-1 = 1.21 × 1013   r = 5.12 × 10-19   X - X = 9.61 × 106   (X - X)/X = 2.77 × 10-4  It is observed that Conjugate Gradient method is very efficient, fast converging, less time consuming and gives very good results.  REFERENCES  [1] S. D. Conte and Carl de Boor. 1987, Elementary Numerical Analysis, McGRAW-HILL BOOK COMPANY. [2] G. W. Stewart. 1973. Conjugate Direction Methods for Solving Systems of Linear Equations,  Numer. Math 21,285-297. [3] J. H. Wilkinson and C Reinch,1971. Linear Algebra 2nd of Handbook for Automatic Computation . Springer-Verlag. [4] V. Saxena.1996.Solving Linear System By Direct Vs. Iterative Methods . M.Sc. Thesis(Project), I.I.T. Kanpur. [5] G. Golub and J. M. Ortega. Scientific Computing An Introduction with Parallel Computing. Academic Press, Inc.  

Conjugate gradient method for ill â€“ conditioned linear system

https://updatepublishing.com/journal/index.php/imrj/article/download/1539/1523/1808

Conjugate gradient method for ill â€“ conditioned linear system

Abstract

Similar works

Full text

Available Versions

Update Publishing (E-Journals)

Update Publishing (E-Journals)