Analytical Methods for Structured Matrix Computations

The design of fast algorithms is not only about achieving faster speeds but also about retaining the ability to control the error and numerical stability. This is crucial to the reliability of computed numerical solutions. This dissertation studies topics related to structured matrix computations with an emphasis on their numerical analysis aspects and algorithms. The methods discussed here are all based on rich analytical results that are mathematically justified. In chapter 2, we present a series of comprehensive error analyses to an analytical matrix compression method and it serves as a theoretical explanation of the proxy point method. These results are also important instructions on optimizing the performance. In chapter 3, we propose a non-Hermitian eigensolver by combining HSS matrix techniques with a contour-integral based method. Moreover, probabilistic analysis enables further acceleration of the method in addition to manipulating the HSS representation algebraically. An application of the HSS matrix is discussed in chapter 4 where we design a structured preconditioner for linear systems generated by AIIM. We improve the numerical stability for the matrix-free HSS construction process and make some additional modifications tailored to this particular problem

Approximation and spectral analysis for large structured linear systems.

In this work we are interested in standard and less standard structured linear systems coming from applications in various _elds of computational mathematics and often modeled by integral and/or di_erential equations. Starting from classical Toeplitz and Circulant structures, we consider some extensions as g-Toeplitz and g-Circulants matrices appearing in several contexts in numerical analysis and applications. Then we consider special matrices arising from collocation methods for di_erential equations: also in this case, under suitable assumptions we observe a Toeplitz structure. More in detail we _rst propose a detailed study of singular values and eigenvalues of g-circulant matrices and then we provide an analysis of distribution of g-Toeplitz sequences. Furthermore, when possible, we consider Krylov space methods with special attention to the minimization of the computational work. When the involved dimensions are large, the Preconditioned Conjugate Gradient (PCG) method is recommended because of the much stronger robustness with respect to the propagation of errors. In that case, crucial issues are the convergence speed of this iterative solver, the use of special techniques (preconditioning, multilevel techniques) for accelerating the convergence, and a careful study of the spectral properties of such matrices. Finally, the use of radial basis functions allow of determining and studying the asymptotic behavior of the spectral radii of collocation matrices approximating elliptic boundary value problems

Approximation and spectral analysis for large structured linear systems.

In this work we are interested in standard and less standard structured linear systems coming from applications in various _elds of computational mathematics and often modeled by integral and/or di_erential equations. Starting from classical Toeplitz and Circulant structures, we consider some extensions as g-Toeplitz and g-Circulants matrices appearing in several contexts in numerical analysis and applications. Then we consider special matrices arising from collocation methods for di_erential equations: also in this case, under suitable assumptions we observe a Toeplitz structure. More in detail we _rst propose a detailed study of singular values and eigenvalues of g-circulant matrices and then we provide an analysis of distribution of g-Toeplitz sequences. Furthermore, when possible, we consider Krylov space methods with special attention to the minimization of the computational work. When the involved dimensions are large, the Preconditioned Conjugate Gradient (PCG) method is recommended because of the much stronger robustness with respect to the propagation of errors. In that case, crucial issues are the convergence speed of this iterative solver, the use of special techniques (preconditioning, multilevel techniques) for accelerating the convergence, and a careful study of the spectral properties of such matrices. Finally, the use of radial basis functions allow of determining and studying the asymptotic behavior of the spectral radii of collocation matrices approximating elliptic boundary value problems

Arnoldi decomposition, GMRES, and preconditioning for linear discrete ill-posed problems

GMRES is one of the most popular iterative methods for the solution of large linear systems of equations that arise from the discretization of linear well-posed problems, such as Dirichlet boundary value problems for elliptic partial differential equations. The method is also applied to iteratively solve linear systems of equations that are obtained by discretizing linear ill-posed problems, such as many inverse problems. However, GMRES does not always perform well when applied to the latter kind of problems. This paper seeks to shed some light on reasons for the poor performance of GMRES in certain situations, and discusses some remedies based on specific kinds of preconditioning. The standard implementation of GMRES is based on the Arnoldi process, which also can be used to define a solution subspace for Tikhonov or TSVD regularization, giving rise to the Arnoldi-Tikhonov and Arnoldi-TSVD methods, respectively. The performance of the GMRES, the Arnoldi-Tikhonov, and the Arnoldi-TSVD methods is discussed. Numerical examples illustrate properties of these methods

Preconditioning for radial basis function partition of unity methods

Meshfree radial basis function (RBF) methods are of interest for solving partial differential equations due to attractive convergence properties, flexibility with respect to geometry, and ease of implementation. For global RBF methods, the computational cost grows rapidly with dimension and problem size, so localised approaches, such as partition of unity or stencil based RBF methods, are currently being developed. An RBF partition of unity method (RBF--PUM) approximates functions through a combination of local RBF approximations. The linear systems that arise are locally unstructured, but with a global structure due to the partitioning of the domain. Due to the sparsity of the matrices, for large scale problems, iterative solution methods are needed both for computational reasons and to reduce memory requirements. In this paper we implement and test different algebraic preconditioning strategies based on the structure of the matrix in combination with incomplete factorisations. We compare their performance for different orderings and problem settings and find that a no-fill incomplete factorisation of the central band of the original discretisation matrix provides a robust and efficient preconditioner

On the asymptotic rate of convergence of Stochastic Newton algorithms and their Weighted Averaged versions

The majority of machine learning methods can be regarded as the minimization of an unavailable risk function. To optimize the latter, given samples provided in a streaming fashion, we define a general stochastic Newton algorithm and its weighted average version. In several use cases, both implementations will be shown not to require the inversion of a Hessian estimate at each iteration, but a direct update of the estimate of the inverse Hessian instead will be favored. This generalizes a trick introduced in [2] for the specific case of logistic regression, by directly updating the estimate of the inverse Hessian. Under mild assumptions such as local strong convexity at the optimum, we establish almost sure convergences and rates of convergence of the algorithms, as well as central limit theorems for the constructed parameter estimates. The unified framework considered in this paper covers the case of linear, logistic or softmax regressions to name a few. Numerical experiments on simulated data give the empirical evidence of the pertinence of the proposed methods, which outperform popular competitors particularly in case of bad initializa-tions.Comment: Computational Optimization and Applications, 202

GMRES-Accelerated ADMM for Quadratic Objectives

We consider the sequence acceleration problem for the alternating direction method-of-multipliers (ADMM) applied to a class of equality-constrained problems with strongly convex quadratic objectives, which frequently arise as the Newton subproblem of interior-point methods. Within this context, the ADMM update equations are linear, the iterates are confined within a Krylov subspace, and the General Minimum RESidual (GMRES) algorithm is optimal in its ability to accelerate convergence. The basic ADMM method solves a $\kappa$-conditioned problem in $O(\sqrt{\kappa})$ iterations. We give theoretical justification and numerical evidence that the GMRES-accelerated variant consistently solves the same problem in $O(\kappa^{1/4})$ iterations for an order-of-magnitude reduction in iterations, despite a worst-case bound of $O(\sqrt{\kappa})$ iterations. The method is shown to be competitive against standard preconditioned Krylov subspace methods for saddle-point problems. The method is embedded within SeDuMi, a popular open-source solver for conic optimization written in MATLAB, and used to solve many large-scale semidefinite programs with error that decreases like $O(1/k^{2})$, instead of $O(1/k)$, where $k$ is the iteration index.Comment: 31 pages, 7 figures. Accepted for publication in SIAM Journal on Optimization (SIOPT

Fast Microwave Tomography Algorithm for Breast Cancer Imaging

Microwave tomography has shown promise for breast cancer imaging. The microwaves are harmless to body tissues, which makes microwave tomography a safe adjuvant screening to mammography. Although many clinical studies have shown the effectiveness of regular screening for the detection of breast cancer, the anatomy of the breast and its critical tissues challenge the identification and diagnosis of tumors in this region. Detection of tumors in the breast is more challenging in heterogeneously dense and extremely dense breasts, and microwave tomography has the potential to be effective in such cases. The sensitivity of microwaves to various breast tissues and the comfort and safety of the screening method have made microwave tomography an attractive imaging technique. Despite the need for an alternative screening technique, microwave tomography has not yet been introduced as a screening modality in regular health care, and is still subject to research. The main obstacles are imperfect hardware systems and inefficient imaging algorithms. The immense computational costs for the image reconstruction algorithm present a crucial challenge. 2D imaging algorithms are proposed to reduce the amount of hardware resources required and the imaging time. Although 2D microwave tomography algorithms are computationally less expensive, few imaging groups have been successful in integrating the acquired 3D data into the 2D tomography algorithms for clinical applications. The microwave tomography algorithms include two main computation problems: the forward problem and the inverse problem. The first part of this thesis focuses on a new fast forward solver, the 2D discrete dipole approximation (DDA), which is formulated and modeled. The effect of frequency, sampling number, target size, and contrast on the accuracy of the solver are studied. Additionally, the 2D DDA time efficiency and computation time as a single forward solver are investigated.\ua0 The second part of this thesis focuses on the inverse problem. This portion of the algorithm is based on a log-magnitude and phase transformation optimization problem and is formulated as the Gauss-Newton iterative algorithm. The synthetic data from a finite-element-based solver (COMSOL Multiphysics) and the experimental data acquired from the breast imaging system at Chalmers University of Technology are used to evaluate the DDA-based image reconstruction algorithm. The investigations of modeling and computational complexity show that the 2D DDA is a fast and accurate forward solver that can be embedded in tomography algorithms to produce images in seconds. The successful development and implementation in this thesis of 2D tomographic breast imaging with acceptable accuracy and high computational cost efficiency has provided significant savings in time and in-use memory and is a dramatic improvement over previous implementations

Decay properties of spectral projectors with applications to electronic structure

Motivated by applications in quantum chemistry and solid state physics, we apply general results from approximation theory and matrix analysis to the study of the decay properties of spectral projectors associated with large and sparse Hermitian matrices. Our theory leads to a rigorous proof of the exponential off-diagonal decay ("nearsightedness") for the density matrix of gapped systems at zero electronic temperature in both orthogonal and non-orthogonal representations, thus providing a firm theoretical basis for the possibility of linear scaling methods in electronic structure calculations for non-metallic systems. We further discuss the case of density matrices for metallic systems at positive electronic temperature. A few other possible applications are also discussed.Comment: 63 pages, 13 figure