Numerical Methods I.

A jegyzet az ELTE Informatikai Kar 2015. évi Jegyzetpályázatának támogatásával készült

The numerical solution of sparse matrix equations by fast methods and associated computational techniques

The numerical solution of sparse matrix equations by fast methods and associated computational technique

Dense and sparse parallel linear algebra algorithms on graphics processing units

Una línea de desarrollo seguida en el campo de la supercomputación es el uso de procesadores de propósito específico para acelerar determinados tipos de cálculo. En esta tesis estudiamos el uso de tarjetas gráficas como aceleradores de la computación y lo aplicamos al ámbito del álgebra lineal. En particular trabajamos con la biblioteca SLEPc para resolver problemas de cálculo de autovalores en matrices de gran dimensión, y para aplicar funciones de matrices en los cálculos de aplicaciones científicas. SLEPc es una biblioteca paralela que se basa en el estándar MPI y está desarrollada con la premisa de ser escalable, esto es, de permitir resolver problemas más grandes al aumentar las unidades de procesado. El problema lineal de autovalores, Ax = lambda x en su forma estándar, lo abordamos con el uso de técnicas iterativas, en concreto con métodos de Krylov, con los que calculamos una pequeña porción del espectro de autovalores. Este tipo de algoritmos se basa en generar un subespacio de tamaño reducido (m) en el que proyectar el problema de gran dimensión (n), siendo m << n. Una vez se ha proyectado el problema, se resuelve este mediante métodos directos, que nos proporcionan aproximaciones a los autovalores del problema inicial que queríamos resolver. Las operaciones que se utilizan en la expansión del subespacio varían en función de si los autovalores deseados están en el exterior o en el interior del espectro. En caso de buscar autovalores en el exterior del espectro, la expansión se hace mediante multiplicaciones matriz-vector. Esta operación la realizamos en la GPU, bien mediante el uso de bibliotecas o mediante la creación de funciones que aprovechan la estructura de la matriz. En caso de autovalores en el interior del espectro, la expansión requiere resolver sistemas de ecuaciones lineales. En esta tesis implementamos varios algoritmos para la resolución de sistemas de ecuaciones lineales para el caso específico de matrices con estructura tridiagonal a bloques, que se ejecutan en GPU. En el cálculo de las funciones de matrices hemos de diferenciar entre la aplicación directa de una función sobre una matriz, f(A), y la aplicación de la acción de una función de matriz sobre un vector, f(A)b. El primer caso implica un cálculo denso que limita el tamaño del problema. El segundo permite trabajar con matrices dispersas grandes, y para resolverlo también hacemos uso de métodos de Krylov. La expansión del subespacio se hace mediante multiplicaciones matriz-vector, y hacemos uso de GPUs de la misma forma que al resolver autovalores. En este caso el problema proyectado comienza siendo de tamaño m, pero se incrementa en m en cada reinicio del método. La resolución del problema proyectado se hace aplicando una función de matriz de forma directa. Nosotros hemos implementado varios algoritmos para calcular las funciones de matrices raíz cuadrada y exponencial, en las que el uso de GPUs permite acelerar el cálculo.One line of development followed in the field of supercomputing is the use of specific purpose processors to speed up certain types of computations. In this thesis we study the use of graphics processing units as computer accelerators and apply it to the field of linear algebra. In particular, we work with the SLEPc library to solve large scale eigenvalue problems, and to apply matrix functions in scientific applications. SLEPc is a parallel library based on the MPI standard and is developed with the premise of being scalable, i.e. to allow solving larger problems by increasing the processing units. We address the linear eigenvalue problem, Ax = lambda x in its standard form, using iterative techniques, in particular with Krylov's methods, with which we calculate a small portion of the eigenvalue spectrum. This type of algorithms is based on generating a subspace of reduced size (m) in which to project the large dimension problem (n), being m << n. Once the problem has been projected, it is solved by direct methods, which provide us with approximations of the eigenvalues of the initial problem we wanted to solve. The operations used in the expansion of the subspace vary depending on whether the desired eigenvalues are from the exterior or from the interior of the spectrum. In the case of searching for exterior eigenvalues, the expansion is done by matrix-vector multiplications. We do this on the GPU, either by using libraries or by creating functions that take advantage of the structure of the matrix. In the case of eigenvalues from the interior of the spectrum, the expansion requires solving linear systems of equations. In this thesis we implemented several algorithms to solve linear systems of equations for the specific case of matrices with a block-tridiagonal structure, that are run on GPU. In the computation of matrix functions we have to distinguish between the direct application of a matrix function, f(A), and the action of a matrix function on a vector, f(A)b. The first case involves a dense computation that limits the size of the problem. The second allows us to work with large sparse matrices, and to solve it we also make use of Krylov's methods. The expansion of subspace is done by matrix-vector multiplication, and we use GPUs in the same way as when solving eigenvalues. In this case the projected problem starts being of size m, but it is increased by m on each restart of the method. The solution of the projected problem is done by directly applying a matrix function. We have implemented several algorithms to compute the square root and the exponential matrix functions, in which the use of GPUs allows us to speed up the computation.Una línia de desenvolupament seguida en el camp de la supercomputació és l'ús de processadors de propòsit específic per a accelerar determinats tipus de càlcul. En aquesta tesi estudiem l'ús de targetes gràfiques com a acceleradors de la computació i ho apliquem a l'àmbit de l'àlgebra lineal. En particular treballem amb la biblioteca SLEPc per a resoldre problemes de càlcul d'autovalors en matrius de gran dimensió, i per a aplicar funcions de matrius en els càlculs d'aplicacions científiques. SLEPc és una biblioteca paral·lela que es basa en l'estàndard MPI i està desenvolupada amb la premissa de ser escalable, açò és, de permetre resoldre problemes més grans en augmentar les unitats de processament. El problema lineal d'autovalors, Ax = lambda x en la seua forma estàndard, ho abordem amb l'ús de tècniques iteratives, en concret amb mètodes de Krylov, amb els quals calculem una xicoteta porció de l'espectre d'autovalors. Aquest tipus d'algorismes es basa a generar un subespai de grandària reduïda (m) en el qual projectar el problema de gran dimensió (n), sent m << n. Una vegada s'ha projectat el problema, es resol aquest mitjançant mètodes directes, que ens proporcionen aproximacions als autovalors del problema inicial que volíem resoldre. Les operacions que s'utilitzen en l'expansió del subespai varien en funció de si els autovalors desitjats estan en l'exterior o a l'interior de l'espectre. En cas de cercar autovalors en l'exterior de l'espectre, l'expansió es fa mitjançant multiplicacions matriu-vector. Aquesta operació la realitzem en la GPU, bé mitjançant l'ús de biblioteques o mitjançant la creació de funcions que aprofiten l'estructura de la matriu. En cas d'autovalors a l'interior de l'espectre, l'expansió requereix resoldre sistemes d'equacions lineals. En aquesta tesi implementem diversos algorismes per a la resolució de sistemes d'equacions lineals per al cas específic de matrius amb estructura tridiagonal a blocs, que s'executen en GPU. En el càlcul de les funcions de matrius hem de diferenciar entre l'aplicació directa d'una funció sobre una matriu, f(A), i l'aplicació de l'acció d'una funció de matriu sobre un vector, f(A)b. El primer cas implica un càlcul dens que limita la grandària del problema. El segon permet treballar amb matrius disperses grans, i per a resoldre-ho també fem ús de mètodes de Krylov. L'expansió del subespai es fa mitjançant multiplicacions matriu-vector, i fem ús de GPUs de la mateixa forma que en resoldre autovalors. En aquest cas el problema projectat comença sent de grandària m, però s'incrementa en m en cada reinici del mètode. La resolució del problema projectat es fa aplicant una funció de matriu de forma directa. Nosaltres hem implementat diversos algorismes per a calcular les funcions de matrius arrel quadrada i exponencial, en les quals l'ús de GPUs permet accelerar el càlcul.Lamas Daviña, A. (2018). Dense and sparse parallel linear algebra algorithms on graphics processing units [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/112425TESI

Parallel numerical methods for analysing optical devices with the BPM

In this work, some developments in the theory of modelling integrated optical devices are discussed. The theory of the Beam Propagation Method (BPM) to analyse longitudinal optical waveguides is established. The BPM is then formulated and implemented numerically to study both two and three-dimensional optical waveguides using several Finite-Difference (FD) techniques. For the 2-D analysis, comparisons between the performance of the implicit Crank Nicholson (CN), the explicit Real Space (RS) and the Explicit Finite-Difference (EFD) are made through systematic tests on slab waveguide geometries. For three-dimensional applications, two explicit highly-parallel three-dimensional FD-BPMs (the RS and the EFD) have been implemented on two different parallel computers, namely a transputer array (MIMD type) and a Connection Machine (SIMD type). To assess the performance of parallel computers in this context, serial computer codes for the two methods have been implemented and a comparison between the speed of the serial and parallel codes has been made. Large gains in the speed of the parallel FD-BPMs have been obtained compared to the serial implementations; both methods, in their parallel form, can execute, per propagational step, a large problem containing 106 discretisation points in a few seconds. In addition, a comparison between the performance of the transputer array and the Connection Machine in executing the two FD-BPMs has been discussed. To assess and compare the two methods, three different rib waveguides and three different directional couplers have been analysed and the results compared with published results. It has been concluded from testing these methods that the parallel EFD-BPM is more efficient than the parallel RS-BPM. Then, the linear parallel EFD-BPM was extended to model nonlinear second harmonic generation process in three-dimensional waveguides, where the source field is allowed to deplete, using the transputer array and the Connection Machine

Digital Signal Processing (Second Edition)

This book provides an account of the mathematical background, computational methods and software engineering associated with digital signal processing. The aim has been to provide the reader with the mathematical methods required for signal analysis which are then used to develop models and algorithms for processing digital signals and finally to encourage the reader to design software solutions for Digital Signal Processing (DSP). In this way, the reader is invited to develop a small DSP library that can then be expanded further with a focus on his/her research interests and applications. There are of course many excellent books and software systems available on this subject area. However, in many of these publications, the relationship between the mathematical methods associated with signal analysis and the software available for processing data is not always clear. Either the publications concentrate on mathematical aspects that are not focused on practical programming solutions or elaborate on the software development of solutions in terms of working ‘black-boxes’ without covering the mathematical background and analysis associated with the design of these software solutions. Thus, this book has been written with the aim of giving the reader a technical overview of the mathematics and software associated with the ‘art’ of developing numerical algorithms and designing software solutions for DSP, all of which is built on firm mathematical foundations. For this reason, the work is, by necessity, rather lengthy and covers a wide range of subjects compounded in four principal parts. Part I provides the mathematical background for the analysis of signals, Part II considers the computational techniques (principally those associated with linear algebra and the linear eigenvalue problem) required for array processing and associated analysis (error analysis for example). Part III introduces the reader to the essential elements of software engineering using the C programming language, tailored to those features that are used for developing C functions or modules for building a DSP library. The material associated with parts I, II and III is then used to build up a DSP system by defining a number of ‘problems’ and then addressing the solutions in terms of presenting an appropriate mathematical model, undertaking the necessary analysis, developing an appropriate algorithm and then coding the solution in C. This material forms the basis for part IV of this work. In most chapters, a series of tutorial problems is given for the reader to attempt with answers provided in Appendix A. These problems include theoretical, computational and programming exercises. Part II of this work is relatively long and arguably contains too much material on the computational methods for linear algebra. However, this material and the complementary material on vector and matrix norms forms the computational basis for many methods of digital signal processing. Moreover, this important and widely researched subject area forms the foundations, not only of digital signal processing and control engineering for example, but also of numerical analysis in general. The material presented in this book is based on the lecture notes and supplementary material developed by the author for an advanced Masters course ‘Digital Signal Processing’ which was first established at Cranfield University, Bedford in 1990 and modified when the author moved to De Montfort University, Leicester in 1994. The programmes are still operating at these universities and the material has been used by some 700++ graduates since its establishment and development in the early 1990s. The material was enhanced and developed further when the author moved to the Department of Electronic and Electrical Engineering at Loughborough University in 2003 and now forms part of the Department’s post-graduate programmes in Communication Systems Engineering. The original Masters programme included a taught component covering a period of six months based on two semesters, each Semester being composed of four modules. The material in this work covers the first Semester and its four parts reflect the four modules delivered. The material delivered in the second Semester is published as a companion volume to this work entitled Digital Image Processing, Horwood Publishing, 2005 which covers the mathematical modelling of imaging systems and the techniques that have been developed to process and analyse the data such systems provide. Since the publication of the first edition of this work in 2003, a number of minor changes and some additions have been made. The material on programming and software engineering in Chapters 11 and 12 has been extended. This includes some additions and further solved and supplementary questions which are included throughout the text. Nevertheless, it is worth pointing out, that while every effort has been made by the author and publisher to provide a work that is error free, it is inevitable that typing errors and various ‘bugs’ will occur. If so, and in particular, if the reader starts to suffer from a lack of comprehension over certain aspects of the material (due to errors or otherwise) then he/she should not assume that there is something wrong with themselves, but with the author

Passivity enforcement via chordal methods

Orientador: Prof. Dr. Gustavo Henrique da Costa OliveiraTese (doutorado) - Universidade Federal do Paraná, Setor de Tecnologia, Programa de Pós-Graduação em Engenharia Elétrica. Defesa : Curitiba, 27/08/2019Inclui referências: p. 164-175Resumo: Neste documento são propostos três algoritmos inéditos associados aos problemas subsequentes de aferição e imposição da passividade, a qual é uma propriedade qualitativa, geral e fundamental na modelagem matemática de transitórios eletromagnéticos de sistemas elétricos passivos, como transformadores. Esses algoritmos baseiam-se numa combinação de teoria dos grafos e otimização convexa. O primeiro deles consiste na aferição de subsistemas passivos contidos num sistema não passivo, intuitivamente busca-se partes passivas contidas num todo não passivo. Já na etapa de imposição de passividade, o segundo algoritmo é consequência natural do primeiro: retendo apenas os parâmetros associados às partes passivas e descartando os demais, parte-se de um sistema passivo parcialmente especificado para se determinar novos parâmetros em substituição àqueles descartados de modo que o sistema como um todo seja passivo. A possibilidade de determinação dos novos parâmetros depende de uma propriedade topológica de um grafo associado às matrizes de parâmetros do modelo, tal propriedade é denominada cordalidade. O terceiro algoritmo aborda novamente a questão de imposição da passividade e também faz uso da cordalidade, não mais como condição de existência de solução, mas sim como uma forma de explorar a esparsidade das matrizes de parâmetros. O problema de imposição da passividade encerra dois desafios no seu processo de solução, a saber: (i) compensação de parâmetros resultando na degradação do modelo bem como (ii) longos tempos de solução. Os algoritmos ora propostos são uma resposta a essas questões e os resultados obtidos demonstraram-se comparáveis àqueles já existentes na literatura especializada, em alguns casos apresentando melhorias, seja em termos de aproximação ou tempo computacionais. Os algoritmos foram testados a partir de dados de medição de um Transformador de Potencial Indutivo bem como de um Transformador de Potência. Palavras-chave: Macro-modelagem Passiva. Teoria de Sistemas. Álgebra Linear Aplicada. Análise de Transitórios. Transformadores.Abstract: Three novel algorithms are herein proposed to solve passivity assessment and enforcement problems. Passivity is a general, qualitative and fundamental property pertaining to the modeling associated with electromagnetic transients in passive power systems, such as transformers. These algorithms make combined use of Graph Theory and Convex Optimization. The first algorithm is concerned with passivity assesment. In particular, it searches for passive subsystems embedded into a larger nonpassive system and eventually specifies a partially specified passive system. Focusing on the subsequent step, algorithm two is a natural consequence of the preceeding one: retaining only the parameter set associated with passive subsystems as determined before, this partially specified passive system is used to further determine the remaining parameters so that the entire system be fully specified and passive. The existence condition for finding a fully specified system hinges on the fulfillment of a topological property of the graph associated the parameter matrices, namely chordality. The third algorithm also solves the passivity enforcement problem by making use of chordality, not as an existence condition, but rather by exploiting chordal sparsity patterns obtained with the parameter matrices. Solving passivity enforcement problems entails two persisting challenges, namely: (i) passivity compensations to parameters prompting increased model degradation as well as (ii) large computation times. The algorithms herein proposed tackle these issues and yield results comparable to those already in use, sometimes resulting in improved performance in terms of either approximation accuracy or runtime. These results herein reported entail data from actual measurements of an Inductive Voltage Transformer and a Power Transformer. Keywords: Passive Macromodeling. System Theory. Applied Linear Algebra. Transient Analysis. Transformers

Random matrices

138 pages, based on lectures by Bertrand Eynard at IPhT, SaclayWe provide a self-contained introduction to random matrices. While some applications are mentioned, our main emphasis is on three different approaches to random matrix models: the Coulomb gas method and its interpretation in terms of algebraic geometry, loop equations and their solution using topological recursion, orthogonal polynomials and their relation with integrable systems. Each approach provides its own definition of the spectral curve, a geometric object which encodes all the properties of a model. We also introduce the two peripheral subjects of counting polygonal surfaces, and computing angular integrals