Natural preconditioners for saddle point systems

The solution of quadratic or locally quadratic extremum problems subject to linear(ized) constraints gives rise to linear systems in saddle point form. This is true whether in the continuous or discrete setting, so saddle point systems arising from discretization of partial differential equation problems such as those describing electromagnetic problems or incompressible flow lead to equations with this structure as does, for example, the widely used sequential quadratic programming approach to nonlinear optimization.\ud This article concerns iterative solution methods for these problems and in particular shows how the problem formulation leads to natural preconditioners which guarantee rapid convergence of the relevant iterative methods. These preconditioners are related to the original extremum problem and their effectiveness -- in terms of rapidity of convergence -- is established here via a proof of general bounds on the eigenvalues of the preconditioned saddle point matrix on which iteration convergence depends

A Comparison of Two Different Methods for Solving Biharmonic Boundary Valve Problems

We use the methods of compactly supported radial basis functions (CS-RBFs) and Delta-shaped basis functions (DBFs) to obtain the numerical solution of a two-dimensional biharmonic boundary value problem. The biharmonic equation is difficult to solve due to its existing fourth order derivatives, besides it requires more than one boundary conditions on the same part of the boundary. In this thesis, we use either a one-level or a two-level technique for constructing the approximate solution in the context of Kansa’s collocation method. This thesis will compare the accuracy of the methods of CS-RBFs and DBFs when applied to the biharmonic boundary value problem. Both methods can be used on an irregular shaped domain. Numerical results show that the DBF approach is superior than that of the CS-RBF

A fast semi-direct least squares algorithm for hierarchically block separable matrices

We present a fast algorithm for linear least squares problems governed by hierarchically block separable (HBS) matrices. Such matrices are generally dense but data-sparse and can describe many important operators including those derived from asymptotically smooth radial kernels that are not too oscillatory. The algorithm is based on a recursive skeletonization procedure that exposes this sparsity and solves the dense least squares problem as a larger, equality-constrained, sparse one. It relies on a sparse QR factorization coupled with iterative weighted least squares methods. In essence, our scheme consists of a direct component, comprised of matrix compression and factorization, followed by an iterative component to enforce certain equality constraints. At most two iterations are typically required for problems that are not too ill-conditioned. For an $M \times N$ HBS matrix with $M \geq N$ having bounded off-diagonal block rank, the algorithm has optimal $\mathcal{O} (M + N)$ complexity. If the rank increases with the spatial dimension as is common for operators that are singular at the origin, then this becomes $\mathcal{O} (M + N)$ in 1D, $\mathcal{O} (M + N^{3/2})$ in 2D, and $\mathcal{O} (M + N^{2})$ in 3D. We illustrate the performance of the method on both over- and underdetermined systems in a variety of settings, with an emphasis on radial basis function approximation and efficient updating and downdating.Comment: 24 pages, 8 figures, 6 tables; to appear in SIAM J. Matrix Anal. App

Radial Basis Function Differential Quadrature Method for the Numerical Solution of Partial Differential Equations

In the numerical solution of partial differential equations (PDEs), there is a need for solving large scale problems. The Radial Basis Function Differential Quadrature (RBFDQ) method and local RBF-DQ method are applied for the solutions of boundary value problems in annular domains governed by the Poisson equation, inhomogeneous biharmonic equation, and the inhomogeneous Cauchy-Navier equations of elasticity. By choosing the collocation points properly, linear systems can be obtained so that the coefficient matrices have block circulant structures. The resulting systems can be efficiently solved using matrix decomposition algorithms (MDAs) and fast Fourier transforms (FFTs). For the local RBFDQ method, the MDAs used are modified to account for the sparsity of the arrays involved in the discretization. An adjusted Fasshauer estimate is used to obtain a good shape parameter value in the applied radial basis functions (RBFs) for the global RBF-DQ method while the leave-one-out cross validation (LOOCV) algorithm is employed for the local RBF-DQ method using a sample of local influence domains. A modification of the kdtree algorithm is used to select the nearest centers for each local domain. In several numerical experiments, it is shown that the proposed algorithms are capable of solving large scale problems while maintaining high accuracy

Interactive Medical Image Registration With Multigrid Methods and Bounded Biharmonic Functions

Interactive image registration is important in some medical applications since automatic image registration is often slow and sometimes error-prone. We consider interactive registration methods that incorporate user-specified local transforms around control handles. The deformation between handles is interpolated by some smooth functions, minimizing some variational energies. Besides smoothness, we expect the impact of a control handle to be local. Therefore we choose bounded biharmonic weight functions to blend local transforms, a cutting-edge technique in computer graphics. However, medical images are usually huge, and this technique takes a lot of time that makes itself impracticable for interactive image registration. To expedite this process, we use a multigrid active set method to solve bounded biharmonic functions (BBF). The multigrid approach is for two scenarios, refining the active set from coarse to fine resolutions, and solving the linear systems constrained by working active sets. We\u27ve implemented both weighted Jacobi method and successive over-relaxation (SOR) in the multigrid solver. Since the problem has box constraints, we cannot directly use regular updates in Jacobi and SOR methods. Instead, we choose a descent step size and clamp the update to satisfy the box constraints. We explore the ways to choose step sizes and discuss their relation to the spectral radii of the iteration matrices. The relaxation factors, which are closely related to step sizes, are estimated by analyzing the eigenvalues of the bilaplacian matrices. We give a proof about the termination of our algorithm and provide some theoretical error bounds. Another minor problem we address is to register big images on GPU with limited memory. We\u27ve implemented an image registration algorithm with virtual image slices on GPU. An image slice is treated similarly to a page in virtual memory. We execute a wavefront of subtasks together to reduce the number of data transfers. Our main contribution is a fast multigrid method for interactive medical image registration that uses bounded biharmonic functions to blend local transforms. We report a novel multigrid approach to refine active set quickly and use clamped updates based on weighted Jacobi and SOR. This multigrid method can be used to efficiently solve other quadratic programs that have active sets distributed over continuous regions

Optimising Spatial and Tonal Data for PDE-based Inpainting

Some recent methods for lossy signal and image compression store only a few selected pixels and fill in the missing structures by inpainting with a partial differential equation (PDE). Suitable operators include the Laplacian, the biharmonic operator, and edge-enhancing anisotropic diffusion (EED). The quality of such approaches depends substantially on the selection of the data that is kept. Optimising this data in the domain and codomain gives rise to challenging mathematical problems that shall be addressed in our work. In the 1D case, we prove results that provide insights into the difficulty of this problem, and we give evidence that a splitting into spatial and tonal (i.e. function value) optimisation does hardly deteriorate the results. In the 2D setting, we present generic algorithms that achieve a high reconstruction quality even if the specified data is very sparse. To optimise the spatial data, we use a probabilistic sparsification, followed by a nonlocal pixel exchange that avoids getting trapped in bad local optima. After this spatial optimisation we perform a tonal optimisation that modifies the function values in order to reduce the global reconstruction error. For homogeneous diffusion inpainting, this comes down to a least squares problem for which we prove that it has a unique solution. We demonstrate that it can be found efficiently with a gradient descent approach that is accelerated with fast explicit diffusion (FED) cycles. Our framework allows to specify the desired density of the inpainting mask a priori. Moreover, is more generic than other data optimisation approaches for the sparse inpainting problem, since it can also be extended to nonlinear inpainting operators such as EED. This is exploited to achieve reconstructions with state-of-the-art quality. We also give an extensive literature survey on PDE-based image compression methods

Time Integration Methods of Fundamental Solutions and Approximate Fundamental Solutions for Nonlinear Elliptic Partial Differential Equations

A time-dependent method is coupled with the Method of Approximate Particular Solutions (MAPS) of Delta-shaped basis functions, the Method of Fundamental Solutions (MFS), and the Method of Approximate Fundamental Solutions (MAFS) to solve a second order nonlinear elliptic partial differential equation (PDE) on regular and irregular shaped domains. The nonlinear PDE boundary value problem is first transformed into a time-dependent quasilinear problem by introducing a fictitious time. Forward Euler integration is then used to ultimately convert the problem into a sequence of time-dependent linear nonhomogeneous modified Helmholtz boundary value problems on which the superposition principle is applied to split the numerical solution at each time step into a homogeneous solution and an approximate particular solution. The Crank-Nicholson method is also examined as an option for the numerical integration as opposed to the forward Euler method. A Delta-shaped basis function, which can handle scattered data in various domains, is used to provide an approximation of the source function at each time step and allows for a derivation of an approximate particular solution of the associated nonhomogeneous equation using the MAPS. The corresponding homogeneous boundary value problem is solved using MFS or MAFS. Numerical results support the accuracy and validity of these computational methods. The proposed numerical methods are additionally applied in nonlinear thermal explosion to determine the steady state critical condition in explosive regimes