Applying the $\chi^2$ Regularization Parameter Estimator by Downsampling Through Relations with The Singular Value Expansion

Abstract

The solution, $x$, of the linear system of equations $A x\approx b$ arising from the discretization of an ill-posed integral equation with a square integrable kernel $H(s,t)$ is considered. The Tikhonov regularized solution $x(\lambda)$ is found as the minimizer of $J( x)=\{ \|A x - b\|_2^2 + \lambda^2 \|L x\|_2^2\}$. $x(\lambda)$ depends on regularization parameter $\lambda$ that trades off the data fidelity, and on the smoothing norm determined by $L$. Here we consider the case where $L$ is diagonal and invertible, and employ the Galerkin method to provide the relationship between the singular value expansion and the singular value decomposition for square integrable kernels. The resulting approximation of the integral equation permits examination of the properties of the regularized solution $x(\lambda)$ independent of the sample size of the data. We prove that estimation of the regularization parameter can be obtained by consistently down sampling the data and the system matrix, leading to solutions of coarse to fine grained resolution. Hence, the estimate of $\lambda$ for a large problem may be found by downsampling to a smaller problem, or to a set of smaller problems, effectively moving the costly estimate of the regularization parameter to the coarse representation of the problem. Moreover, the full singular value decomposition for the fine scale system is replaced by a number of dominant terms which is determined from the coarse resolution system, again reducing the computational cost. Numerical results illustrate the theory and demonstrate the practicality of the approach for regularization parameter estimation using generalized cross validation, unbiased predictive risk estimation and the discrepancy principle applied for both the system of equations, and the augmented system of equations