Semivariogram methods for modeling Whittle-Mat\'ern priors in Bayesian inverse problems

We present a new technique, based on semivariogram methodology, for obtaining point estimates for use in prior modeling for solving Bayesian inverse problems. This method requires a connection between Gaussian processes with covariance operators defined by the Mat\'ern covariance function and Gaussian processes with precision (inverse-covariance) operators defined by the Green's functions of a class of elliptic stochastic partial differential equations (SPDEs). We present a detailed mathematical description of this connection. We will show that there is an equivalence between these two Gaussian processes when the domain is infinite -- for us, $\mathbb{R}^2$ -- which breaks down when the domain is finite due to the effect of boundary conditions on Green's functions of PDEs. We show how this connection can be re-established using extended domains. We then introduce the semivariogram method for estimating the Mat\'ern covariance parameters, which specify the Gaussian prior needed for stabilizing the inverse problem. Results are extended from the isotropic case to the anisotropic case where the correlation length in one direction is larger than another. Finally, we consider the situation where the correlation length is spatially dependent rather than constant. We implement each method in two-dimensional image inpainting test cases to show that it works on practical examples

Matrix completion with queries

In many applications, e.g., recommender systems and traffic monitoring, the data comes in the form of a matrix that is only partially observed and low rank. A fundamental data-analysis task for these datasets is matrix completion, where the goal is to accurately infer the entries missing from the matrix. Even when the data satisfies the low-rank assumption, classical matrix-completion methods may output completions with significant error -- in that the reconstructed matrix differs significantly from the true underlying matrix. Often, this is due to the fact that the information contained in the observed entries is insufficient. In this work, we address this problem by proposing an active version of matrix completion, where queries can be made to the true underlying matrix. Subsequently, we design Order&Extend, which is the first algorithm to unify a matrix-completion approach and a querying strategy into a single algorithm. Order&Extend is able identify and alleviate insufficient information by judiciously querying a small number of additional entries. In an extensive experimental evaluation on real-world datasets, we demonstrate that our algorithm is efficient and is able to accurately reconstruct the true matrix while asking only a small number of queries.Comment: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Minin

Solving systems of phaseless equations via Kaczmarz methods: A proof of concept study

We study the Kaczmarz methods for solving systems of quadratic equations, i.e., the generalized phase retrieval problem. The methods extend the Kaczmarz methods for solving systems of linear equations by integrating a phase selection heuristic in each iteration and overall have the same per iteration computational complexity. Extensive empirical performance comparisons establish the computational advantages of the Kaczmarz methods over other state-of-the-art phase retrieval algorithms both in terms of the number of measurements needed for successful recovery and in terms of computation time. Preliminary convergence analysis is presented for the randomized Kaczmarz methods

Performance Analysis of Sparse Recovery Based on Constrained Minimal Singular Values

The stability of sparse signal reconstruction is investigated in this paper. We design efficient algorithms to verify the sufficient condition for unique $\ell_1$ sparse recovery. One of our algorithm produces comparable results with the state-of-the-art technique and performs orders of magnitude faster. We show that the $\ell_1$-constrained minimal singular value ($\ell_1$-CMSV) of the measurement matrix determines, in a very concise manner, the recovery performance of $\ell_1$-based algorithms such as the Basis Pursuit, the Dantzig selector, and the LASSO estimator. Compared with performance analysis involving the Restricted Isometry Constant, the arguments in this paper are much less complicated and provide more intuition on the stability of sparse signal recovery. We show also that, with high probability, the subgaussian ensemble generates measurement matrices with $\ell_1$-CMSVs bounded away from zero, as long as the number of measurements is relatively large. To compute the $\ell_1$-CMSV and its lower bound, we design two algorithms based on the interior point algorithm and the semi-definite relaxation