Atomic norm denoising with applications to line spectral estimation

Motivated by recent work on atomic norms in inverse problems, we propose a new approach to line spectral estimation that provides theoretical guarantees for the mean-squared-error (MSE) performance in the presence of noise and without knowledge of the model order. We propose an abstract theory of denoising with atomic norms and specialize this theory to provide a convex optimization problem for estimating the frequencies and phases of a mixture of complex exponentials. We show that the associated convex optimization problem can be solved in polynomial time via semidefinite programming (SDP). We also show that the SDP can be approximated by an l1-regularized least-squares problem that achieves nearly the same error rate as the SDP but can scale to much larger problems. We compare both SDP and l1-based approaches with classical line spectral analysis methods and demonstrate that the SDP outperforms the l1 optimization which outperforms MUSIC, Cadzow's, and Matrix Pencil approaches in terms of MSE over a wide range of signal-to-noise ratios.Comment: 27 pages, 10 figures. A preliminary version of this work appeared in the Proceedings of the 49th Annual Allerton Conference in September 2011. Numerous numerical experiments added to this version in accordance with suggestions by anonymous reviewer

Sensing Matrix Design and Sparse Recovery on the Sphere and the Rotation Group

In this paper, {the goal is to design deterministic sampling patterns on the sphere and the rotation group} and, thereby, construct sensing matrices for sparse recovery of band-limited functions. It is first shown that random sensing matrices, which consists of random samples of Wigner D-functions, satisfy the Restricted Isometry Property (RIP) with proper preconditioning and can be used for sparse recovery on the rotation group. The mutual coherence, however, is used to assess the performance of deterministic and regular sensing matrices. We show that many of widely used regular sampling patterns yield sensing matrices with the worst possible mutual coherence, and therefore are undesirable for sparse recovery. Using tools from angular momentum analysis in quantum mechanics, we provide a new expression for the mutual coherence, which encourages the use of regular elevation samples. We construct low coherence deterministic matrices by fixing the regular samples on the elevation and minimizing the mutual coherence over the azimuth-polarization choice. It is shown that once the elevation sampling is fixed, the mutual coherence has a lower bound that depends only on the elevation samples. This lower bound, however, can be achieved for spherical harmonics, which leads to new sensing matrices with better coherence than other representative regular sampling patterns. This is reflected as well in our numerical experiments where our proposed sampling patterns perfectly match the phase transition of random sampling patterns.Comment: IEEE Trans. on Signal Processin

Weighted eigenfunction estimates with applications to compressed sensing

Using tools from semiclassical analysis, we give weighted L^\infty estimates for eigenfunctions of strictly convex surfaces of revolution. These estimates give rise to new sampling techniques and provide improved bounds on the number of samples necessary for recovering sparse eigenfunction expansions on surfaces of revolution. On the sphere, our estimates imply that any function having an s-sparse expansion in the first N spherical harmonics can be efficiently recovered from its values at m > s N^(1/6) log^4(N) sampling points.Comment: 25 pages, 5 figure

Sparsity Promoting Off-grid Methods with Applications in Direction Finding

University of Minnesota Ph.D. dissertation. May 2017. Major: Electrical/Computer Engineering. Advisor: Mostafa Kaveh. 1 computer file (PDF); x, 99 pages.In this dissertation, the problem of directions-of-arrival (DoA) estimation is studied by the compressed sensing application of sparsity-promoting regularization techniques. Compressed sensing can recover high-dimensional signals with a sparse representation from very few linear measurements by nonlinear optimization. By exploiting the sparse representation for the multiple measurement vectors or the spatial covariance matrix of correlated or uncorrelated sources, the DoA estimation problem can be formulated in the framework of sparse signal recovery with high resolution. There are three main topics covered in this dissertation. These topics are recovery methods for the sparse model with structured perturbations, continuous sparse recovery methods in the super-resolution framework, and the off-grid DoA estimation with array self-calibration. These topics are summarized below. For the first topic, structured perturbation in the sparse model is considered. A major limitation of most methods exploiting sparse spectral models for the purpose of estimating directions-of-arrival stems from the fixed model dictionary that is formed by array response vectors over a discrete search grid of possible directions. In general, the array responses to actual DoAs will most likely not be members of such a dictionary. Thus, the sparse spectral signal model with uncertainty of linearized dictionary parameter mismatch is considered, and the dictionary matrix is reformulated into a multiplication of a fixed base dictionary and a sparse matrix. Based on this sparse model, we propose several convex optimization algorithms. However, we are also concerned with the development of a computationally efficient optimization algorithm for off-grid direction finding using a sparse observation model. With an emphasis on designing efficient algorithms, various sparse problem formulations are considered, such as unconstrained formulation, primal-dual formulation, or conic formulation. But, because of the nature of nondifferentiable objective functions, those problems are still challenging to solve in an efficient way. Thus, the Nesterov smoothing methodology is utilized to reformulate nonsmooth functions into smooth ones, and the accelerated proximal gradient algorithm is adopted to solve the smoothed optimization problem. Convergence analysis is conducted as well. The accuracy and efficiency of smoothed sparse recovery methods are demonstrated for the DoA estimation example. In the second topic, estimation of directions-of-arrival in the spatial covariance model is studied. Unlike the compressed sensing methods which discretize the search domain into possible directions on a grid, the theory of super resolution is applied to estimate DoAs in the continuous domain. We reformulate the spatial spectral covariance model into a multiple measurement vectors (MMV)-like model, and propose a block total variation norm minimization approach, which is the analog of Group Lasso in the super-resolution framework and that promotes the group-sparsity. The DoAs can be estimated by solving its dual problem via semidefinite programming. This gridless recovery approach is verified by simulation results for both uncorrelated and correlated source signals. In the last topic, we consider the array calibration issue for DoA estimation, and extend the previously considered single measurement vector model to multiple measurement vectors. By exploiting multiple measurement snapshots, a modified nuclear norm minimization problem is proposed to recover a low-rank matrix with high probability. The definition of linear operator for the MMV model is given, and its corresponding matrix representation is derived so that a reformulated convex optimization problem can be solved numerically. In order to alleviate computational complexity of the method, we use singular value decomposition (SVD) to reduce the problem size. Furthermore, the structured perturbation in the sparse array self-calibration estimation problem is considered as well. The performance and efficiency of the proposed methods are demonstrated by numerical results

Schnelle Löser für partielle Differentialgleichungen

[no abstract available

Discrete and Continuous Sparse Recovery Methods and Their Applications

Low dimensional signal processing has drawn an increasingly broad amount of attention in the past decade, because prior information about a low-dimensional space can be exploited to aid in the recovery of the signal of interest. Among all the different forms of low di- mensionality, in this dissertation we focus on the synthesis and analysis models of sparse recovery. This dissertation comprises two major topics. For the first topic, we discuss the synthesis model of sparse recovery and consider the dictionary mismatches in the model. We further introduce a continuous sparse recovery to eliminate the existing off-grid mismatches for DOA estimation. In the second topic, we focus on the analysis model, with an emphasis on efficient algorithms and performance analysis. In considering the sparse recovery method with structured dictionary mismatches for the synthesis model, we exploit the joint sparsity between the mismatch parameters and original sparse signal. We demonstrate that by exploiting this information, we can obtain a robust reconstruction under mild conditions on the sensing matrix. This model is very useful for radar and passive array applications. We propose several efficient algorithms to solve the joint sparse recovery problem. Using numerical examples, we demonstrate that our proposed algorithms outperform several methods in the literature. We further extend the mismatch model to a continuous sparse model, using the mathematical theory of super resolution. Statistical analysis shows the robustness of the proposed algorithm. A number-detection algorithm is also proposed for the co-prime arrays. By using numerical examples, we show that continuous sparse recovery further improves the DOA estimation accuracy, over both the joint sparse method and also MUSIC with spatial smoothing. In the second topic, we visit the corresponding analysis model of sparse recovery. Instead of assuming a sparse decomposition of the original signal, the analysis model focuses on the existence of a linear transformation which can make the original signal sparse. In this work we use a monotone version of the fast iterative shrinkage- thresholding algorithm (MFISTA) to yield efficient algorithms to solve the sparse recovery. We examine two widely used relaxation techniques, namely smoothing and decomposition, to relax the optimization. We show that although these two techniques are equivalent in their objective functions, the smoothing technique converges faster than the decomposition technique. We also compute the performance guarantee for the analysis model when a LASSO type of reconstruction is performed. By using numerical examples, we are able to show that the proposed algorithm is more efficient than other state of the art algorithms