Estimation of sparse distributions

Research Doctorate - Doctor of Philosophy (PhD)A vector is called sparse when most of its components are zero. Many natural signals admit sparse representations with respect to some bases. This dissertation deals with the problem of recovering a sparse signal from a small number of measurements formed by computing the inner product of the signal with the rows of a measurement matrix. The task of recovering a sparse signal from its small number of measurements is a problem of finding the sparsest solution of a underdetermined system of linear equations. The problem can be solved by using a l0 norm based optimization problem which is NP-hard in general. Some recent researches have demonstrated that the NP-hard problem can be solved tractably by using linear programming approach under some reasonable conditions. However, real-world applications often demand more efficient algorithms which are robust to measurement noise. To this aim, an efficient sparse signal recovery algorithm is developed in the first part of the thesis. The proposed algorithm uses a convex-concave procedure to optimize its cost function. A range of theoretical results are presented. The theoretical analysis of the algorithm gives a bound on the run-time estimation error. In many real world problems the resulting sparse signals exhibit additional structures. The proposed algorithm is then extended to exploit the structures of sparse signal. Experimental results demonstrated that in most settings the extended algorithm outperforms other conventional algorithms with a large margin. Finally an interesting sparse signal recovery approach is considered when a part of the support of the sparse signal is known in advance. A maximum a posteriori (MAP) estimation framework is considered to deal with the issue. Second part of the thesis explores the applicability of the proposed algorithms for solving some practical problems. In some cases, the proposed algorithm exhibits additional advantages. For example, in the broadband direction of arrival (DOA) estimation problem, the proposed algorithm allows relaxing the half-wavelength sensor spacing restriction, which leads to a significant performance improvement. In some applications, the proposed algorithm can exploit underlying structure of the linear system. For example, in frequency estimation problem, it is possible to exploit the structure of the Fourier basis to achieve a significant reduction of the computational complexity

Sparse sequence recovery via a maximum a posteriori estimation

A maximum a posteriori (MAP) estimation algorithm is given for reconstructing sparse signals, where a part of the support, and an approximate estimate of the sparse signal are known. This method is useful, e.g., in magnetic resonance image (MRI) sequence, natural video sequences, etc, where it is required to recursively reconstruct a sequence of mutually correlated sparse signals or images. Here we use the last signal as an a priori estimate of the current signal. The priori information is often inaccurate, and we adopt MAP estimation framework to deal with this issue. Simulation studies are performed, and the algorithm is applied to reconstruct MRI image sequences

A robust algorithm for joint-sparse recovery

We address the problem of finding a set of sparse signals that have nonzero coefficients in the same locations from a set of their compressed measurements. A mixed lscr₂,₀ norm optimization approach is considered. A cost function appropriate to the joint-sparse problem is developed, and an algorithm is derived. Compared to other convex relaxation based techniques, the results obtained by the proposed method show a clear improvement in both noiseless and noisy environments

Maximum a posteriori based approach for target detection in MTI radar

We propose a sparse recovery approach to detect moving targets in clutter. In presence of clutter, the target space is not sparse. We propose a simple way to estimate the clutter region. We then enforce sparsity by modeling the clutter as a single extended cluster of nonzero components. This done by solving a sparse signal recovery problem with partially known support within a maximum <i>a posteriori</i> estimation framework. The resulting algorithm is applied in angle-Doppler imaging for moving target indication in an airborne radar. Our approach has a number of advantages including improved robustness to noise and increased resolution with limited data

An L0 norm based method for frequency estimation from irregularly sampled data

We present a frequency estimation method based on a sparse representation of irregular samples with an overcomplete basis. We enforce sparsity by imposing penalties based on an approximate ℓ₀-norm. A number of recent theoretical results on compressed sensing justify this choice. Explicitly enforcing the sparsity of the representation is motivated by a desire to obtain a sharp estimate of the frequency spectrum that exhibits super-resolution. Our formulation leads to an optimization problem, which we solve efficiently in an iterative algorithm. The simulation results demonstrate that that the proposed algorithm outperforms several other state-of-art methods

A fast decoder for compressed sensing based multiple description image coding

Multiple description coding (MDC) offers an elegant approach to data transmission over lossy packet-based networks. This paper proposes an MDC decoder for Compressed Sensing (CS) based MDC. Our decoder minimizes ℓ⁰ norm of the total variation of the image in a recursive manner, making it effective when different descriptions experience different time delays in the network. The proposed approach brings in a significant performance improvement in reconstruction accuracy and reconstruction time

An improved smoothed ℓ⁰ approximation algorithm for sparse representation

ℓ⁰ norm based algorithms have numerous potential applications where a sparse signal is recovered from a small number of measurements. The direct ℓ⁰ norm optimization problem is P-hard. In this paper we work with the the smoothed ℓ⁰ (SL0) approximation algorithm for sparse representation. We give an upper bound on the run-time estimation error. This upper bound is tighter than the previously known bound. Subsequently, we develop a reliable stopping criterion. This criterion is helpful in avoiding the problems due to the underlying discontinuities of the ℓ⁰ cost function. Furthermore, we propose an alternative optimization strategy, which results in a Newton like algorithm