Off-the-Grid Line Spectrum Denoising and Estimation with Multiple Measurement Vectors

Compressed Sensing suggests that the required number of samples for reconstructing a signal can be greatly reduced if it is sparse in a known discrete basis, yet many real-world signals are sparse in a continuous dictionary. One example is the spectrally-sparse signal, which is composed of a small number of spectral atoms with arbitrary frequencies on the unit interval. In this paper we study the problem of line spectrum denoising and estimation with an ensemble of spectrally-sparse signals composed of the same set of continuous-valued frequencies from their partial and noisy observations. Two approaches are developed based on atomic norm minimization and structured covariance estimation, both of which can be solved efficiently via semidefinite programming. The first approach aims to estimate and denoise the set of signals from their partial and noisy observations via atomic norm minimization, and recover the frequencies via examining the dual polynomial of the convex program. We characterize the optimality condition of the proposed algorithm and derive the expected convergence rate for denoising, demonstrating the benefit of including multiple measurement vectors. The second approach aims to recover the population covariance matrix from the partially observed sample covariance matrix by motivating its low-rank Toeplitz structure without recovering the signal ensemble. Performance guarantee is derived with a finite number of measurement vectors. The frequencies can be recovered via conventional spectrum estimation methods such as MUSIC from the estimated covariance matrix. Finally, numerical examples are provided to validate the favorable performance of the proposed algorithms, with comparisons against several existing approaches.Comment: 14 pages, 10 figure

On Measure Transformed Canonical Correlation Analysis

In this paper linear canonical correlation analysis (LCCA) is generalized by applying a structured transform to the joint probability distribution of the considered pair of random vectors, i.e., a transformation of the joint probability measure defined on their joint observation space. This framework, called measure transformed canonical correlation analysis (MTCCA), applies LCCA to the data after transformation of the joint probability measure. We show that judicious choice of the transform leads to a modified canonical correlation analysis, which, in contrast to LCCA, is capable of detecting non-linear relationships between the considered pair of random vectors. Unlike kernel canonical correlation analysis, where the transformation is applied to the random vectors, in MTCCA the transformation is applied to their joint probability distribution. This results in performance advantages and reduced implementation complexity. The proposed approach is illustrated for graphical model selection in simulated data having non-linear dependencies, and for measuring long-term associations between companies traded in the NASDAQ and NYSE stock markets

Atomic norm denoising with applications to line spectral estimation

Motivated by recent work on atomic norms in inverse problems, we propose a new approach to line spectral estimation that provides theoretical guarantees for the mean-squared-error (MSE) performance in the presence of noise and without knowledge of the model order. We propose an abstract theory of denoising with atomic norms and specialize this theory to provide a convex optimization problem for estimating the frequencies and phases of a mixture of complex exponentials. We show that the associated convex optimization problem can be solved in polynomial time via semidefinite programming (SDP). We also show that the SDP can be approximated by an l1-regularized least-squares problem that achieves nearly the same error rate as the SDP but can scale to much larger problems. We compare both SDP and l1-based approaches with classical line spectral analysis methods and demonstrate that the SDP outperforms the l1 optimization which outperforms MUSIC, Cadzow's, and Matrix Pencil approaches in terms of MSE over a wide range of signal-to-noise ratios.Comment: 27 pages, 10 figures. A preliminary version of this work appeared in the Proceedings of the 49th Annual Allerton Conference in September 2011. Numerous numerical experiments added to this version in accordance with suggestions by anonymous reviewer

Denoising using local projective subspace methods

In this paper we present denoising algorithms for enhancing noisy signals based on Local ICA (LICA), Delayed AMUSE (dAMUSE) and Kernel PCA (KPCA). The algorithm LICA relies on applying ICA locally to clusters of signals embedded in a high-dimensional feature space of delayed coordinates. The components resembling the signals can be detected by various criteria like estimators of kurtosis or the variance of autocorrelations depending on the statistical nature of the signal. The algorithm proposed can be applied favorably to the problem of denoising multi-dimensional data. Another projective subspace denoising method using delayed coordinates has been proposed recently with the algorithm dAMUSE. It combines the solution of blind source separation problems with denoising efforts in an elegant way and proofs to be very efficient and fast. Finally, KPCA represents a non-linear projective subspace method that is well suited for denoising also. Besides illustrative applications to toy examples and images, we provide an application of all algorithms considered to the analysis of protein NMR spectra.info:eu-repo/semantics/publishedVersio

On asymptotics of ICA estimators and their performance indices

Independent component analysis (ICA) has become a popular multivariate analysis and signal processing technique with diverse applications. This paper is targeted at discussing theoretical large sample properties of ICA unmixing matrix functionals. We provide a formal definition of unmixing matrix functional and consider two popular estimators in detail: the family based on two scatter matrices with the independence property (e.g., FOBI estimator) and the family of deflation-based fastICA estimators. The limiting behavior of the corresponding estimates is discussed and the asymptotic normality of the deflation-based fastICA estimate is proven under general assumptions. Furthermore, properties of several performance indices commonly used for comparison of different unmixing matrix estimates are discussed and a new performance index is proposed. The proposed index fullfills three desirable features which promote its use in practice and distinguish it from others. Namely, the index possesses an easy interpretation, is fast to compute and its asymptotic properties can be inferred from asymptotics of the unmixing matrix estimate. We illustrate the derived asymptotical results and the use of the proposed index with a small simulation study

Sensor Array Signal Processing via Eigenanalysis of Matrix Pencils Composed of Data Derived from Translationally Invariant Subarrays

An algorithm is developed for estimating characteristic parameters associated with a scene of radiating sources given the data derived from a pair of translationally invariant arrays, the X and Y arrays, which are displaced relative to one another. The algorithm is referred to as PR O—E SPRIT and is predicated on invoking two recent mathematical developments: (1) the SVD based solution to the Procrustes problem of optimally approximating an invariant subspace rotation and (2) the Total Least Squares method for perturbing each of the two estimates of a common subspace in a minimal fashion until the two perturbed spaces are the same. For uniform linear array scenarios, the use of forward-backward averaging (FBAVG) in conjunction with PR O—E S PR IT is shown to effect a substantial reduction in the computational burden, a significant improvement in performance, a simple scheme for estimating the number of sources and source decorrelation. These gains may be attributed to FBAVG’s judicious exploitation of the diagonal invariance operator relating the Direction of Arrival matrix of the Y array to that associated with the X array. Similar gains may be achieved in the case where the X and Y arrays are either not linear or not uniformly spaced through the use of pseudo-forward-backward averaging (PFBAVG). However, the use of PFBAVG does not effect source decorrelation and reduces the maximum number of resolvable sources by a factor of two. Simulation studies and the results of applying PR O—E S PR IT to real data demonstrate the excellent performance of the method

dAMUSE : a new tool for denoising and blind source separation

In this work a generalized version of AMUSE, called dAMUSE is proposed. The main modification consists in embedding the observed mixed signals in a high-dimensional feature space of delayed coordinates. With the embedded signals a matrix pencil is formed and its generalized eigendecomposition is computed similar to the algorithm AMUSE. We show that in this case the uncorrelated output signals are filtered versions of the unknown source signals. Further, denoising the data can be achieved conveniently in parallel with the signal separation. Numerical simulations using artificially mixed signals are presented to show the performance of the method. Further results of a heart rate variability (HRV) study are discussed showing that the output signals are related with LF (low frequency) and HF (high frequency) fluctuations. Finally, an application to separate artifacts from 2D NOESY NMR spectra and to denoise the reconstructed artefact-free spectra is presented also.info:eu-repo/semantics/publishedVersio