A Compact Formulation for the ℓ2,1\ell_{2,1} Mixed-Norm Minimization Problem

Parameter estimation from multiple measurement vectors (MMVs) is a fundamental problem in many signal processing applications, e.g., spectral analysis and direction-of- arrival estimation. Recently, this problem has been address using prior information in form of a jointly sparse signal structure. A prominent approach for exploiting joint sparsity considers mixed-norm minimization in which, however, the problem size grows with the number of measurements and the desired resolution, respectively. In this work we derive an equivalent, compact reformulation of the $\ell_{2,1}$ mixed-norm minimization problem which provides new insights on the relation between different existing approaches for jointly sparse signal reconstruction. The reformulation builds upon a compact parameterization, which models the row-norms of the sparse signal representation as parameters of interest, resulting in a significant reduction of the MMV problem size. Given the sparse vector of row-norms, the jointly sparse signal can be computed from the MMVs in closed form. For the special case of uniform linear sampling, we present an extension of the compact formulation for gridless parameter estimation by means of semidefinite programming. Furthermore, we derive in this case from our compact problem formulation the exact equivalence between the $\ell_{2,1}$ mixed-norm minimization and the atomic-norm minimization. Additionally, for the case of irregular sampling or a large number of samples, we present a low complexity, grid-based implementation based on the coordinate descent method

Sparsity-Cognizant Total Least-Squares for Perturbed Compressive Sampling

Solving linear regression problems based on the total least-squares (TLS) criterion has well-documented merits in various applications, where perturbations appear both in the data vector as well as in the regression matrix. However, existing TLS approaches do not account for sparsity possibly present in the unknown vector of regression coefficients. On the other hand, sparsity is the key attribute exploited by modern compressive sampling and variable selection approaches to linear regression, which include noise in the data, but do not account for perturbations in the regression matrix. The present paper fills this gap by formulating and solving TLS optimization problems under sparsity constraints. Near-optimum and reduced-complexity suboptimum sparse (S-) TLS algorithms are developed to address the perturbed compressive sampling (and the related dictionary learning) challenge, when there is a mismatch between the true and adopted bases over which the unknown vector is sparse. The novel S-TLS schemes also allow for perturbations in the regression matrix of the least-absolute selection and shrinkage selection operator (Lasso), and endow TLS approaches with ability to cope with sparse, under-determined "errors-in-variables" models. Interesting generalizations can further exploit prior knowledge on the perturbations to obtain novel weighted and structured S-TLS solvers. Analysis and simulations demonstrate the practical impact of S-TLS in calibrating the mismatch effects of contemporary grid-based approaches to cognitive radio sensing, and robust direction-of-arrival estimation using antenna arrays.Comment: 30 pages, 10 figures, submitted to IEEE Transactions on Signal Processin

Three more Decades in Array Signal Processing Research: An Optimization and Structure Exploitation Perspective

The signal processing community currently witnesses the emergence of sensor array processing and Direction-of-Arrival (DoA) estimation in various modern applications, such as automotive radar, mobile user and millimeter wave indoor localization, drone surveillance, as well as in new paradigms, such as joint sensing and communication in future wireless systems. This trend is further enhanced by technology leaps and availability of powerful and affordable multi-antenna hardware platforms. The history of advances in super resolution DoA estimation techniques is long, starting from the early parametric multi-source methods such as the computationally expensive maximum likelihood (ML) techniques to the early subspace-based techniques such as Pisarenko and MUSIC. Inspired by the seminal review paper Two Decades of Array Signal Processing Research: The Parametric Approach by Krim and Viberg published in the IEEE Signal Processing Magazine, we are looking back at another three decades in Array Signal Processing Research under the classical narrowband array processing model based on second order statistics. We revisit major trends in the field and retell the story of array signal processing from a modern optimization and structure exploitation perspective. In our overview, through prominent examples, we illustrate how different DoA estimation methods can be cast as optimization problems with side constraints originating from prior knowledge regarding the structure of the measurement system. Due to space limitations, our review of the DoA estimation research in the past three decades is by no means complete. For didactic reasons, we mainly focus on developments in the field that easily relate the traditional multi-source estimation criteria and choose simple illustrative examples.Comment: 16 pages, 8 figures. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Subspace-based order estimation techniques in massive MIMO

Order estimation, also known as source enumeration, is a classical problem in array signal processing which consists in estimating the number of signals received by an array of sensors. In the last decades, numerous approaches to this problem have been proposed. However, the need of working with large-scale arrays (like in massive MIMO systems), low signal-to-noise- ratios, and poor sample regime scenarios, introduce new challenges to order estimation problems. For instance, most of the classical approaches are based on information theoretic criteria, which usually require a large sample size, typically several times larger than the number of sensors. Obtaining a number of samples several times larger than the number of sensors is not always possible with large-scale arrays. In addition, most of the methods found in literature assume that the noise is spatially white, which is very restrictive for many practical scenarios. This dissertation deals with the problem of source enumeration for large-scale arrays, and proposes solutions that work robustly in the small sample regime under various noise models. The first part of the dissertation solves the problem by applying the idea of subspace averaging. The input data are modelled as subspaces, and an average or central subspace is computed. The source enumeration problem can be seen as an estimation of the dimension of the central subspace. A key element of the proposed method is to construct a bootstrap procedure, based on a newly proposed discrete distribution on the manifold of projection matrices, for stochastically generating subspaces from a function of experimentally determined eigenvalues. In this way, the proposed subspace averaging (SA) technique determines the order based on the eigenvalues of an average projection matrix, rather than on the likelihood of a covariance model, penalized by functions of the model order. The proposed SA criterion is especially effective in high-dimensional scenarios with low sample support for uniform linear arrays in the presence of white noise. Further, the proposed SA method is extended for: i) non-white noises, and ii) non-uniform linear arrays. The SA criterion is sensitive with the chosen dimension of extracted subspaces. To solve this problem, we combine the SA technique with a majority vote approach. The number of sources is detected for increasing dimensions of the SA technique and then a majority vote is applied to determine the final estimate. Further, to extend SA for arrays with arbitrary geometries, the SA is combined with a sparse reconstruction (SR) step. In the first step, each received snapshot is approximated by a sparse linear combination of the rest of snapshots. The SR problem is regularized by the logarithm-based surrogate of the l-0 norm and solved using a majorization-minimization approach. Based on the SR solution, a sampling mechanism is proposed in the second step to generate a collection of subspaces, all of which approximately span the same signal subspace. Finally, the dimension of the average of this collection of subspaces provides a robust estimate for the number of sources. The second half of the dissertation introduces a completely different approach to the order estimation for uniform linear arrays, which is based on matrix completion algorithms. This part first discusses the problem of order estimation in the presence of noise whose spatial covariance structure is a diagonal matrix with possibly different variances. The diagonal terms of the sample covariance matrix are removed and, after applying Toeplitz rectification as a denoising step, the signal covariance matrix is reconstructed by using a low-rank matrix completion method adapted to enforce the Toeplitz structure of the sought solution. The proposed source enumeration criterion is based on the Frobenius norm of the reconstructed signal covariance matrix obtained for increasing rank values. The proposed method performs robustly for both small and large-scale arrays with few snapshots. Finally, an approach to work with a reduced number of radio–frequency (RF) chains is proposed. The receiving array relies on antenna switching so that at every time instant only the signals received by a randomly selected subset of antennas are downconverted to baseband and sampled. Low-rank matrix completion (MC) techniques are then used to reconstruct the missing entries of the signal data matrix to keep the angular resolution of the original large-scale array. The proposed MC algorithm exploits not only the low- rank structure of the signal subspace, but also the shift-invariance property of uniform linear arrays, which results in a better estimation of the signal subspace. In addition, the effect of MC on DOA estimation is discussed under the perturbation theory framework. Further, this approach is extended to devise a novel order estimation criterion for missing data scenario. The proposed source enumeration criterion is based on the chordal subspace distance between two sub-matrices extracted from the reconstructed matrix after using MC for increasing rank values. We show that the proposed order estimation criterion performs consistently with a very few available entries in the data matrix.This work was supported by the Ministerio de Ciencia e Innovación (MICINN) of Spain, under grants TEC2016-75067-C4-4-R (CARMEN) and BES-2017-080542

Robust and Efficient Inference of Scene and Object Motion in Multi-Camera Systems

Multi-camera systems have the ability to overcome some of the fundamental limitations of single camera based systems. Having multiple view points of a scene goes a long way in limiting the influence of field of view, occlusion, blur and poor resolution of an individual camera. This dissertation addresses robust and efficient inference of object motion and scene in multi-camera and multi-sensor systems. The first part of the dissertation discusses the role of constraints introduced by projective imaging towards robust inference of multi-camera/sensor based object motion. We discuss the role of the homography and epipolar constraints for fusing object motion perceived by individual cameras. For planar scenes, the homography constraints provide a natural mechanism for data association. For scenes that are not planar, the epipolar constraint provides a weaker multi-view relationship. We use the epipolar constraint for tracking in multi-camera and multi-sensor networks. In particular, we show that the epipolar constraint reduces the dimensionality of the state space of the problem by introducing a ``shared'' state space for the joint tracking problem. This allows for robust tracking even when one of the sensors fail due to poor SNR or occlusion. The second part of the dissertation deals with challenges in the computational aspects of tracking algorithms that are common to such systems. Much of the inference in the multi-camera and multi-sensor networks deal with complex non-linear models corrupted with non-Gaussian noise. Particle filters provide approximate Bayesian inference in such settings. We analyze the computational drawbacks of traditional particle filtering algorithms, and present a method for implementing the particle filter using the Independent Metropolis Hastings sampler, that is highly amenable to pipelined implementations and parallelization. We analyze the implementations of the proposed algorithm, and in particular concentrate on implementations that have minimum processing times. The last part of the dissertation deals with the efficient sensing paradigm of compressing sensing (CS) applied to signals in imaging, such as natural images and reflectance fields. We propose a hybrid signal model on the assumption that most real-world signals exhibit subspace compressibility as well as sparse representations. We show that several real-world visual signals such as images, reflectance fields, videos etc., are better approximated by this hybrid of two models. We derive optimal hybrid linear projections of the signal and show that theoretical guarantees and algorithms designed for CS can be easily extended to hybrid subspace-compressive sensing. Such methods reduce the amount of information sensed by a camera, and help in reducing the so called data deluge problem in large multi-camera systems