Efficient algorithms and data structures for compressive sensing

Wegen der kontinuierlich anwachsenden Anzahl von Sensoren, und den stetig wachsenden Datenmengen, die jene produzieren, stößt die konventielle Art Signale zu verarbeiten, beruhend auf dem Nyquist-Kriterium, auf immer mehr Hindernisse und Probleme. Die kürzlich entwickelte Theorie des Compressive Sensing (CS) formuliert das Versprechen einige dieser Hindernisse zu beseitigen, indem hier allgemeinere Signalaufnahme und -rekonstruktionsverfahren zum Einsatz kommen können. Dies erlaubt, dass hierbei einzelne Abtastwerte komplexer strukturierte Informationen über das Signal enthalten können als dies bei konventiellem Nyquistsampling der Fall ist. Gleichzeitig verändert sich die Signalrekonstruktion notwendigerweise zu einem nicht-linearen Vorgang und ebenso müssen viele Hardwarekonzepte für praktische Anwendungen neu überdacht werden. Das heißt, dass man zwischen der Menge an Information, die man über Signale gewinnen kann, und dem Aufwand für das Design und Betreiben eines Signalverarbeitungssystems abwägen kann und muss. Die hier vorgestellte Arbeit trägt dazu bei, dass bei diesem Abwägen CS mehr begünstigt werden kann, indem neue Resultate vorgestellt werden, die es erlauben, dass CS einfacher in der Praxis Anwendung finden kann, wobei die zu erwartende Leistungsfähigkeit des Systems theoretisch fundiert ist. Beispielsweise spielt das Konzept der Sparsity eine zentrale Rolle, weshalb diese Arbeit eine Methode präsentiert, womit der Grad der Sparsity eines Vektors mittels einer einzelnen Beobachtung geschätzt werden kann. Wir zeigen auf, dass dieser Ansatz für Sparsity Order Estimation zu einem niedrigeren Rekonstruktionsfehler führt, wenn man diesen mit einer Rekonstruktion vergleicht, welcher die Sparsity des Vektors unbekannt ist. Um die Modellierung von Signalen und deren Rekonstruktion effizienter zu gestalten, stellen wir das Konzept von der matrixfreien Darstellung linearer Operatoren vor. Für die einfachere Anwendung dieser Darstellung präsentieren wir eine freie Softwarearchitektur und demonstrieren deren Vorzüge, wenn sie für die Rekonstruktion in einem CS-System genutzt wird. Konkret wird der Nutzen dieser Bibliothek, einerseits für das Ermitteln von Defektpositionen in Prüfkörpern mittels Ultraschall, und andererseits für das Schätzen von Streuern in einem Funkkanal aus Ultrabreitbanddaten, demonstriert. Darüber hinaus stellen wir für die Verarbeitung der Ultraschalldaten eine Rekonstruktionspipeline vor, welche Daten verarbeitet, die im Frequenzbereich Unterabtastung erfahren haben. Wir beschreiben effiziente Algorithmen, die bei der Modellierung und der Rekonstruktion zum Einsatz kommen und wir leiten asymptotische Resultate für die benötigte Anzahl von Messwerten, sowie die zu erwartenden Lokalisierungsgenauigkeiten der Defekte her. Wir zeigen auf, dass das vorgestellte System starke Kompression zulässt, ohne die Bildgebung und Defektlokalisierung maßgeblich zu beeinträchtigen. Für die Lokalisierung von Streuern mittels Ultrabreitbandradaren stellen wir ein CS-System vor, welches auf einem Random Demodulators basiert. Im Vergleich zu existierenden Messverfahren ist die hieraus resultierende Schätzung der Kanalimpulsantwort robuster gegen die Effekte von zeitvarianten Funkkanälen. Um den inhärenten Modellfehler, den gitterbasiertes CS begehen muss, zu beseitigen, zeigen wir auf wie Atomic Norm Minimierung es erlaubt ohne die Einschränkung auf ein endliches und diskretes Gitter R-dimensionale spektrale Komponenten aus komprimierten Beobachtungen zu schätzen. Hierzu leiten wir eine R-dimensionale Variante des ADMM her, welcher dazu in der Lage ist die Signalkovarianz in diesem allgemeinen Szenario zu schätzen. Weiterhin zeigen wir, wie dieser Ansatz zur Richtungsschätzung mit realistischen Antennenarraygeometrien genutzt werden kann. In diesem Zusammenhang präsentieren wir auch eine Methode, welche mittels Stochastic gradient descent Messmatrizen ermitteln kann, die sich gut für Parameterschätzung eignen. Die hieraus resultierenden Kompressionsverfahren haben die Eigenschaft, dass die Schätzgenauigkeit über den gesamten Parameterraum ein möglichst uniformes Verhalten zeigt. Zuletzt zeigen wir auf, dass die Kombination des ADMM und des Stochastic Gradient descent das Design eines CS-Systems ermöglicht, welches in diesem gitterfreien Szenario wünschenswerte Eigenschaften hat.Along with the ever increasing number of sensors, which are also generating rapidly growing amounts of data, the traditional paradigm of sampling adhering the Nyquist criterion is facing an equally increasing number of obstacles. The rather recent theory of Compressive Sensing (CS) promises to alleviate some of these drawbacks by proposing to generalize the sampling and reconstruction schemes such that the acquired samples can contain more complex information about the signal than Nyquist samples. The proposed measurement process is more complex and the reconstruction algorithms necessarily need to be nonlinear. Additionally, the hardware design process needs to be revisited as well in order to account for this new acquisition scheme. Hence, one can identify a trade-off between information that is contained in individual samples of a signal and effort during development and operation of the sensing system. This thesis addresses the necessary steps to shift the mentioned trade-off more to the favor of CS. We do so by providing new results that make CS easier to deploy in practice while also maintaining the performance indicated by theoretical results. The sparsity order of a signal plays a central role in any CS system. Hence, we present a method to estimate this crucial quantity prior to recovery from a single snapshot. As we show, this proposed Sparsity Order Estimation method allows to improve the reconstruction error compared to an unguided reconstruction. During the development of the theory we notice that the matrix-free view on the involved linear mappings offers a lot of possibilities to render the reconstruction and modeling stage much more efficient. Hence, we present an open source software architecture to construct these matrix-free representations and showcase its ease of use and performance when used for sparse recovery to detect defects from ultrasound data as well as estimating scatterers in a radio channel using ultra-wideband impulse responses. For the former of these two applications, we present a complete reconstruction pipeline when the ultrasound data is compressed by means of sub-sampling in the frequency domain. Here, we present the algorithms for the forward model, the reconstruction stage and we give asymptotic bounds for the number of measurements and the expected reconstruction error. We show that our proposed system allows significant compression levels without substantially deteriorating the imaging quality. For the second application, we develop a sampling scheme to acquire the channel Impulse Response (IR) based on a Random Demodulator that allows to capture enough information in the recorded samples to reliably estimate the IR when exploiting sparsity. Compared to the state of the art, this in turn allows to improve the robustness to the effects of time-variant radar channels while also outperforming state of the art methods based on Nyquist sampling in terms of reconstruction error. In order to circumvent the inherent model mismatch of early grid-based compressive sensing theory, we make use of the Atomic Norm Minimization framework and show how it can be used for the estimation of the signal covariance with R-dimensional parameters from multiple compressive snapshots. To this end, we derive a variant of the ADMM that can estimate this covariance in a very general setting and we show how to use this for direction finding with realistic antenna geometries. In this context we also present a method based on a Stochastic gradient descent iteration scheme to find compression schemes that are well suited for parameter estimation, since the resulting sub-sampling has a uniform effect on the whole parameter space. Finally, we show numerically that the combination of these two approaches yields a well performing grid-free CS pipeline

A fast and accurate basis pursuit denoising algorithm with application to super-resolving tomographic SAR

$L_1$ regularization is used for finding sparse solutions to an underdetermined linear system. As sparse signals are widely expected in remote sensing, this type of regularization scheme and its extensions have been widely employed in many remote sensing problems, such as image fusion, target detection, image super-resolution, and others and have led to promising results. However, solving such sparse reconstruction problems is computationally expensive and has limitations in its practical use. In this paper, we proposed a novel efficient algorithm for solving the complex-valued $L_1$ regularized least squares problem. Taking the high-dimensional tomographic synthetic aperture radar (TomoSAR) as a practical example, we carried out extensive experiments, both with simulation data and real data, to demonstrate that the proposed approach can retain the accuracy of second order methods while dramatically speeding up the processing by one or two orders. Although we have chosen TomoSAR as the example, the proposed method can be generally applied to any spectral estimation problems.Comment: 11 pages, IEEE Transactions on Geoscience and Remote Sensin

Robust Subspace Tracking Algorithms in Signal Processing: A Brief Survey

Principal component analysis (PCA) and subspace estimation (SE) are popular data analysis tools and used in a wide range of applications. The main interest in PCA/SE is for dimensionality reduction and low-rank approximation purposes. The emergence of big data streams have led to several essential issues for performing PCA/SE. Among them are (i) the size of such data streams increases over time, (ii) the underlying models may be time-dependent, and (iii) problem of dealing with the uncertainty and incompleteness in data. A robust variant of PCA/SE for such data streams, namely robust online PCA or robust subspace tracking (RST), has been introduced as a good alternative. The main goal of this paper is to provide a brief survey on recent RST algorithms in signal processing. Particularly, we begin this survey by introducing the basic ideas of the RST problem. Then, different aspects of RST are reviewed with respect to different kinds of non-Gaussian noises and sparse constraints. Our own contributions on this topic are also highlighted

Time-Domain Channel Estimation for Extremely Large MIMO THz Communications with Beam Squint

In this paper, we study the problem of extremely large (XL) multiple-input multiple-output (MIMO) channel estimation in the Terahertz (THz) frequency band, considering the presence of propagation delays across the entire array apertures, which leads to frequency selectivity, a problem known as beam squint. Multi-carrier transmission schemes which are usually deployed to address this problem, suffer from high peak-to-average power ratio, which is specifically dominant in THz communications where low transmit power is realized. Diverging from the usual approach, we devise a novel channel estimation problem formulation in the time domain for single-carrier (SC) modulation, which favors transmissions in THz, and incorporate the beam-squint effect in a sparse vector recovery problem that is solved via sparse optimization tools. In particular, the beam squint and the sparse MIMO channel are jointly tracked by using an alternating minimization approach that decomposes the two estimation problems. The presented performance evaluation results validate that the proposed SC technique exhibits superior performance than the conventional one as well as than state-of-the-art multi-carrier approaches

Sparsity Promoting Off-grid Methods with Applications in Direction Finding

University of Minnesota Ph.D. dissertation. May 2017. Major: Electrical/Computer Engineering. Advisor: Mostafa Kaveh. 1 computer file (PDF); x, 99 pages.In this dissertation, the problem of directions-of-arrival (DoA) estimation is studied by the compressed sensing application of sparsity-promoting regularization techniques. Compressed sensing can recover high-dimensional signals with a sparse representation from very few linear measurements by nonlinear optimization. By exploiting the sparse representation for the multiple measurement vectors or the spatial covariance matrix of correlated or uncorrelated sources, the DoA estimation problem can be formulated in the framework of sparse signal recovery with high resolution. There are three main topics covered in this dissertation. These topics are recovery methods for the sparse model with structured perturbations, continuous sparse recovery methods in the super-resolution framework, and the off-grid DoA estimation with array self-calibration. These topics are summarized below. For the first topic, structured perturbation in the sparse model is considered. A major limitation of most methods exploiting sparse spectral models for the purpose of estimating directions-of-arrival stems from the fixed model dictionary that is formed by array response vectors over a discrete search grid of possible directions. In general, the array responses to actual DoAs will most likely not be members of such a dictionary. Thus, the sparse spectral signal model with uncertainty of linearized dictionary parameter mismatch is considered, and the dictionary matrix is reformulated into a multiplication of a fixed base dictionary and a sparse matrix. Based on this sparse model, we propose several convex optimization algorithms. However, we are also concerned with the development of a computationally efficient optimization algorithm for off-grid direction finding using a sparse observation model. With an emphasis on designing efficient algorithms, various sparse problem formulations are considered, such as unconstrained formulation, primal-dual formulation, or conic formulation. But, because of the nature of nondifferentiable objective functions, those problems are still challenging to solve in an efficient way. Thus, the Nesterov smoothing methodology is utilized to reformulate nonsmooth functions into smooth ones, and the accelerated proximal gradient algorithm is adopted to solve the smoothed optimization problem. Convergence analysis is conducted as well. The accuracy and efficiency of smoothed sparse recovery methods are demonstrated for the DoA estimation example. In the second topic, estimation of directions-of-arrival in the spatial covariance model is studied. Unlike the compressed sensing methods which discretize the search domain into possible directions on a grid, the theory of super resolution is applied to estimate DoAs in the continuous domain. We reformulate the spatial spectral covariance model into a multiple measurement vectors (MMV)-like model, and propose a block total variation norm minimization approach, which is the analog of Group Lasso in the super-resolution framework and that promotes the group-sparsity. The DoAs can be estimated by solving its dual problem via semidefinite programming. This gridless recovery approach is verified by simulation results for both uncorrelated and correlated source signals. In the last topic, we consider the array calibration issue for DoA estimation, and extend the previously considered single measurement vector model to multiple measurement vectors. By exploiting multiple measurement snapshots, a modified nuclear norm minimization problem is proposed to recover a low-rank matrix with high probability. The definition of linear operator for the MMV model is given, and its corresponding matrix representation is derived so that a reformulated convex optimization problem can be solved numerically. In order to alleviate computational complexity of the method, we use singular value decomposition (SVD) to reduce the problem size. Furthermore, the structured perturbation in the sparse array self-calibration estimation problem is considered as well. The performance and efficiency of the proposed methods are demonstrated by numerical results

Sparse Array Signal Processing

This dissertation details three approaches for direction-of-arrival (DOA) estimation or beamforming in array signal processing from the perspective of sparsity. In the first part of this dissertation, we consider sparse array beamformer design based on the alternating direction method of multipliers (ADMM); in the second part of this dissertation, the problem of joint DOA estimation and distorted sensor detection is investigated; and off-grid DOA estimation is studied in the last part of this dissertation. In the first part of this thesis, we devise a sparse array design algorithm for adaptive beamforming. Our strategy is based on finding a sparse beamformer weight to maximize the output signal-to-interference-plus-noise ratio (SINR). The proposed method utilizes ADMM, and admits closed-form solutions at each ADMM iteration. The algorithm convergence properties are analyzed by showing the monotonicity and boundedness of the augmented Lagrangian function. In addition, we prove that the proposed algorithm converges to the set of Karush-Kuhn-Tucker stationary points. Numerical results exhibit its excellent performance, which is comparable to that of the exhaustive search approach, slightly better than those of the state-of-the-art solvers, and significantly outperforms several other sparse array design strategies, in terms of output SINR. Moreover, the proposed ADMM algorithm outperforms its competitors, in terms of computational cost. Distorted sensors could occur randomly and may lead to the breakdown of a sensor array system. In the second part of this thesis, we consider an array model in which a small number of sensors are distorted by unknown sensor gain and phase errors. With such an array model, the problem of joint DOA estimation and distorted sensor detection is formulated under the framework of low-rank and row-sparse decomposition. We derive an iteratively reweighted least squares (IRLS) algorithm to solve the resulting problem. The convergence property of the IRLS algorithm is analyzed by means of the monotonicity and boundedness of the objective function. Extensive simulations are conducted in view of parameter selection, convergence speed, computational complexity, and performance of DOA estimation as well as distorted sensor detection. Even though the IRLS algorithm is slightly worse than the ADMM in detecting the distorted sensors, the results show that our approach outperforms several state-of-the-art techniques in terms of convergence speed, computational cost, and DOA estimation performance. In the last part of this thesis, the problem of off-grid DOA estimation is investigated. We develop a method to jointly estimate the closest spatial frequency (the sine of DOA) grids, and the gaps between the estimated grids and the corresponding frequencies. By using a second-order Taylor approximation, the data model under the framework of joint-sparse representation is formulated. We point out an important property of the signals of interest in the model, namely the proportionality relationship. The proportionality relationship is empirically demonstrated to be useful in the sense that it increases the probability of the mixing matrix satisfying the block restricted isometry property. Simulation examples demonstrate the effectiveness and superiority of the proposed method against several state-of-the-art grid-based approaches