Efficient algorithms and data structures for compressive sensing

Wegen der kontinuierlich anwachsenden Anzahl von Sensoren, und den stetig wachsenden Datenmengen, die jene produzieren, stößt die konventielle Art Signale zu verarbeiten, beruhend auf dem Nyquist-Kriterium, auf immer mehr Hindernisse und Probleme. Die kürzlich entwickelte Theorie des Compressive Sensing (CS) formuliert das Versprechen einige dieser Hindernisse zu beseitigen, indem hier allgemeinere Signalaufnahme und -rekonstruktionsverfahren zum Einsatz kommen können. Dies erlaubt, dass hierbei einzelne Abtastwerte komplexer strukturierte Informationen über das Signal enthalten können als dies bei konventiellem Nyquistsampling der Fall ist. Gleichzeitig verändert sich die Signalrekonstruktion notwendigerweise zu einem nicht-linearen Vorgang und ebenso müssen viele Hardwarekonzepte für praktische Anwendungen neu überdacht werden. Das heißt, dass man zwischen der Menge an Information, die man über Signale gewinnen kann, und dem Aufwand für das Design und Betreiben eines Signalverarbeitungssystems abwägen kann und muss. Die hier vorgestellte Arbeit trägt dazu bei, dass bei diesem Abwägen CS mehr begünstigt werden kann, indem neue Resultate vorgestellt werden, die es erlauben, dass CS einfacher in der Praxis Anwendung finden kann, wobei die zu erwartende Leistungsfähigkeit des Systems theoretisch fundiert ist. Beispielsweise spielt das Konzept der Sparsity eine zentrale Rolle, weshalb diese Arbeit eine Methode präsentiert, womit der Grad der Sparsity eines Vektors mittels einer einzelnen Beobachtung geschätzt werden kann. Wir zeigen auf, dass dieser Ansatz für Sparsity Order Estimation zu einem niedrigeren Rekonstruktionsfehler führt, wenn man diesen mit einer Rekonstruktion vergleicht, welcher die Sparsity des Vektors unbekannt ist. Um die Modellierung von Signalen und deren Rekonstruktion effizienter zu gestalten, stellen wir das Konzept von der matrixfreien Darstellung linearer Operatoren vor. Für die einfachere Anwendung dieser Darstellung präsentieren wir eine freie Softwarearchitektur und demonstrieren deren Vorzüge, wenn sie für die Rekonstruktion in einem CS-System genutzt wird. Konkret wird der Nutzen dieser Bibliothek, einerseits für das Ermitteln von Defektpositionen in Prüfkörpern mittels Ultraschall, und andererseits für das Schätzen von Streuern in einem Funkkanal aus Ultrabreitbanddaten, demonstriert. Darüber hinaus stellen wir für die Verarbeitung der Ultraschalldaten eine Rekonstruktionspipeline vor, welche Daten verarbeitet, die im Frequenzbereich Unterabtastung erfahren haben. Wir beschreiben effiziente Algorithmen, die bei der Modellierung und der Rekonstruktion zum Einsatz kommen und wir leiten asymptotische Resultate für die benötigte Anzahl von Messwerten, sowie die zu erwartenden Lokalisierungsgenauigkeiten der Defekte her. Wir zeigen auf, dass das vorgestellte System starke Kompression zulässt, ohne die Bildgebung und Defektlokalisierung maßgeblich zu beeinträchtigen. Für die Lokalisierung von Streuern mittels Ultrabreitbandradaren stellen wir ein CS-System vor, welches auf einem Random Demodulators basiert. Im Vergleich zu existierenden Messverfahren ist die hieraus resultierende Schätzung der Kanalimpulsantwort robuster gegen die Effekte von zeitvarianten Funkkanälen. Um den inhärenten Modellfehler, den gitterbasiertes CS begehen muss, zu beseitigen, zeigen wir auf wie Atomic Norm Minimierung es erlaubt ohne die Einschränkung auf ein endliches und diskretes Gitter R-dimensionale spektrale Komponenten aus komprimierten Beobachtungen zu schätzen. Hierzu leiten wir eine R-dimensionale Variante des ADMM her, welcher dazu in der Lage ist die Signalkovarianz in diesem allgemeinen Szenario zu schätzen. Weiterhin zeigen wir, wie dieser Ansatz zur Richtungsschätzung mit realistischen Antennenarraygeometrien genutzt werden kann. In diesem Zusammenhang präsentieren wir auch eine Methode, welche mittels Stochastic gradient descent Messmatrizen ermitteln kann, die sich gut für Parameterschätzung eignen. Die hieraus resultierenden Kompressionsverfahren haben die Eigenschaft, dass die Schätzgenauigkeit über den gesamten Parameterraum ein möglichst uniformes Verhalten zeigt. Zuletzt zeigen wir auf, dass die Kombination des ADMM und des Stochastic Gradient descent das Design eines CS-Systems ermöglicht, welches in diesem gitterfreien Szenario wünschenswerte Eigenschaften hat.Along with the ever increasing number of sensors, which are also generating rapidly growing amounts of data, the traditional paradigm of sampling adhering the Nyquist criterion is facing an equally increasing number of obstacles. The rather recent theory of Compressive Sensing (CS) promises to alleviate some of these drawbacks by proposing to generalize the sampling and reconstruction schemes such that the acquired samples can contain more complex information about the signal than Nyquist samples. The proposed measurement process is more complex and the reconstruction algorithms necessarily need to be nonlinear. Additionally, the hardware design process needs to be revisited as well in order to account for this new acquisition scheme. Hence, one can identify a trade-off between information that is contained in individual samples of a signal and effort during development and operation of the sensing system. This thesis addresses the necessary steps to shift the mentioned trade-off more to the favor of CS. We do so by providing new results that make CS easier to deploy in practice while also maintaining the performance indicated by theoretical results. The sparsity order of a signal plays a central role in any CS system. Hence, we present a method to estimate this crucial quantity prior to recovery from a single snapshot. As we show, this proposed Sparsity Order Estimation method allows to improve the reconstruction error compared to an unguided reconstruction. During the development of the theory we notice that the matrix-free view on the involved linear mappings offers a lot of possibilities to render the reconstruction and modeling stage much more efficient. Hence, we present an open source software architecture to construct these matrix-free representations and showcase its ease of use and performance when used for sparse recovery to detect defects from ultrasound data as well as estimating scatterers in a radio channel using ultra-wideband impulse responses. For the former of these two applications, we present a complete reconstruction pipeline when the ultrasound data is compressed by means of sub-sampling in the frequency domain. Here, we present the algorithms for the forward model, the reconstruction stage and we give asymptotic bounds for the number of measurements and the expected reconstruction error. We show that our proposed system allows significant compression levels without substantially deteriorating the imaging quality. For the second application, we develop a sampling scheme to acquire the channel Impulse Response (IR) based on a Random Demodulator that allows to capture enough information in the recorded samples to reliably estimate the IR when exploiting sparsity. Compared to the state of the art, this in turn allows to improve the robustness to the effects of time-variant radar channels while also outperforming state of the art methods based on Nyquist sampling in terms of reconstruction error. In order to circumvent the inherent model mismatch of early grid-based compressive sensing theory, we make use of the Atomic Norm Minimization framework and show how it can be used for the estimation of the signal covariance with R-dimensional parameters from multiple compressive snapshots. To this end, we derive a variant of the ADMM that can estimate this covariance in a very general setting and we show how to use this for direction finding with realistic antenna geometries. In this context we also present a method based on a Stochastic gradient descent iteration scheme to find compression schemes that are well suited for parameter estimation, since the resulting sub-sampling has a uniform effect on the whole parameter space. Finally, we show numerically that the combination of these two approaches yields a well performing grid-free CS pipeline

Underdetermined DOA Estimation Under the Compressive Sensing Framework: A Review

Direction of arrival (DOA) estimation from the perspective of sparse signal representation has attracted tremendous attention in past years, where the underlying spatial sparsity reconstruction problem is linked to the compressive sensing (CS) framework. Although this is an area with ongoing intensive research and new methods and results are reported regularly, it is time to have a review about the basic approaches and methods for CS-based DOA estimation, in particular for the underdetermined case. We start from the basic time-domain CSbased formulation for narrowband arrays and then move to the case for recently developed methods for sparse arrays based on the co-array concept. After introducing two specifically designed structures (the two-level nested array and the co-prime array) for optimizing the virtual sensors corresponding to the difference coarray, this CS-based DOA estimation approach is extended to the wideband case by employing the group sparsity concept, where a much larger physical aperture can be achieved by allowing a larger unit inter-element spacing and therefore leading to further improved performance. Finally, a specifically designed ULA structure with associated CS-based underdetermined DOA estimation is presented to exploit the difference co-array concept in the spatio-spectral domain, leading to a significant increase in DOFs. Representative simulation results for typical narrowband and wideband scenarios are provided to demonstrate their performance

Non-uniform Array and Frequency Spacing for Regularization-free Gridless DOA

Gridless direction-of-arrival (DOA) estimation with multiple frequencies can be applied in acoustics source localization problems. We formulate this as an atomic norm minimization (ANM) problem and derive an equivalent regularization-free semi-definite program (SDP) thereby avoiding regularization bias. The DOA is retrieved using a Vandermonde decomposition on the Toeplitz matrix obtained from the solution of the SDP. We also propose a fast SDP program to deal with non-uniform array and frequency spacing. For non-uniform spacings, the Toeplitz structure will not exist, but the DOA is retrieved via irregular Vandermonde decomposition (IVD), and we theoretically guarantee the existence of the IVD. We extend ANM to the multiple measurement vector (MMV) cases and derive its equivalent regularization-free SDP. Using multiple frequencies and the MMV model, we can resolve more sources than the number of physical sensors for a uniform linear array. Numerical results demonstrate that the regularization-free framework is robust to noise and aliasing, and it overcomes the regularization bias

Three more Decades in Array Signal Processing Research: An Optimization and Structure Exploitation Perspective

The signal processing community currently witnesses the emergence of sensor array processing and Direction-of-Arrival (DoA) estimation in various modern applications, such as automotive radar, mobile user and millimeter wave indoor localization, drone surveillance, as well as in new paradigms, such as joint sensing and communication in future wireless systems. This trend is further enhanced by technology leaps and availability of powerful and affordable multi-antenna hardware platforms. The history of advances in super resolution DoA estimation techniques is long, starting from the early parametric multi-source methods such as the computationally expensive maximum likelihood (ML) techniques to the early subspace-based techniques such as Pisarenko and MUSIC. Inspired by the seminal review paper Two Decades of Array Signal Processing Research: The Parametric Approach by Krim and Viberg published in the IEEE Signal Processing Magazine, we are looking back at another three decades in Array Signal Processing Research under the classical narrowband array processing model based on second order statistics. We revisit major trends in the field and retell the story of array signal processing from a modern optimization and structure exploitation perspective. In our overview, through prominent examples, we illustrate how different DoA estimation methods can be cast as optimization problems with side constraints originating from prior knowledge regarding the structure of the measurement system. Due to space limitations, our review of the DoA estimation research in the past three decades is by no means complete. For didactic reasons, we mainly focus on developments in the field that easily relate the traditional multi-source estimation criteria and choose simple illustrative examples.Comment: 16 pages, 8 figures. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

A review of closed-form Cramér-Rao Bounds for DOA estimation in the presence of Gaussian noise under a unified framework

The Cramér-Rao Bound (CRB) for direction of arrival (DOA) estimation has been extensively studied over the past four decades, with a plethora of CRB expressions reported for various parametric models. In the literature, there are different methods to derive a closed-form CRB expression, but many derivations tend to involve intricate matrix manipulations which appear difficult to understand. Starting from the Slepian-Bangs formula and following the simplest derivation approach, this paper reviews a number of closed-form Gaussian CRB expressions for the DOA parameter under a unified framework, based on which all the specific CRB presentations can be derived concisely. The results cover three scenarios: narrowband complex circular signals, narrowband complex noncircular signals, and wideband signals. Three signal models are considered: the deterministic model, the stochastic Gaussian model, and the stochastic Gaussian model with the a priori knowledge that the sources are spatially uncorrelated. Moreover, three Gaussian noise models distinguished by the structure of the noise covariance matrix are concerned: spatially uncorrelated noise with unknown either identical or distinct variances at different sensors, and arbitrary unknown noise. In each scenario, a unified framework for the DOA-related block of the deterministic/stochastic CRB is developed, which encompasses one class of closed-form deterministic CRB expressions and two classes of stochastic ones under the three noise models. Comparisons among different CRBs across classes and scenarios are presented, yielding a series of equalities and inequalities which reflect the benchmark for the estimation efficiency under various situations. Furthermore, validity of all CRB expressions are examined, with some specific results for linear arrays provided, leading to several upper bounds on the number of resolvable Gaussian sources in the underdetermined case

Compressive Acquisition and Processing of Sparse Analog Signals

Since the advent of the first digital processing units, the importance of digital signal processing has been steadily rising. Today, most signal processing happens in the digital domain, requiring that analog signals be first sampled and digitized before any relevant data can be extracted from them. The recent explosion of the demands for data acquisition, storage and processing, however, has pushed the capabilities of conventional acquisition systems to their limits in many application areas. By offering an alternative view on the signal acquisition process, ideas from sparse signal processing and one of its main beneficiaries compressed sensing (CS), aim at alleviating some of these problems. In this thesis, we look into the ways the application of a compressive measurement kernel impacts the signal recovery performance and investigate methods to infer the current signal complexity from the compressive observations. We then study a particular application, namely that of sub-Nyquist sampling and processing of sparse analog multiband signals in spectral, angular and spatial domains.Seit dem Aufkommen der ersten digitalen Verarbeitungseinheiten hat die Bedeutung der digitalen Signalverarbeitung stetig zugenommen. Heutzutage findet die meiste Signalverarbeitung im digitalen Bereich statt, was erfordert, dass analoge Signale zuerst abgetastet und digitalisiert werden, bevor relevante Daten daraus extrahiert werden können. Jahrzehntelang hat die herkömmliche äquidistante Abtastung, die durch das Nyquist-Abtasttheorem bestimmt wird, zu diesem Zweck ein nahezu universelles Mittel bereitgestellt. Der kürzliche explosive Anstieg der Anforderungen an die Datenerfassung, -speicherung und -verarbeitung hat jedoch die Fähigkeiten herkömmlicher Erfassungssysteme in vielen Anwendungsbereichen an ihre Grenzen gebracht. Durch eine alternative Sichtweise auf den Signalerfassungsprozess können Ideen aus der sparse Signalverarbeitung und einer ihrer Hauptanwendungsgebiete, Compressed Sensing (CS), dazu beitragen, einige dieser Probleme zu mindern. Basierend auf der Annahme, dass der Informationsgehalt eines Signals oft viel geringer ist als was von der nativen Repräsentation vorgegeben, stellt CS ein alternatives Konzept für die Erfassung und Verarbeitung bereit, das versucht, die Abtastrate unter Beibehaltung des Signalinformationsgehalts zu reduzieren. In dieser Arbeit untersuchen wir einige der Grundlagen des endlichdimensionalen CSFrameworks und seine Verbindung mit Sub-Nyquist Abtastung und Verarbeitung von sparsen analogen Signalen. Obwohl es seit mehr als einem Jahrzehnt ein Schwerpunkt aktiver Forschung ist, gibt es noch erhebliche Lücken beim Verständnis der Auswirkungen von komprimierenden Ansätzen auf die Signalwiedergewinnung und die Verarbeitungsleistung, insbesondere bei rauschbehafteten Umgebungen und in Bezug auf praktische Messaufgaben. In dieser Dissertation untersuchen wir, wie sich die Anwendung eines komprimierenden Messkerns auf die Signal- und Rauschcharakteristiken auf die Signalrückgewinnungsleistung auswirkt. Wir erforschen auch Methoden, um die aktuelle Signal-Sparsity-Order aus den komprimierten Messungen abzuleiten, ohne auf die Nyquist-Raten-Verarbeitung zurückzugreifen, und zeigen den Vorteil, den sie für den Wiederherstellungsprozess bietet. Nachdem gehen wir zu einer speziellen Anwendung, nämlich der Sub-Nyquist-Abtastung und Verarbeitung von sparsen analogen Multibandsignalen. Innerhalb des Sub-Nyquist-Abtastung untersuchen wir drei verschiedene Multiband-Szenarien, die Multiband-Sensing in der spektralen, Winkel und räumlichen-Domäne einbeziehen.Since the advent of the first digital processing units, the importance of digital signal processing has been steadily rising. Today, most signal processing happens in the digital domain, requiring that analog signals be first sampled and digitized before any relevant data can be extracted from them. For decades, conventional uniform sampling that is governed by the Nyquist sampling theorem has provided an almost universal means to this end. The recent explosion of the demands for data acquisition, storage and processing, however, has pushed the capabilities of conventional acquisition systems to their limits in many application areas. By offering an alternative view on the signal acquisition process, ideas from sparse signal processing and one of its main beneficiaries compressed sensing (CS), have the potential to assist alleviating some of these problems. Building on the premise that the signal information rate is often much lower than what is dictated by its native representation, CS provides an alternative acquisition and processing framework that attempts to reduce the sampling rate while preserving the information content of the signal. In this thesis, we explore some of the basic foundations of the finite-dimensional CS framework and its connection to sub-Nyquist sampling and processing of sparse continuous analog signals with application to multiband sensing. Despite being a focus of active research for over a decade, there still remain signi_cant gaps in understanding the implications that compressive approaches have on the signal recovery and processing performance, especially against noisy settings and in relation to practical sampling problems. This dissertation aims at filling some of these gaps. More specifically, we look into the ways the application of a compressive measurement kernel impacts signal and noise characteristics and the relation it has to the signal recovery performance. We also investigate methods to infer the current complexity of the signal scene from the reduced-rate compressive observations without resorting to Nyquist-rate processing and show the advantage this knowledge offers to the recovery process. Having considered some of the universal aspects of compressive systems, we then move to studying a particular application, namely that of sub-Nyquist sampling and processing of sparse analog multiband signals. Within the sub-Nyquist sampling framework, we examine three different multiband scenarios that involve multiband sensing in spectral, angular and spatial domains. For each of them, we provide a sub-Nyquist receiver architecture, develop recovery methods and numerically evaluate their performance