Online Audio-Visual Multi-Source Tracking and Separation: A Labeled Random Finite Set Approach

The dissertation proposes an online solution for separating an unknown and time-varying number of moving sources using audio and visual data. The random finite set framework is used for the modeling and fusion of audio and visual data. This enables an online tracking algorithm to estimate the source positions and identities for each time point. With this information, a set of beamformers can be designed to separate each desired source and suppress the interfering sources

Robust and Efficient Inference of Scene and Object Motion in Multi-Camera Systems

Multi-camera systems have the ability to overcome some of the fundamental limitations of single camera based systems. Having multiple view points of a scene goes a long way in limiting the influence of field of view, occlusion, blur and poor resolution of an individual camera. This dissertation addresses robust and efficient inference of object motion and scene in multi-camera and multi-sensor systems. The first part of the dissertation discusses the role of constraints introduced by projective imaging towards robust inference of multi-camera/sensor based object motion. We discuss the role of the homography and epipolar constraints for fusing object motion perceived by individual cameras. For planar scenes, the homography constraints provide a natural mechanism for data association. For scenes that are not planar, the epipolar constraint provides a weaker multi-view relationship. We use the epipolar constraint for tracking in multi-camera and multi-sensor networks. In particular, we show that the epipolar constraint reduces the dimensionality of the state space of the problem by introducing a ``shared'' state space for the joint tracking problem. This allows for robust tracking even when one of the sensors fail due to poor SNR or occlusion. The second part of the dissertation deals with challenges in the computational aspects of tracking algorithms that are common to such systems. Much of the inference in the multi-camera and multi-sensor networks deal with complex non-linear models corrupted with non-Gaussian noise. Particle filters provide approximate Bayesian inference in such settings. We analyze the computational drawbacks of traditional particle filtering algorithms, and present a method for implementing the particle filter using the Independent Metropolis Hastings sampler, that is highly amenable to pipelined implementations and parallelization. We analyze the implementations of the proposed algorithm, and in particular concentrate on implementations that have minimum processing times. The last part of the dissertation deals with the efficient sensing paradigm of compressing sensing (CS) applied to signals in imaging, such as natural images and reflectance fields. We propose a hybrid signal model on the assumption that most real-world signals exhibit subspace compressibility as well as sparse representations. We show that several real-world visual signals such as images, reflectance fields, videos etc., are better approximated by this hybrid of two models. We derive optimal hybrid linear projections of the signal and show that theoretical guarantees and algorithms designed for CS can be easily extended to hybrid subspace-compressive sensing. Such methods reduce the amount of information sensed by a camera, and help in reducing the so called data deluge problem in large multi-camera systems

Compressive Acquisition and Processing of Sparse Analog Signals

Since the advent of the first digital processing units, the importance of digital signal processing has been steadily rising. Today, most signal processing happens in the digital domain, requiring that analog signals be first sampled and digitized before any relevant data can be extracted from them. The recent explosion of the demands for data acquisition, storage and processing, however, has pushed the capabilities of conventional acquisition systems to their limits in many application areas. By offering an alternative view on the signal acquisition process, ideas from sparse signal processing and one of its main beneficiaries compressed sensing (CS), aim at alleviating some of these problems. In this thesis, we look into the ways the application of a compressive measurement kernel impacts the signal recovery performance and investigate methods to infer the current signal complexity from the compressive observations. We then study a particular application, namely that of sub-Nyquist sampling and processing of sparse analog multiband signals in spectral, angular and spatial domains.Seit dem Aufkommen der ersten digitalen Verarbeitungseinheiten hat die Bedeutung der digitalen Signalverarbeitung stetig zugenommen. Heutzutage findet die meiste Signalverarbeitung im digitalen Bereich statt, was erfordert, dass analoge Signale zuerst abgetastet und digitalisiert werden, bevor relevante Daten daraus extrahiert werden können. Jahrzehntelang hat die herkömmliche äquidistante Abtastung, die durch das Nyquist-Abtasttheorem bestimmt wird, zu diesem Zweck ein nahezu universelles Mittel bereitgestellt. Der kürzliche explosive Anstieg der Anforderungen an die Datenerfassung, -speicherung und -verarbeitung hat jedoch die Fähigkeiten herkömmlicher Erfassungssysteme in vielen Anwendungsbereichen an ihre Grenzen gebracht. Durch eine alternative Sichtweise auf den Signalerfassungsprozess können Ideen aus der sparse Signalverarbeitung und einer ihrer Hauptanwendungsgebiete, Compressed Sensing (CS), dazu beitragen, einige dieser Probleme zu mindern. Basierend auf der Annahme, dass der Informationsgehalt eines Signals oft viel geringer ist als was von der nativen Repräsentation vorgegeben, stellt CS ein alternatives Konzept für die Erfassung und Verarbeitung bereit, das versucht, die Abtastrate unter Beibehaltung des Signalinformationsgehalts zu reduzieren. In dieser Arbeit untersuchen wir einige der Grundlagen des endlichdimensionalen CSFrameworks und seine Verbindung mit Sub-Nyquist Abtastung und Verarbeitung von sparsen analogen Signalen. Obwohl es seit mehr als einem Jahrzehnt ein Schwerpunkt aktiver Forschung ist, gibt es noch erhebliche Lücken beim Verständnis der Auswirkungen von komprimierenden Ansätzen auf die Signalwiedergewinnung und die Verarbeitungsleistung, insbesondere bei rauschbehafteten Umgebungen und in Bezug auf praktische Messaufgaben. In dieser Dissertation untersuchen wir, wie sich die Anwendung eines komprimierenden Messkerns auf die Signal- und Rauschcharakteristiken auf die Signalrückgewinnungsleistung auswirkt. Wir erforschen auch Methoden, um die aktuelle Signal-Sparsity-Order aus den komprimierten Messungen abzuleiten, ohne auf die Nyquist-Raten-Verarbeitung zurückzugreifen, und zeigen den Vorteil, den sie für den Wiederherstellungsprozess bietet. Nachdem gehen wir zu einer speziellen Anwendung, nämlich der Sub-Nyquist-Abtastung und Verarbeitung von sparsen analogen Multibandsignalen. Innerhalb des Sub-Nyquist-Abtastung untersuchen wir drei verschiedene Multiband-Szenarien, die Multiband-Sensing in der spektralen, Winkel und räumlichen-Domäne einbeziehen.Since the advent of the first digital processing units, the importance of digital signal processing has been steadily rising. Today, most signal processing happens in the digital domain, requiring that analog signals be first sampled and digitized before any relevant data can be extracted from them. For decades, conventional uniform sampling that is governed by the Nyquist sampling theorem has provided an almost universal means to this end. The recent explosion of the demands for data acquisition, storage and processing, however, has pushed the capabilities of conventional acquisition systems to their limits in many application areas. By offering an alternative view on the signal acquisition process, ideas from sparse signal processing and one of its main beneficiaries compressed sensing (CS), have the potential to assist alleviating some of these problems. Building on the premise that the signal information rate is often much lower than what is dictated by its native representation, CS provides an alternative acquisition and processing framework that attempts to reduce the sampling rate while preserving the information content of the signal. In this thesis, we explore some of the basic foundations of the finite-dimensional CS framework and its connection to sub-Nyquist sampling and processing of sparse continuous analog signals with application to multiband sensing. Despite being a focus of active research for over a decade, there still remain signi_cant gaps in understanding the implications that compressive approaches have on the signal recovery and processing performance, especially against noisy settings and in relation to practical sampling problems. This dissertation aims at filling some of these gaps. More specifically, we look into the ways the application of a compressive measurement kernel impacts signal and noise characteristics and the relation it has to the signal recovery performance. We also investigate methods to infer the current complexity of the signal scene from the reduced-rate compressive observations without resorting to Nyquist-rate processing and show the advantage this knowledge offers to the recovery process. Having considered some of the universal aspects of compressive systems, we then move to studying a particular application, namely that of sub-Nyquist sampling and processing of sparse analog multiband signals. Within the sub-Nyquist sampling framework, we examine three different multiband scenarios that involve multiband sensing in spectral, angular and spatial domains. For each of them, we provide a sub-Nyquist receiver architecture, develop recovery methods and numerically evaluate their performance

Bearing estimation techniques for improved performance spread spectrum receivers

The main topic of this thesis is the use of bearing estimation techniques combined with multiple antenna elements for spread spectrum receivers. The motivation behind this work is twofold: firstly, this type of receiver structure may offer the ability to locate the position of a mobile radio in an urban environment. Secondly, these algorithms permit the application of space division multiple access (SDMA) to cellular mobile radio, which can offer large system capacity increases. The structure of these receivers may naturally be divided into two parts: signal detection and spatial filtering blocks. The signal detection problem involves locating the bearings of the multipath components which arise from the transmission of the desired user’s signal. There are a number of approaches to this problem, but here the MUSIC algorithm will be adopted. This algorithm requires an initial estimate of the number of signals impinging on the receiver, a task which can be performed by model order determination techniques. A major deficiency of MUSIC is its inability to resolve the highly–correlated and coherent multipath signals which frequently occur in a spread spectrum system. One of the simplest ways to overcome this problem is to employ spatial smoothing techniques, which trade the size of the antenna array for the ability to resolve coherent signals. The minimum description length (MDL) is one method for determining the signal model order and it can easily be extended to calculating the required degree of spatial smoothing. In this thesis, an approach to analysing the probability of correct model order determination for the MDL with spatial smoothing is presented. The performance of MUSIC, combined with spatial smoothing, is also of great significance. Two smoothing algorithms, spatial smoothing and forward–backward spatial smoothing, are analysed to compare their performance. If SDMA techniques are to be deployed in cellular systems, it is important to first estimate the performance improvements available from applying antenna array spatial filters. Initially, an additive white Gaussian noise channel is used for estimating the capacity of a perfect power–controlled code division multiple access system with SDMA techniques. Results suggest that the mean interference levels are almost halved as the antenna array size doubles, permitting large capacity increases. More realistic multipath models for urban cellular radio channels are also considered. If the transmitter gives rise to a number of point source multipath components, the bearing estimation receiver is able to capture the signal energy of each multipath. However, when a multipath component has significant angular spread, bearing estimation receivers need to combine separate directional components, at an increased cost in complexity, to obtain similar results to a matched filter. Finally, a source location algorithm for urban environments is presented, based on bearing estimation of multipath components. This algorithm requires accurate knowledge of the positions of the major multipath reflectors present in the environment. With this knowledge it is possible to determine the position of a transmitting mobile unit. Simulation results suggest that the algorithm is very sensitive to angular separation of the multipath components used for the source location technique