Ground moving target tracking with space-time adaptive radar

Ground moving target tracking by airborne radar provides situational awareness of vehicle movements in the supervised region. Vehicles are detected by applying space time adaptive processing to the received multi channel radar data. The detections are then fed to a tracking algorithm that processes them to tracks. In literature, radar signal processing and ground target tracking are treated as two separate topics and results are not validated by experimental data. The first objective of this thesis is to provide a closer link between these fields. The second objective is to show that tracking performance can be improved by providing additional data from the radar signal processing to the tracking step. The third objective is to validate the algorithm and the performance improvement using experimental data. As a result this thesis presents a unified treatment of ground moving target tracking from radar raw data to established tracks. A complete reference algorithm for ground moving target tracking based on the Gaussian mixture probability hypothesis density filter is presented. In particular, Jacobians of the observation process are derived. They are presented in such a form that immediate implementation in a programming language is possible. In the course of this thesis a measurement campaign with the experimental radar PAMIR of Fraunhofer FHR was conducted. The experiment included two GPS equipped reference vehicles and a multitude of targets of opportunity. Tracking results obtained with this experimental data and the reference tracking algorithm of this thesis are shown. The thesis also enhances the reference target tracking algorithm by a parameter that characterizes the variance of the direction of arrival measurement of the target signal. This parameter is determined adaptively depending on the estimated signal strength and the clutter background. The major contribution with regard to this enhancement is a thorough experimental validation: Firstly, a comparison between GPS based measurements and radar based measurements of the direction of arrival shows that this variance captures the distribution of measurement errors excellently. Secondly, tracking results are compared to the GPS tracks of the ground truth vehicles. It is found that the enhanced algorithm yields superior track quality with respect to both track accuracy and track continuity.Bodenzielverfolgung mit luftgestütztem Radar liefert das Lagebild von Fahrzeug­bewegungen innerhalb des beobachteten Gebiets. Fahrzeuge werden durch die Anwendung von Raum-Zeit adaptiver Signalverarbeitung (STAP) entdeckt. Die Entdeckungen werden dann von einem Zielverfolgungsalgorithmus zu Zielspuren verarbeitet. In der Literatur werden Radarsignalverarbeitung und Zielverfolgung als zwei getrennte Forschungsfelder behandelt und die Bodenzielverfolgung wird nicht anhand von Realdaten validiert. Das erste Ziel dieser Arbeit ist, eine engere Verbindung zwischen beiden Feldern herzustellen. Das zweite Ziel ist zu zeigen, dass die Qualität der Zielverfolgung durch das Verwenden zusätzlicher, durch die Radarsignalverarbeitung gewonnene Information verbessert werden kann. Das dritte Ziel ist, die Funktionalität der Zielverfolgung und die Verbesserung der Leistung durch experimentelle Realdaten zu belegen. Somit stellt diese Arbeit eine Gesamtbehandlung der Bodenzielverfolgung von den Radar-Rohdaten bis zu Zielspuren dar. Es wird ein vollständiger, auf dem Gaussian Mixture Probability Hypothesis Density Filter basierender Referenzalgorithmus für die Bodenzielverfolgung entwickelt. Insbesondere werden Jacobimatrizen der Beobachtungsfunktion hergeleitet. Sie werden in der Arbeit so dargestellt, dass sie direkt in einer Programmiersprache implementiert werden können. Im Zuge dieser Arbeit wurde ein Zielverfolgungs-Experiment mit dem Experimentalsystem PAMIR des Fraunhofer FHR durchgeführt. In dem Experiment wurden neben einer Vielzahl von Gelegenheitszielen zwei mit GPS-Geräten ausgerüstete Fahrzeuge von dem Radar beobachtet. Auf Basis dieses Experiments und des Referenzalgorithmus werden Zielverfolgungsergebnisse vorgestellt. Darüber hinaus erweitert diese Arbeit den Referenzalgorithmus um einen Parameter, der die Varianz der Richtungsschätzung des Zielsignals charakterisiert. Dieser Parameter wird adaptiv anhand der geschätzten Signalstärke und der Stärke störender Bodenrückstreuungen festgelegt. Der wesentliche Beitrag dieser Arbeit in Bezug auf diese Erweiterung ist eine gründliche experimentelle Validierung. Erstens zeigt der Vergleich von GPS- und Radar-basierten Richtungsschätzungen, dass dieser Parameter die Verteilung des Messfehlers exzellent beschreibt. Zweitens werden Zielverfolgungsergebnisse mit den GPS-Spuren verglichen. Es zeigt sich, dass der erweiterte Algorithmus sowohl in Bezug auf die Spurgenauigkeit als auch in Bezug auf die Spurkontinuität die Zielverfolgung verbessert

Suivi Multi-Locuteurs avec des Informations Audio-Visuelles pour la Perception des Robots

Robot perception plays a crucial role in human-robot interaction (HRI). Perception system provides the robot information of the surroundings and enables the robot to give feedbacks. In a conversational scenario, a group of people may chat in front of the robot and move freely. In such situations, robots are expected to understand where are the people, who are speaking, or what are they talking about. This thesis concentrates on answering the first two questions, namely speaker tracking and diarization. We use different modalities of the robot’s perception system to achieve the goal. Like seeing and hearing for a human-being, audio and visual information are the critical cues for a robot in a conversational scenario. The advancement of computer vision and audio processing of the last decade has revolutionized the robot perception abilities. In this thesis, we have the following contributions: we first develop a variational Bayesian framework for tracking multiple objects. The variational Bayesian framework gives closed-form tractable problem solutions, which makes the tracking process efficient. The framework is first applied to visual multiple-person tracking. Birth and death process are built jointly with the framework to deal with the varying number of the people in the scene. Furthermore, we exploit the complementarity of vision and robot motorinformation. On the one hand, the robot’s active motion can be integrated into the visual tracking system to stabilize the tracking. On the other hand, visual information can be used to perform motor servoing. Moreover, audio and visual information are then combined in the variational framework, to estimate the smooth trajectories of speaking people, and to infer the acoustic status of a person- speaking or silent. In addition, we employ the model to acoustic-only speaker localization and tracking. Online dereverberation techniques are first applied then followed by the tracking system. Finally, a variant of the acoustic speaker tracking model based on von-Mises distribution is proposed, which is specifically adapted to directional data. All the proposed methods are validated on datasets according to applications.La perception des robots joue un rôle crucial dans l’interaction homme-robot (HRI). Le système de perception fournit les informations au robot sur l’environnement, ce qui permet au robot de réagir en consequence. Dans un scénario de conversation, un groupe de personnes peut discuter devant le robot et se déplacer librement. Dans de telles situations, les robots sont censés comprendre où sont les gens, ceux qui parlent et de quoi ils parlent. Cette thèse se concentre sur les deux premières questions, à savoir le suivi et la diarisation des locuteurs. Nous utilisons différentes modalités du système de perception du robot pour remplir cet objectif. Comme pour l’humain, l’ouie et la vue sont essentielles pour un robot dans un scénario de conversation. Les progrès de la vision par ordinateur et du traitement audio de la dernière décennie ont révolutionné les capacités de perception des robots. Dans cette thèse, nous développons les contributions suivantes : nous développons d’abord un cadre variationnel bayésien pour suivre plusieurs objets. Le cadre bayésien variationnel fournit des solutions explicites, rendant le processus de suivi très efficace. Cette approche est d’abord appliqué au suivi visuel de plusieurs personnes. Les processus de créations et de destructions sont en adéquation avecle modèle probabiliste proposé pour traiter un nombre variable de personnes. De plus, nous exploitons la complémentarité de la vision et des informations du moteur du robot : d’une part, le mouvement actif du robot peut être intégré au système de suivi visuel pour le stabiliser ; d’autre part, les informations visuelles peuvent être utilisées pour effectuer l’asservissement du moteur. Par la suite, les informations audio et visuelles sont combinées dans le modèle variationnel, pour lisser les trajectoires et déduire le statut acoustique d’une personne : parlant ou silencieux. Pour experimenter un scenario où l’informationvisuelle est absente, nous essayons le modèle pour la localisation et le suivi des locuteurs basé sur l’information acoustique uniquement. Les techniques de déréverbération sont d’abord appliquées, dont le résultat est fourni au système de suivi. Enfin, une variante du modèle de suivi des locuteurs basée sur la distribution de von-Mises est proposée, celle-ci étant plus adaptée aux données directionnelles. Toutes les méthodes proposées sont validées sur des bases de données specifiques à chaque application

Acoustic Speaker Localization with Strong Reverberation and Adaptive Feature Filtering with a Bayes RFS Framework

The thesis investigates the challenges of speaker localization in presence of strong reverberation, multi-speaker tracking, and multi-feature multi-speaker state filtering, using sound recordings from microphones. Novel reverberation-robust speaker localization algorithms are derived from the signal and room acoustics models. A multi-speaker tracking filter and a multi-feature multi-speaker state filter are developed based upon the generalized labeled multi-Bernoulli random finite set framework. Experiments and comparative studies have verified and demonstrated the benefits of the proposed methods

Signals and Images in Sea Technologies

Life below water is the 14th Sustainable Development Goal (SDG) envisaged by the United Nations and is aimed at conserving and sustainably using the oceans, seas, and marine resources for sustainable development. It is not difficult to argue that signals and image technologies may play an essential role in achieving the foreseen targets linked to SDG 14. Besides increasing the general knowledge of ocean health by means of data analysis, methodologies based on signal and image processing can be helpful in environmental monitoring, in protecting and restoring ecosystems, in finding new sensor technologies for green routing and eco-friendly ships, in providing tools for implementing best practices for sustainable fishing, as well as in defining frameworks and intelligent systems for enforcing sea law and making the sea a safer and more secure place. Imaging is also a key element for the exploration of the underwater world for various scopes, ranging from the predictive maintenance of sub-sea pipelines and other infrastructure projects, to the discovery, documentation, and protection of sunken cultural heritage. The scope of this Special Issue encompasses investigations into techniques and ICT approaches and, in particular, the study and application of signal- and image-based methods and, in turn, exploration of the advantages of their application in the previously mentioned areas

Eye movements and natural tasks in an extended environment

Eye movements can be thought of as a window onto pre-conscious thought. Patterns of visual fixations over time as well as space can reveal cognitive strategies that are not amenable to conscious control or verbalization. A spatial analysis of an eye movement trace usually emphasizes the role that eye movements have in moving the retinal image of an object of interest from the periphery to the fovea for closer inspection. It is generally believed that a sequence of fixations across a region of space builds up the perception of a high-resolution field of view everywhere. Recent studies have shown that this perception is largely illusory. The visual-perceptual system prefers to maintain a limited internal representation of physical objects in the world and uses the environment as an external source of information, accessing the information only at the time it is needed. The goal of this research effort was to investigate the role that eye movements have in the performance of everyday tasks in a natural environment. A series of four experiments were conducted that represent an attempt to step away from the classical psychophysical approach of studying eye movements widiin the confines and contaol of the laboratory. There exists little precedence for this kind of approach, partly because past research efforts have emphasized a linear systems method to render the analysis tractable, and partly because the technology that is required to perform these experiments has not existed until recently. The hardware that was developed by the Visual Perception Laboratory at RIT specifically addresses the portability concerns that are crucial for successfully studying eye movements during natural tasks in a non-linear extended environment. A model was developed to describe the temporal sequencing of eye movements in terms of a hierarchical structure of goal-oriented tasks, with individual fixations considered the lowest level of the hierarchy. The analysis gives evidence for the sequencing of eye movements based on a desire to maximize the efficiency of task performance over time by anticipating future activities. The purpose of this sequencing is to enhance interaction with the world under conditions of limited memory representations rather than to create the perception of a high-resolution field of view

Online Audio-Visual Multi-Source Tracking and Separation: A Labeled Random Finite Set Approach

The dissertation proposes an online solution for separating an unknown and time-varying number of moving sources using audio and visual data. The random finite set framework is used for the modeling and fusion of audio and visual data. This enables an online tracking algorithm to estimate the source positions and identities for each time point. With this information, a set of beamformers can be designed to separate each desired source and suppress the interfering sources

MIMO OFDM Radar-Communication System with Mutual Interference Cancellation

This work describes the OFDM-based MIMO Radar-Communication System, intended for operation in a multiple-user network, especially the automotive sector in the vehicle-to vehicle/infrastructure network. The OFDM signals however are weak towards frequency offsets causing subcarrier misalignment and corrupts the radar estimation and the demodulation of the communication signal. A simple yet effective interference cancellation algorithm is detailed here with real time measurement verification

Acoustic source localisation and tracking using microphone arrays

This thesis considers the domain of acoustic source localisation and tracking in an indoor environment. Acoustic tracking has applications in security, human-computer interaction, and the diarisation of meetings. Source localisation and tracking is typically a computationally expensive task, making it hard to process on-line, especially as the number of speakers to track increases. Much of the literature considers single-source localisation, however a practical system must be able to cope with multiple speakers, possibly active simultaneously, without knowing beforehand how many speakers are present. Techniques are explored for reducing the computational requirements of an acoustic localisation system. Techniques to localise and track multiple active sources are also explored, and developed to be more computationally efficient than the current state of the art algorithms, whilst being able to track more speakers. The first contribution is the modification of a recent single-speaker source localisation technique, which improves the localisation speed. This is achieved by formalising the implicit assumption by the modified algorithm that speaker height is uniformly distributed on the vertical axis. Estimating height information effectively reduces the search space where speakers have previously been detected, but who may have moved over the horizontal-plane, and are unlikely to have significantly changed height. This is developed to allow multiple non-simultaneously active sources to be located. This is applicable when the system is given information from a secondary source such as a set of cameras allowing the efficient identification of active speakers rather than just the locations of people in the environment. The next contribution of the thesis is the application of a particle swarm technique to significantly further decrease the computational cost of localising a single source in an indoor environment, compared the state of the art. Several variants of the particle swarm technique are explored, including novel variants designed specifically for localising acoustic sources. Each method is characterised in terms of its computational complexity as well as the average localisation error. The techniques’ responses to acoustic noise are also considered, and they are found to be robust. A further contribution is made by using multi-optima swarm techniques to localise multiple simultaneously active sources. This makes use of techniques which extend the single-source particle swarm techniques to finding multiple optima of the acoustic objective function. Several techniques are investigated and their performance in terms of localisation accuracy and computational complexity is characterised. Consideration is also given to how these metrics change when an increasing number of active speakers are to be localised. Finally, the application of the multi-optima localisation methods as an input to a multi-target tracking system is presented. Tracking multiple speakers is a more complex task than tracking single acoustic source, as observations of audio activity must be associated in some way with distinct speakers. The tracker used is known to be a relatively efficient technique, and the nature of the multi-optima output format is modified to allow the application of this technique to the task of speaker tracking