Situation assessment: an end-to-end process for the detection of objects of interest

International audienceIn this article, semi-automatic approaches are developed for wide area situation assessment in near-real-time. The two-step method consists of two granularity levels. The first entity assessment uses a new multi-target tracking algorithm (hybridization of GM-CPHD filter and MHT with road constraints) on GMTI data. The situation is then assessed by detecting objects of interest such as convoys with other data types (SAR, video). These detections are based on Bayesian networks and their credibilistic counterpart

Audio‐Visual Speaker Tracking

Target motion tracking found its application in interdisciplinary fields, including but not limited to surveillance and security, forensic science, intelligent transportation system, driving assistance, monitoring prohibited area, medical science, robotics, action and expression recognition, individual speaker discrimination in multi‐speaker environments and video conferencing in the fields of computer vision and signal processing. Among these applications, speaker tracking in enclosed spaces has been gaining relevance due to the widespread advances of devices and technologies and the necessity for seamless solutions in real‐time tracking and localization of speakers. However, speaker tracking is a challenging task in real‐life scenarios as several distinctive issues influence the tracking process, such as occlusions and an unknown number of speakers. One approach to overcome these issues is to use multi‐modal information, as it conveys complementary information about the state of the speakers compared to single‐modal tracking. To use multi‐modal information, several approaches have been proposed which can be classified into two categories, namely deterministic and stochastic. This chapter aims at providing multimedia researchers with a state‐of‐the‐art overview of tracking methods, which are used for combining multiple modalities to accomplish various multimedia analysis tasks, classifying them into different categories and listing new and future trends in this field

Multi Sensor Multi Target Perception and Tracking for Informed Decisions in Public Road Scenarios

Multi-target tracking in public traffic calls for a tracking system with automated track initiation and termination facilities in a randomly evolving driving environment. Besides, the key problem of data association needs to be handled effectively considering the limitations in the computational resources on-board an autonomous car. The challenge of the tracking problem is further evident in the use of high-resolution automotive sensors which return multiple detections per object. Furthermore, it is customary to use multiple sensors that cover different and/or over-lapping Field of View and fuse sensor detections to provide robust and reliable tracking. As a consequence, in high-resolution multi-sensor settings, the data association uncertainty, and the corresponding tracking complexity increases pointing to a systematic approach to handle and process sensor detections. In this work, we present a multi-target tracking system that addresses target birth/initiation and death/termination processes with automatic track management features. These tracking functionalities can help facilitate perception during common events in public traffic as participants (suddenly) change lanes, navigate intersections, overtake and/or brake in emergencies, etc. Various tracking approaches including the ones based on joint integrated probability data association (JIPDA) filter, Linear Multi-target Integrated Probabilistic Data Association (LMIPDA) Filter, and their multi-detection variants are adapted to specifically include algorithms that handle track initiation and termination, clutter density estimation and track management. The utility of the filtering module is further elaborated by integrating it into a trajectory tracking problem based on model predictive control. To cope with tracking complexity in the case of multiple high-resolution sensors, we propose a hybrid scheme that combines the approaches of data clustering at the local sensor and multiple detections tracking schemes at the fusion layer. We implement a track-to-track fusion scheme that de-correlates local (sensor) tracks to avoid double counting and apply a measurement partitioning scheme to re-purpose the LMIPDA tracking algorithm to multi-detection cases. In addition to the measurement partitioning approach, a joint extent and kinematic state estimation scheme are integrated into the LMIPDA approach to facilitate perception and tracking of an individual as well as group targets as applied to multi-lane public traffic. We formulate the tracking problem as a two hierarchical layer. This arrangement enhances the multi-target tracking performance in situations including but not limited to target initialization(birth process), target occlusion, missed detections, unresolved measurement, target maneuver, etc. Also, target groups expose complex individual target interactions to help in situation assessment which is challenging to capture otherwise. The simulation studies are complemented by experimental studies performed on single and multiple (group) targets. Target detections are collected from a high-resolution radar at a frequency of 20Hz; whereas RTK-GPS data is made available as ground truth for one of the target vehicle\u27s trajectory

Bayesian multi-target tracking: application to total internal reflection fluorescence microscopy

This thesis focuses on the problem of automated tracking of tiny cellular and sub-cellular structures, known as particles, in the sequences acquired from total internal reflection fluorescence microscopy (TIRFM) imaging technique. Our primary biological motivation is to develop an automated system for tracking the sub-cellular structures involving exocytosis (an intracellular mechanism) which is helpful for studying the possible causes of the defects in diseases such as diabetes and obesity. However, all methods proposed in this thesis are generalized to be applicable for a wide range of particle tracking applications. A reliable multi-particle tracking method should be capable of tracking numerous similar objects in the presence of high levels of noise, high target density and complex motions and interactions. In this thesis, we choose the Bayesian filtering framework as our main approach to deal with this problem. We focus on the approaches that work based on detections. Therefore, in this thesis, we first propose a method that robustly detects the particles in the noisy TIRFM sequences with inhomogeneous and time-varying background. In order to evaluate our detection and tracking methods on the sequences with known and reliable ground truth, we also present a framework for generating realistic synthetic TIRFM data. To propose a reliable multi-particle tracking method for TIRFM sequences, we suggest a framework by combining two robust Bayesian filters, the interacting multiple model and joint probabilistic data association (IMM-JPDA) filters. The performance of our particle tracking method is compared against those of several popular and state-of-the art particle tracking approaches on both synthetic and real sequences. Although our approach performs well in tracking particles, it can be very computationally demanding for the applications with dense targets with poor detections. To propose a computationally cheap, but reliable, multi-particle tracking method, we investigate the performance of a recent multi-target Bayesian filter based on random finite theory, the probability hypothesis density (PHD) filter, on our application. To this end, we propose a general framework for tracking particles using this filter. Moreover, we assess the performance of our proposed PHD filter on both synthetic and real sequences with high level of noise and particle density. We compare its results from both aspects of accuracy and processing time against our IMM-JPDA filter. Finally, we suggest a framework for tracking particles in a challenging problem where the noise characteristic and the background intensity of sequences change during the acquisition process which make detection profile and clutter rate time-variant. To deal with this, we propose a bootstrap filter using another type of the random finite set based Bayesian filters, the cardinalized PHD (CPHD) filter, composed of an estimator and a tracker. The estimator adaptively estimates the required meta parameters for the tracker such as clutter rate and the detection probability while the tracker estimates the state of the targets. We evaluate the performance of our bootstrap on both synthetic and real sequences under these time-varying conditions. Moreover, its performance is compared against those of our other particle trackers as well as the state-of-the art particle tracking approaches

Robust Multi-target Tracking with Bootstrapped-GLMB Filter

This dissertation presents novel multi-target tracking algorithms that obviate the need for prior knowledge of system parameters such as clutter rate, detection probabilities, and birth models. Information on these parameters is unknown but important to tracking performance. The proposed algorithms exploit the advantages of existing RFS trackers and filters by bootstrapping them. This configuration inherits the efficiency of tracking target trajectories from the RFS trackers and low complexity in parameter estimation from the RFS filters