Data association and occlusion handling for vision-based people tracking by mobile robots

This paper presents an approach for tracking multiple persons on a mobile robot with a combination of colour and thermal vision sensors, using several new techniques. First, an adaptive colour model is incorporated into the measurement model of the tracker. Second, a new approach for detecting occlusions is introduced, using a machine learning classifier for pairwise comparison of persons (classifying which one is in front of the other). Third, explicit occlusion handling is incorporated into the tracker. The paper presents a comprehensive, quantitative evaluation of the whole system and its different components using several real world data sets

Extended Object Tracking: Introduction, Overview and Applications

This article provides an elaborate overview of current research in extended object tracking. We provide a clear definition of the extended object tracking problem and discuss its delimitation to other types of object tracking. Next, different aspects of extended object modelling are extensively discussed. Subsequently, we give a tutorial introduction to two basic and well used extended object tracking approaches - the random matrix approach and the Kalman filter-based approach for star-convex shapes. The next part treats the tracking of multiple extended objects and elaborates how the large number of feasible association hypotheses can be tackled using both Random Finite Set (RFS) and Non-RFS multi-object trackers. The article concludes with a summary of current applications, where four example applications involving camera, X-band radar, light detection and ranging (lidar), red-green-blue-depth (RGB-D) sensors are highlighted.Comment: 30 pages, 19 figure

Improved data association and occlusion handling for vision-based people tracking by mobile robots

This paper presents an approach for tracking multiple persons using a combination of colour and thermal vision sensors on a mobile robot. First, an adaptive colour model is incorporated into the measurement model of the tracker. Second, a new approach for detecting occlusions is introduced, using a machine learning classifier for pairwise comparison of persons (classifying which one is in front of the other). Third, explicit occlusion handling is then incorporated into the tracker

Tracking and Fusion Methods for Extended Targets Parameterized by Center, Orientation, and Semi-axes

The improvements in sensor technology, e.g., the development of automotive Radio Detection and Ranging (RADAR) or Light Detection and Ranging (LIDAR), which are able to provide a higher detail of the sensor’s environment, have introduced new opportunities but also new challenges to target tracking. In classic target tracking, targets are assumed as points. However, this assumption is no longer valid if targets occupy more than one sensor resolution cell, creating the need for extended targets, modeling the shape in addition to the kinematic parameters. Different shape models are possible and this thesis focuses on an elliptical shape, parameterized with center, orientation, and semi-axes lengths. This parameterization can be used to model rectangles as well. Furthermore, this thesis is concerned with multi-sensor fusion for extended targets, which can be used to improve the target tracking by providing information gathered from different sensors or perspectives. We also consider estimation of extended targets, i.e., to account for uncertainties, the target is modeled by a probability density, so we need to find a so-called point estimate. Extended target tracking provides a variety of challenges due to the spatial extent, which need to be handled, even for basic shapes like ellipses and rectangles. Among these challenges are the choice of the target model, e.g., how the measurements are distributed across the shape. Additional challenges arise for sensor fusion, as it is unclear how to best consider the geometric properties when combining two extended targets. Finally, the extent needs to be involved in the estimation. Traditional methods often use simple uniform distributions across the shape, which do not properly portray reality, while more complex methods require the use of optimization techniques or large amounts of data. In addition, for traditional estimation, metrics such as the Euclidean distance between state vectors are used. However, they might no longer be valid because they do not consider the geometric properties of the targets’ shapes, e.g., rotating an ellipse by 180 degree results in the same ellipse, but the Euclidean distance between them is not 0. In multi-sensor fusion, the same holds, i.e., simply combining the corresponding elements of the state vectors can lead to counter-intuitive fusion results. In this work, we compare different elliptic trackers and discuss more complex measurement distributions across the shape’s surface or contour. Furthermore, we discuss the problems which can occur when fusing extended target estimates from different sensors and how to handle them by providing a transformation into a special density. We then proceed to discuss how a different metric, namely the Gaussian Wasserstein (GW) distance, can be used to improve target estimation. We define an estimator and propose an approximation based on an extension of the square root distance. It can be applied on the posterior densities of the aforementioned trackers to incorporate the unique properties of ellipses in the estimation process. We also discuss how this can be applied to rectangular targets as well. Finally, we evaluate and discuss our approaches. We show the benefits of more complex target models in simulations and on real data and we demonstrate our estimation and fusion approaches compared to classic methods on simulated data.2022-01-2

Image Segmentation Using Weak Shape Priors

The problem of image segmentation is known to become particularly challenging in the case of partial occlusion of the object(s) of interest, background clutter, and the presence of strong noise. To overcome this problem, the present paper introduces a novel approach segmentation through the use of "weak" shape priors. Specifically, in the proposed method, an segmenting active contour is constrained to converge to a configuration at which its geometric parameters attain their empirical probability densities closely matching the corresponding model densities that are learned based on training samples. It is shown through numerical experiments that the proposed shape modeling can be regarded as "weak" in the sense that it minimally influences the segmentation, which is allowed to be dominated by data-related forces. On the other hand, the priors provide sufficient constraints to regularize the convergence of segmentation, while requiring substantially smaller training sets to yield less biased results as compared to the case of PCA-based regularization methods. The main advantages of the proposed technique over some existing alternatives is demonstrated in a series of experiments.Comment: 27 pages, 8 figure

Conjugate priors for Bayesian object tracking

Object tracking refers to the problem of using noisy sensor measurements to determine the location and characteristics of objects of interest in clutter. Nowadays, object tracking has found applications in numerous research venues as well as application areas, including air traffic control, maritime navigation, remote sensing, intelligent video surveillance, and more recently environmental perception, which is a key enabling technology in autonomous vehicles. This thesis studies conjugate priors for Bayesian object tracking with focus on multi-object tracking (MOT) based on sets of trajectories. Finite Set Statistics provides an elegant Bayesian formulation of MOT in terms of the theory of random finite sets (RFSs). Conjugate priors are also of great interest as they provide families of distributions that are suitable to work with when seeking accurate approximations to the true posterior distributions. Many RFS-based MOT approaches are only concerned with multi-object filtering without attempting to estimate object trajectories. An appealing approach to building tracks is by computing the multi-object densities on sets of trajectories. This leads to the development of trajectory filters, e.g., filters based on Poisson multi-Bernoulli mixture (PMBM) conjugate priors.In this thesis, [Paper A] and [Paper B] consider the problem of point object tracking where an object generates at most one measurement per scan. In [Paper A], it is shown that the trajectory MBM filter is the solution to the MOT problem for standard point object models with multi-Bernoulli birth. In addition, the multi-scan implementations of trajectory PMBM and MBM filters are presented. In [Paper B], a solution for recovering full trajectory information, via the calculation of the posterior of the set of trajectories from a sequence of multi-object filtering densities and the multi-object dynamic model, is presented. [Paper C] and [Paper D] consider the problem of ex- tended object tracking where an object may generate multiple measurements per scan. In [Paper C], the extended object PMBM filter for sets of objects is generalized to sets of trajectories. In [Paper D], a learning-based extended ob- ject tracking algorithm using a hierarchical truncated Gaussian measurement model tailored for automotive radar measurements is presented

Poisson Multi-Bernoulli Mixtures for Multiple Object Tracking

Multi-object tracking (MOT) refers to the process of estimating object trajectories of interest based on sequences of noisy sensor measurements obtained from multiple sources. Nowadays, MOT has found applications in numerous areas, including, e.g., air traffic control, maritime navigation, remote sensing, intelligent video surveillance, and more recently environmental perception, which is a key enabling technology in automated vehicles. This thesis studies Poisson multi-Bernoulli mixture (PMBM) conjugate priors for MOT. Finite Set Statistics provides an elegant Bayesian formulation of MOT based on random finite sets (RFSs), and a significant trend in RFSs-based MOT is the development of conjugate distributions in Bayesian probability theory, such as the PMBM distributions. Multi-object conjugate priors are of great interest as they provide families of distributions that are suitable to work with when seeking accurate approximations to the true posterior distributions. Many RFS-based MOT approaches are only concerned with multi-object filtering without attempting to estimate object trajectories. An appealing approach to building trajectories is by computing the multi-object densities on sets of trajectories. This leads to the development of many multi-object filters based on sets of trajectories, e.g., the trajectory PMBM filters. In this thesis, [Paper A] and [Paper B] consider the problem of point object tracking where an object generates at most one measurement per time scan. In [Paper A], a multi-scan implementation of trajectory PMBM filters via dual decomposition is presented. In [Paper B], a multi-trajectory particle smoother using backward simulation is presented for computing the multi-object posterior for sets of trajectories using a sequence of multi-object filtering densities and a multi-object dynamic model. [Paper C] and [Paper D] consider the problem of extended object tracking where an object may generate multiple measurements per time scan. In [Paper C], an extended object Poisson multi-Bernoulli (PMB) filter is presented, where the PMBM posterior density after the update step is approximated as a PMB. In [Paper D], a trajectory PMB filter for extended object tracking using belief propagation is presented, where the efficient PMB approximation is enabled by leveraging the PMBM conjugacy and the factor graph formulation

Poisson multi-Bernoulli conjugate prior for multiple extended object filtering

This paper presents a Poisson multi-Bernoulli mixture (PMBM) conjugate prior for multiple extended object filtering. A Poisson point process is used to describe the existence of yet undetected targets, while a multi-Bernoulli mixture describes the distribution of the targets that have been detected. The prediction and update equations are presented for the standard transition density and measurement likelihood. Both the prediction and the update preserve the PMBM form of the density, and in this sense the PMBM density is a conjugate prior. However, the unknown data associations lead to an intractably large number of terms in the PMBM density, and approximations are necessary for tractability. A gamma Gaussian inverse Wishart implementation is presented, along with methods to handle the data association problem. A simulation study shows that the extended target PMBM filter performs well in comparison to the extended target d-GLMB and LMB filters. An experiment with Lidar data illustrates the benefit of tracking both detected and undetected targets