Extended Object Tracking: Introduction, Overview and Applications

This article provides an elaborate overview of current research in extended object tracking. We provide a clear definition of the extended object tracking problem and discuss its delimitation to other types of object tracking. Next, different aspects of extended object modelling are extensively discussed. Subsequently, we give a tutorial introduction to two basic and well used extended object tracking approaches - the random matrix approach and the Kalman filter-based approach for star-convex shapes. The next part treats the tracking of multiple extended objects and elaborates how the large number of feasible association hypotheses can be tackled using both Random Finite Set (RFS) and Non-RFS multi-object trackers. The article concludes with a summary of current applications, where four example applications involving camera, X-band radar, light detection and ranging (lidar), red-green-blue-depth (RGB-D) sensors are highlighted.Comment: 30 pages, 19 figure

A Gaussian Process Approach for Extended Object Tracking with Random Shapes and for Dealing with Intractable Likelihoods

Tracking of arbitrarily shaped extended objects is a complex task due to the intractable analytical expression of measurement to object associations. The presence of sensor noise and clutter worsens the situation. Although a significant work has been done on the extended object tracking (EOT) problems, most of the developed methods are restricted by assumptions on the shape of the object such as stick, circle, or other axis-symmetric properties etc. This paper proposes a novel Gaussian process approach for tracking an extended object using a convolution particle filter (CPF). The new approach is shown to track irregularly shaped objects efficiently in presence of measurement noise and clutter. The mean recall and precision values for the shape, calculated by the proposed method on simulated data are around 0.9, respectively, by using 1000 particles

Deep Learning Assisted Intelligent Visual and Vehicle Tracking Systems

Sensor fusion and tracking is the ability to bring together measurements from multiple sensors of the current and past time to estimate the current state of a system. The resulting state estimate is more accurate compared with the direct sensor measurement because it balances between the state prediction based on the assumed motion model and the noisy sensor measurement. Systems can then use the information provided by the sensor fusion and tracking process to support more-intelligent actions and achieve autonomy in a system like an autonomous vehicle. In the past, widely used sensor data are structured, which can be directly used in the tracking system, e.g., distance, temperature, acceleration, and force. The measurements\u27 uncertainty can be estimated from experiments. However, currently, a large number of unstructured data sources can be generated from sensors such as cameras and LiDAR sensors, which bring new challenges to the fusion and tracking system. The traditional algorithm cannot directly use these unstructured data, and it needs another method or process to “understand” them first. For example, if a system tries to track a particular person in a video sequence, it needs to understand where the person is in the first place. However, the traditional tracking method cannot finish such a task. The measurement model for unstructured data is usually difficult to construct. Deep learning techniques provide promising solutions to this type of problem. A deep learning method can learn and understand the unstructured data to accomplish tasks such as object detection in images, object localization in LiDAR point clouds, and driver behavior prediction from the current traffic conditions. Deep-learning architectures such as deep neural networks, deep belief networks, recurrent neural networks, and convolutional neural networks have been applied to fields including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, and machine translation, where they have produced results comparable with human expert performance. How to incorporate information obtained via deep learning into our tracking system is one of the topics of this dissertation. Another challenging task is using learning methods to improve a tracking filter\u27s performance. In a tracking system, many manually tuned system parameters affect the tracking performance, e.g., the process noise covariance and measurement noise covariance in a Kalman Filter (KF). These parameters used to be estimated by running the tracking algorithm several times and selecting the one that gives the optimal performance. How to learn the system parameters automatically from data, and how to use machine learning techniques directly to provide useful information to the tracking systems are critical to the proposed tracking system. The proposed research on the intelligent tracking system has two objectives. The first objective is to make a visual tracking filter smart enough to understand unstructured data sources. The second objective is to apply learning algorithms to improve a tracking filter\u27s performance. The goal is to develop an intelligent tracking system that can understand the unstructured data and use the data to improve itself

Bayesian multiple extended target tracking using labelled random finite sets and splines

In this paper, we propose a technique for the joint tracking and labelling of multiple extended targets. To achieve multiple extended target tracking using this technique, models for the target measurement rate, kinematic component and target extension are defined and jointly propagated in time under the generalised labelled multi-Bernoulli (GLMB) filter framework. In particular, we developed a Poisson mixture variational Bayesian (PMVB) model to simultaneously estimate the measurement rate of multiple extended targets and extended target extension was modelled using B-splines. We evaluated our proposed method with various performance metrics. Results demonstrate the effectiveness of our approach

Mathematical Morphology for Quantification in Biological & Medical Image Analysis

Mathematical morphology is an established field of image processing first introduced as an application of set and lattice theories. Originally used to characterise particle distributions, mathematical morphology has gone on to be a core tool required for such important analysis methods as skeletonisation and the watershed transform. In this thesis, I introduce a selection of new image analysis techniques based on mathematical morphology. Utilising assumptions of shape, I propose a new approach for the enhancement of vessel-like objects in images: the bowler-hat transform. Built upon morphological operations, this approach is successful at challenges such as junctions and robust against noise. The bowler-hat transform is shown to give better results than competitor methods on challenging data such as retinal/fundus imagery. Building further on morphological operations, I introduce two novel methods for particle and blob detection. The first of which is developed in the context of colocalisation, a standard biological assay, and the second, which is based on Hilbert-Edge Detection And Ranging (HEDAR), with regard to nuclei detection and counting in fluorescent microscopy. These methods are shown to produce accurate and informative results for sub-pixel and supra-pixel object counting in complex and noisy biological scenarios. I propose a new approach for the automated extraction and measurement of object thickness for intricate and complicated vessels, such as brain vascular in medical images. This pipeline depends on two key technologies: semi-automated segmentation by advanced level-set methods and automatic thickness calculation based on morphological operations. This approach is validated and results demonstrating the broad range of challenges posed by these images and the possible limitations of this pipeline are shown. This thesis represents a significant contribution to the field of image processing using mathematical morphology and the methods within are transferable to a range of complex challenges present across biomedical image analysis

Space Image Processing and Orbit Estimation Using Small Aperture Optical Systems

Angles-only initial orbit determination (AIOD) methods have been used to find the orbit of satellites since the beginning of the Space Race. Given the ever increasing number of objects in orbit today, the need for accurate space situational awareness (SSA) data has never been greater. Small aperture (\u3c 0:5m) optical systems, increasingly popular in both amateur and professional circles, provide an inexpensive source of such data. However, utilizing these types of systems requires understanding their limits. This research uses a combination of image processing techniques and orbit estimation algorithms to evaluate the limits and improve the resulting orbit solution obtained using small aperture systems. Characterization of noise from physical, electronic, and digital sources leads to a better understanding of reducing noise in the images used to provide the best solution possible. Given multiple measurements, choosing the best images for use is a non-trivial process and often results in trying all combinations. In an effort to help autonomize the process, a novel “observability metric” using only information from the captured images was shown empirically as a method of choosing the best observations. A method of identifying resident space objects (RSOs) in a single image using a gradient based search algorithm was developed and tested on actual space imagery captured with a small aperture optical system. The algorithm was shown to correctly identify candidate RSOs in a variety of observational scenarios

Three-dimensional Image Processing of Identifying Toner Particle Centroids

Powder-based 3D printed products are composed of fine particles. The structure formed by the particles in the powder is expected to affect the performance of the final products constructed from them (Finney, 1970; Dinsmore, 2001; Chang, 2015; Patil, 2015). A prior study done by Patil (2015) demonstrated a method for determining the centroids and radii of spherical particles and consequently reconstructed the structure formed by the particles. Patil’s method used a Confocal Laser Scanning Microscope to capture a stack of cross-sections of fluorescent toner particles and Matlab image analysis tools to determine the particle centroid positions and radii. Patil identified each particle centroid’s XY coordinates and particle radius layer by layer, called “frame-by-frame” method; where the Z-position of the particle centroid was estimated by comparing the radius change at different layers. This thesis extends Patil’s work by automatically locating particle centroids in 3D space. The researcher built an algorithm, named “3D particle sighting method,” for processing the same stacks of two-dimensional images that Patil used. The algorithm at first, created a three-dimensional image matrix and then processed it by convolving with a 3D kernel to locate local maxima, which pinpointed the centroid locations of the particles. This method treated the stack of images as a 3D image matrix and the convolution operation automatically located the particle centroids. By treating Patil’s results as the ground truth, the results revealed that the average delta distance between the particle centroids identified through Patil’s method and the automated method was 1.02 microns (+/- 0.93 microns). Since the diameter of the particles is around 10 microns, this error is small compared to the size of the particles, and the results of the 3D particle sighting method are acceptable. In addition, this automated method need 1/5 of the processing time compared to Patil’s frame-by-frame method

Advanced signal processing techniques for multi-target tracking

The multi-target tracking problem essentially involves the recursive joint estimation of the state of unknown and time-varying number of targets present in a tracking scene, given a series of observations. This problem becomes more challenging because the sequence of observations is noisy and can become corrupted due to miss-detections and false alarms/clutter. Additionally, the detected observations are indistinguishable from clutter. Furthermore, whether the target(s) of interest are point or extended (in terms of spatial extent) poses even more technical challenges. An approach known as random finite sets provides an elegant and rigorous framework for the handling of the multi-target tracking problem. With a random finite sets formulation, both the multi-target states and multi-target observations are modelled as finite set valued random variables, that is, random variables which are random in both the number of elements and the values of the elements themselves. Furthermore, compared to other approaches, the random finite sets approach possesses a desirable characteristic of being free of explicit data association prior to tracking. In addition, a framework is available for dealing with random finite sets and is known as finite sets statistics. In this thesis, advanced signal processing techniques are employed to provide enhancements to and develop new random finite sets based multi-target tracking algorithms for the tracking of both point and extended targets with the aim to improve tracking performance in cluttered environments. To this end, firstly, a new and efficient Kalman-gain aided sequential Monte Carlo probability hypothesis density (KG-SMC-PHD) filter and a cardinalised particle probability hypothesis density (KG-SMC-CPHD) filter are proposed. These filters employ the Kalman- gain approach during weight update to correct predicted particle states by minimising the mean square error between the estimated measurement and the actual measurement received at a given time in order to arrive at a more accurate posterior. This technique identifies and selects those particles belonging to a particular target from a given PHD for state correction during weight computation. The proposed SMC-CPHD filter provides a better estimate of the number of targets. Besides the improved tracking accuracy, fewer particles are required in the proposed approach. Simulation results confirm the improved tracking performance when evaluated with different measures. Secondly, the KG-SMC-(C)PHD filters are particle filter (PF) based and as with PFs, they require a process known as resampling to avoid the problem of degeneracy. This thesis proposes a new resampling scheme to address a problem with the systematic resampling method which causes a high tendency of resampling very low weight particles especially when a large number of resampled particles are required; which in turn affect state estimation. Thirdly, the KG-SMC-(C)PHD filters proposed in this thesis perform filtering and not tracking , that is, they provide only point estimates of target states but do not provide connected estimates of target trajectories from one time step to the next. A new post processing step using game theory as a solution to this filtering - tracking problem is proposed. This approach was named the GTDA method. This method was employed in the KG-SMC-(C)PHD filter as a post processing technique and was evaluated using both simulated and real data obtained using the NI-USRP software defined radio platform in a passive bi-static radar system. Lastly, a new technique for the joint tracking and labelling of multiple extended targets is proposed. To achieve multiple extended target tracking using this technique, models for the target measurement rate, kinematic component and target extension are defined and jointly propagated in time under the generalised labelled multi-Bernoulli (GLMB) filter framework. The GLMB filter is a random finite sets-based filter. In particular, a Poisson mixture variational Bayesian (PMVB) model is developed to simultaneously estimate the measurement rate of multiple extended targets and extended target extension was modelled using B-splines. The proposed method was evaluated with various performance metrics in order to demonstrate its effectiveness in tracking multiple extended targets

Tracing of active particles

Tese de mestrado, Ciência de Dados, Universidade de Lisboa, Faculdade de Ciências, 2022Matéria mole refere-se a sistemas onde os fenómenos interessantes ocorrem tipicamente à temperatura ambiente. Ao estudar o seu comportamento podemos inferir propriedades sobre materiais úteis a muitas aplicações tecnológicas. Compreender as trajetórias de partículas individuais em tais sistemas é sem dúvida uma das tarefas mais desafiantes do estudo da sua dinâmica coletiva. Recentemente, tem havido um esforço crescente para desenvolver técnicas de aprendizagem automática, por exemplo, redes neuronais, para identificar diferentes tipos de partículas bem como seguir as suas trajetórias. Aqui, estendemos as técnicas existentes para considerar partículas anisotrópicas. Treinamos uma rede neuronal para identificar a posição e orientação da partícula anisotrópica (x, y, θ) usando imagens experimentais. Usamos camadas de convolução devido ao seu poder de extrair informação. O nosso objetivo foi recolher dados posicionais usando vídeos, uma tarefa normalmente conhecida como rastreamento visual. Para tal, foi necessário decompor os vídeos em imagens e só depois fazer previsões para detetar as partículas alvo. Posteriormente, todas as previsões foram ligadas para completar o rastreio final das partículas. Treinámos o nosso modelo profundo para duas experiências diferentes. Na primeira experiência, juntámos grãos de arroz num plano bidimensional, detetámos as partículas e observamos o desempenho do método; na segunda experiência, consideramos robôs com forma alongada (aproximadamente elíptica) na presença de obstáculos cilíndricos, após completar o rastreamento das partículas, foi possível deduzir conhecimento sobre o sistemas de estudos. Em ambas as experiências obtivemos resultados promissores usando a implementação de aprendizagem profunda. Explorámos as limitações do nosso método, mas acima de tudo, apresentamos um argumento convincente em favor do uso desta técnica em estudos futuros envolvendo informações da orientação das partículas de sistemas de matéria mole.Ensembles of soft matter compose systems where interesting phenomena typically occur at room temperature. By studying their behavior we are able to infer properties about materials useful in many technological applications. To resolve the trajectory of individual particles is arguably one of the most challenging tasks in the study of their collective dynamics. Recently, there has been an increasing effort to develop machine learning techniques, e.g. neural networks, to identify different particle species and follow their trajectories. Here, we extend existing techniques to consider anisotropic particles. We trained a neural network to identify anisotropic particle’s position and orientation (x, y, θ) using experimental images. We used convolution layers due to their power in extracting features from images. Our goal is to collect positional data using videos, a task commonly known as visual tracking, so we decomposed the videos into images and then made predictions to detect the target particles on each individual frame. We linked all predictions from different frames to complete tracking. We trained our deep model for two different experiments. In the first experiment, we packed rice grains on a two-dimensional plane, we made detec tion and observed the method’s performance; in the second experiment, we considered rod-like robots in the presence of cylindrical obstacles, we tracked the particles and deduced knowledge about the system they composed. In both experiments, we obtained promising results using the deep learning implementation. We explored the limitations of the method, but above all, we made a compelling argument in favor of this technique’s use in future studies involving orientational information of particles in soft matter systems