2,087 research outputs found
Neuromorphic Vision Based Multivehicle Detection and Tracking for Intelligent Transportation System
Neuromorphic vision sensor is a new passive sensing modality and a frameless sensor with a number of advantages over traditional cameras. Instead of wastefully sending entire images at fixed frame rate, neuromorphic vision sensor only transmits the local pixel-level changes caused by the movement in a scene"jats:italic" at the time they occur"/jats:italic". This results in advantageous characteristics, in terms of low energy consumption, high dynamic range, sparse event stream, and low response latency, which can be very useful in intelligent perception systems for modern intelligent transportation system (ITS) that requires efficient wireless data communication and low power embedded computing resources. In this paper, we propose the first neuromorphic vision based multivehicle detection and tracking system in ITS. The performance of the system is evaluated with a dataset recorded by a neuromorphic vision sensor mounted on a highway bridge. We performed a preliminary multivehicle tracking-by-clustering study using three classical clustering approaches and four tracking approaches. Our experiment results indicate that, by making full use of the low latency and sparse event stream, we could easily integrate an online tracking-by-clustering system running at a high frame rate, which far exceeds the real-time capabilities of traditional frame-based cameras. If the accuracy is prioritized, the tracking task can also be performed robustly at a relatively high rate with different combinations of algorithms. We also provide our dataset and evaluation approaches serving as the first neuromorphic benchmark in ITS and hopefully can motivate further research on neuromorphic vision sensors for ITS solutions.
Document type: Articl
Novel data association methods for online multiple human tracking
PhD ThesisVideo-based multiple human tracking has played a crucial role in many applications
such as intelligent video surveillance, human behavior analysis, and
health-care systems. The detection based tracking framework has become
the dominant paradigm in this research eld, and the major task is to accurately
perform the data association between detections across the frames.
However, online multiple human tracking, which merely relies on the detections
given up to the present time for the data association, becomes more
challenging with noisy detections, missed detections, and occlusions. To
address these challenging problems, there are three novel data association
methods for online multiple human tracking are presented in this thesis,
which are online group-structured dictionary learning, enhanced detection
reliability and multi-level cooperative fusion.
The rst proposed method aims to address the noisy detections and
occlusions. In this method, sequential Monte Carlo probability hypothesis
density (SMC-PHD) ltering is the core element for accomplishing the
tracking task, where the measurements are produced by the detection based
tracking framework. To enhance the measurement model, a novel adaptive
gating strategy is developed to aid the classi cation of measurements. In
addition, online group-structured dictionary learning with a maximum voting
method is proposed to estimate robustly the target birth intensity. It
enables the new-born targets in the tracking process to be accurately initialized
from noisy sensor measurements. To improve the adaptability of the
group-structured dictionary to target appearance changes, the simultaneous
codeword optimization (SimCO) algorithm is employed for the dictionary
update.
The second proposed method relates to accurate measurement selection
of detections, which is further to re ne the noisy detections prior to the tracking
pipeline. In order to achieve more reliable measurements in the Gaussian
mixture (GM)-PHD ltering process, a global-to-local enhanced con dence
rescoring strategy is proposed by exploiting the classi cation power of a mask
region-convolutional neural network (R-CNN). Then, an improved pruning
algorithm namely soft-aggregated non-maximal suppression (Soft-ANMS) is
devised to further enhance the selection step. In addition, to avoid the misuse
of ambiguous measurements in the tracking process, person re-identi cation
(ReID) features driven by convolutional neural networks (CNNs) are integrated
to model the target appearances.
The third proposed method focuses on addressing the issues of missed
detections and occlusions. This method integrates two human detectors
with di erent characteristics (full-body and body-parts) in the GM-PHD
lter, and investigates their complementary bene ts for tracking multiple
targets. For each detector domain, a novel discriminative correlation matching
(DCM) model for integration in the feature-level fusion is proposed, and
together with spatio-temporal information is used to reduce the ambiguous
identity associations in the GM-PHD lter. Moreover, a robust fusion
center is proposed within the decision-level fusion to mitigate the sensitivity
of missed detections in the fusion process, thereby improving the fusion
performance and tracking consistency.
The e ectiveness of these proposed methods are investigated using the
MOTChallenge benchmark, which is a framework for the standardized evaluation
of multiple object tracking methods. Detailed evaluations on challenging
video datasets, as well as comparisons with recent state-of-the-art
techniques, con rm the improved multiple human tracking performance
Single to multiple target, multiple type visual tracking
Visual tracking is a key task in applications such as intelligent surveillance, humancomputer interaction (HCI), human-robot interaction (HRI), augmented reality (AR), driver assistance systems, and medical applications. In this thesis, we make three main novel contributions for target tracking in video sequences.
First, we develop a long-term model-free single target tracking by learning discriminative correlation filters and an online classifier that can track a target of interest in both sparse and crowded scenes. In this case, we learn two different correlation filters, translation and scale correlation filters, using different visual features. We also include a re-detection module that can re-initialize the tracker in case of tracking failures due to long-term occlusions.
Second, a multiple target, multiple type filtering algorithm is developed using Random Finite Set (RFS) theory. In particular, we extend the standard Probability Hypothesis Density (PHD) filter for multiple type of targets, each with distinct detection properties, to develop multiple target, multiple type filtering, N-type PHD filter, where N ≥ 2, for handling confusions that can occur among target types at the measurements level. This method takes into account not only background false positives (clutter), but also confusions between target detections, which are in general different in character from background clutter. Then, under the assumptions of Gaussianity and linearity, we extend Gaussian mixture (GM) implementation of the standard PHD filter for the proposed N-type PHD filter termed as N-type GM-PHD filter.
Third, we apply this N-type GM-PHD filter to real video sequences by integrating object detectors’ information into this filter for two scenarios. In the first scenario, a tri-GM-PHD filter is applied to real video sequences containing three types of multiple targets in the same scene, two football teams and a referee, using separate but confused detections. In the second scenario, we use a dual GM-PHD filter for tracking pedestrians and vehicles in the same scene handling their detectors’ confusions. For both cases, Munkres’s variant of the Hungarian assignment algorithm is used to associate tracked target identities between frames.
We make extensive evaluations of these developed algorithms and find out that our methods outperform their corresponding state-of-the-art approaches by a large margin.EPSR
Tracking Target Signal Strengths on a Grid using Sparsity
Multi-target tracking is mainly challenged by the nonlinearity present in the
measurement equation, and the difficulty in fast and accurate data association.
To overcome these challenges, the present paper introduces a grid-based model
in which the state captures target signal strengths on a known spatial grid
(TSSG). This model leads to \emph{linear} state and measurement equations,
which bypass data association and can afford state estimation via
sparsity-aware Kalman filtering (KF). Leveraging the grid-induced sparsity of
the novel model, two types of sparsity-cognizant TSSG-KF trackers are
developed: one effects sparsity through -norm regularization, and the
other invokes sparsity as an extra measurement. Iterative extended KF and
Gauss-Newton algorithms are developed for reduced-complexity tracking, along
with accurate error covariance updates for assessing performance of the
resultant sparsity-aware state estimators. Based on TSSG state estimates, more
informative target position and track estimates can be obtained in a follow-up
step, ensuring that track association and position estimation errors do not
propagate back into TSSG state estimates. The novel TSSG trackers do not
require knowing the number of targets or their signal strengths, and exhibit
considerably lower complexity than the benchmark hidden Markov model filter,
especially for a large number of targets. Numerical simulations demonstrate
that sparsity-cognizant trackers enjoy improved root mean-square error
performance at reduced complexity when compared to their sparsity-agnostic
counterparts.Comment: Submitted to IEEE Trans. on Signal Processin
Robust Multi-target Tracking with Bootstrapped-GLMB Filter
This dissertation presents novel multi-target tracking algorithms that obviate the need for prior knowledge of system parameters such as clutter rate, detection probabilities, and birth models. Information on these parameters is unknown but important to tracking performance. The proposed algorithms exploit the advantages of existing RFS trackers and filters by bootstrapping them. This configuration inherits the efficiency of tracking target trajectories from the RFS trackers and low complexity in parameter estimation from the RFS filters
- …