Single to multiple target, multiple type visual tracking

Visual tracking is a key task in applications such as intelligent surveillance, humancomputer interaction (HCI), human-robot interaction (HRI), augmented reality (AR), driver assistance systems, and medical applications. In this thesis, we make three main novel contributions for target tracking in video sequences. First, we develop a long-term model-free single target tracking by learning discriminative correlation filters and an online classifier that can track a target of interest in both sparse and crowded scenes. In this case, we learn two different correlation filters, translation and scale correlation filters, using different visual features. We also include a re-detection module that can re-initialize the tracker in case of tracking failures due to long-term occlusions. Second, a multiple target, multiple type filtering algorithm is developed using Random Finite Set (RFS) theory. In particular, we extend the standard Probability Hypothesis Density (PHD) filter for multiple type of targets, each with distinct detection properties, to develop multiple target, multiple type filtering, N-type PHD filter, where N ≥ 2, for handling confusions that can occur among target types at the measurements level. This method takes into account not only background false positives (clutter), but also confusions between target detections, which are in general different in character from background clutter. Then, under the assumptions of Gaussianity and linearity, we extend Gaussian mixture (GM) implementation of the standard PHD filter for the proposed N-type PHD filter termed as N-type GM-PHD filter. Third, we apply this N-type GM-PHD filter to real video sequences by integrating object detectors’ information into this filter for two scenarios. In the first scenario, a tri-GM-PHD filter is applied to real video sequences containing three types of multiple targets in the same scene, two football teams and a referee, using separate but confused detections. In the second scenario, we use a dual GM-PHD filter for tracking pedestrians and vehicles in the same scene handling their detectors’ confusions. For both cases, Munkres’s variant of the Hungarian assignment algorithm is used to associate tracked target identities between frames. We make extensive evaluations of these developed algorithms and find out that our methods outperform their corresponding state-of-the-art approaches by a large margin.EPSR

Advanced signal processing techniques for multi-target tracking

The multi-target tracking problem essentially involves the recursive joint estimation of the state of unknown and time-varying number of targets present in a tracking scene, given a series of observations. This problem becomes more challenging because the sequence of observations is noisy and can become corrupted due to miss-detections and false alarms/clutter. Additionally, the detected observations are indistinguishable from clutter. Furthermore, whether the target(s) of interest are point or extended (in terms of spatial extent) poses even more technical challenges. An approach known as random finite sets provides an elegant and rigorous framework for the handling of the multi-target tracking problem. With a random finite sets formulation, both the multi-target states and multi-target observations are modelled as finite set valued random variables, that is, random variables which are random in both the number of elements and the values of the elements themselves. Furthermore, compared to other approaches, the random finite sets approach possesses a desirable characteristic of being free of explicit data association prior to tracking. In addition, a framework is available for dealing with random finite sets and is known as finite sets statistics. In this thesis, advanced signal processing techniques are employed to provide enhancements to and develop new random finite sets based multi-target tracking algorithms for the tracking of both point and extended targets with the aim to improve tracking performance in cluttered environments. To this end, firstly, a new and efficient Kalman-gain aided sequential Monte Carlo probability hypothesis density (KG-SMC-PHD) filter and a cardinalised particle probability hypothesis density (KG-SMC-CPHD) filter are proposed. These filters employ the Kalman- gain approach during weight update to correct predicted particle states by minimising the mean square error between the estimated measurement and the actual measurement received at a given time in order to arrive at a more accurate posterior. This technique identifies and selects those particles belonging to a particular target from a given PHD for state correction during weight computation. The proposed SMC-CPHD filter provides a better estimate of the number of targets. Besides the improved tracking accuracy, fewer particles are required in the proposed approach. Simulation results confirm the improved tracking performance when evaluated with different measures. Secondly, the KG-SMC-(C)PHD filters are particle filter (PF) based and as with PFs, they require a process known as resampling to avoid the problem of degeneracy. This thesis proposes a new resampling scheme to address a problem with the systematic resampling method which causes a high tendency of resampling very low weight particles especially when a large number of resampled particles are required; which in turn affect state estimation. Thirdly, the KG-SMC-(C)PHD filters proposed in this thesis perform filtering and not tracking , that is, they provide only point estimates of target states but do not provide connected estimates of target trajectories from one time step to the next. A new post processing step using game theory as a solution to this filtering - tracking problem is proposed. This approach was named the GTDA method. This method was employed in the KG-SMC-(C)PHD filter as a post processing technique and was evaluated using both simulated and real data obtained using the NI-USRP software defined radio platform in a passive bi-static radar system. Lastly, a new technique for the joint tracking and labelling of multiple extended targets is proposed. To achieve multiple extended target tracking using this technique, models for the target measurement rate, kinematic component and target extension are defined and jointly propagated in time under the generalised labelled multi-Bernoulli (GLMB) filter framework. The GLMB filter is a random finite sets-based filter. In particular, a Poisson mixture variational Bayesian (PMVB) model is developed to simultaneously estimate the measurement rate of multiple extended targets and extended target extension was modelled using B-splines. The proposed method was evaluated with various performance metrics in order to demonstrate its effectiveness in tracking multiple extended targets

Random finite sets in multi-target tracking - efficient sequential MCMC implementation

Over the last few decades multi-target tracking (MTT) has proved to be a challenging and attractive research topic. MTT applications span a wide variety of disciplines, including robotics, radar/sonar surveillance, computer vision and biomedical research. The primary focus of this dissertation is to develop an effective and efficient multi-target tracking algorithm dealing with an unknown and time-varying number of targets. The emerging and promising Random Finite Set (RFS) framework provides a rigorous foundation for optimal Bayes multi-target tracking. In contrast to traditional approaches, the collection of individual targets is treated as a set-valued state. The intent of this dissertation is two-fold; first to assert that the RFS framework not only is a natural, elegant and rigorous foundation, but also leads to practical, efficient and reliable algorithms for Bayesian multi-target tracking, and second to provide several novel RFS based tracking algorithms suitable for the specific Track-Before-Detect (TBD) surveillance application. One main contribution of this dissertation is a rigorous derivation and practical implementation of a novel algorithm well suited to deal with multi-target tracking problems for a given cardinality. The proposed Interacting Population-based MCMC-PF algorithm makes use of several Metropolis-Hastings samplers running in parallel, which interact through genetic variation. Another key contribution concerns the design and implementation of two novel algorithms to handle a varying number of targets. The first approach exploits Reversible Jumps. The second approach is built upon the concepts of labeled RFSs and multiple cardinality hypotheses. The performance of the proposed algorithms is also demonstrated in practical scenarios, and shown to significantly outperform conventional multi-target PF in terms of track accuracy and consistency. The final contribution seeks to exploit external information to increase the performance of the surveillance system. In multi-target scenarios, kinematic constraints from the interaction of targets with their environment or other targets can restrict target motion. Such motion constraint information is integrated by using a fixed-lag smoothing procedure, named Knowledge-Based Fixed-Lag Smoother (KB-Smoother). The proposed combination IP-MCMC-PF/KB-Smoother yields enhanced tracking

Novel data association methods for online multiple human tracking

PhD ThesisVideo-based multiple human tracking has played a crucial role in many applications such as intelligent video surveillance, human behavior analysis, and health-care systems. The detection based tracking framework has become the dominant paradigm in this research eld, and the major task is to accurately perform the data association between detections across the frames. However, online multiple human tracking, which merely relies on the detections given up to the present time for the data association, becomes more challenging with noisy detections, missed detections, and occlusions. To address these challenging problems, there are three novel data association methods for online multiple human tracking are presented in this thesis, which are online group-structured dictionary learning, enhanced detection reliability and multi-level cooperative fusion. The rst proposed method aims to address the noisy detections and occlusions. In this method, sequential Monte Carlo probability hypothesis density (SMC-PHD) ltering is the core element for accomplishing the tracking task, where the measurements are produced by the detection based tracking framework. To enhance the measurement model, a novel adaptive gating strategy is developed to aid the classi cation of measurements. In addition, online group-structured dictionary learning with a maximum voting method is proposed to estimate robustly the target birth intensity. It enables the new-born targets in the tracking process to be accurately initialized from noisy sensor measurements. To improve the adaptability of the group-structured dictionary to target appearance changes, the simultaneous codeword optimization (SimCO) algorithm is employed for the dictionary update. The second proposed method relates to accurate measurement selection of detections, which is further to re ne the noisy detections prior to the tracking pipeline. In order to achieve more reliable measurements in the Gaussian mixture (GM)-PHD ltering process, a global-to-local enhanced con dence rescoring strategy is proposed by exploiting the classi cation power of a mask region-convolutional neural network (R-CNN). Then, an improved pruning algorithm namely soft-aggregated non-maximal suppression (Soft-ANMS) is devised to further enhance the selection step. In addition, to avoid the misuse of ambiguous measurements in the tracking process, person re-identi cation (ReID) features driven by convolutional neural networks (CNNs) are integrated to model the target appearances. The third proposed method focuses on addressing the issues of missed detections and occlusions. This method integrates two human detectors with di erent characteristics (full-body and body-parts) in the GM-PHD lter, and investigates their complementary bene ts for tracking multiple targets. For each detector domain, a novel discriminative correlation matching (DCM) model for integration in the feature-level fusion is proposed, and together with spatio-temporal information is used to reduce the ambiguous identity associations in the GM-PHD lter. Moreover, a robust fusion center is proposed within the decision-level fusion to mitigate the sensitivity of missed detections in the fusion process, thereby improving the fusion performance and tracking consistency. The e ectiveness of these proposed methods are investigated using the MOTChallenge benchmark, which is a framework for the standardized evaluation of multiple object tracking methods. Detailed evaluations on challenging video datasets, as well as comparisons with recent state-of-the-art techniques, con rm the improved multiple human tracking performance

지능형 전기자동차를 위한 최적화 기법의 적용

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2015. 2. 서승우.Batteries are often damaged by a high peak power and a steep variation of the power since it has a relatively low power density. In order to reduce battery damage, the battery/super-capacitor (SC) hybrid energy storage system (HESS) has been utilized since the SC can act as a buffer against large magnitudes and rapid fluctuations in power. The major objective regarding the battery/SC HESS is to minimize the magnitude/variation of the battery power and the power loss. To achieve the objective, I formulate optimization problems to provide the optimal HESS power using given load operation profiles. In addition, I propose an algorithm using a barrier method and a Multiplicative Increase Additive Decrease method for providing a feasible optimal solution for energy management in HESS. The battery/SC HESS can be effectively utilized for Electric Vehicles (EVs) because high peak power or rapid charging/discharging occur frequently in driving situations. However, the optimization method proposed in the second chapter cannot be adopted for EVs because it is difficult to obtain the future driving profile in advance. To calculate the optimal power of the battery/SC without the future profiles, I propose a method for computing the reference voltage of the SC based on the characteristic of power-train and the vehicle dynamics. In addition, I formulate the real-time optimization problem that minimizes the magnitude/variation of the battery power and the power loss simultaneously. To improve the power control for the battery/SC HESS in EVs, it is necessary to know the future motor power in advance. They can be derived from the future speed/acceleration of the vehicle through the method proposed in the third chapter if the future speed/acceleration can be predicted. Fortunately, there are many prediction techniques such as car following models, path planning algorithms and model predictive schemes, which are based on results of target tracking. Therefore, the driving environments, e.g., moving objects, should be accurately estimated. To improve the multi-target estimation accuracy even if there are many false detections, I propose a robust multi-target tracking scheme based on the GMPHD filter. The proposed scheme includes the processing step of evaluating multiple states/measurements which is designed to overcome the weight under/overestimation problem. Furthermore, it includes the step of generating the birth intensity for the next iteration using the measurements not associated with any tracked states. I also show that the proposed method can be extended to nonlinear Gaussian models.1 Introduction 1 1.1 Background and Motivations . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Contributions and Outline of the Dissertation . . . . . . . . . . . . . 3 1.2.1 EnergyManagement Optimization in a Battery/Supercapacitor Hybrid Energy Storage System . . . . . . . . . . . . . . . . . 3 1.2.2 Real-time Optimization for Power Management Systems of a Battery/Supercapacitor Hybrid Energy Storage System in Electric Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2.3 Robust Multi-Target Tracking Scheme against False Detections based on Gaussian Mixture Probability Hypothesis Density Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2 EnergyManagement Optimization in a Battery/Supercapacitor Hy- brid Energy Storage System 7 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Active Hybrid Energy Storage Systems . . . . . . . . . . . . . . . . . 11 2.2.1 A Review of Active Hybrid Energy Storage Systems . . . . . 13 2.2.2 Considered HESS Topology . . . . . . . . . . . . . . . . . . . 14 2.3 HESS Energy Management Optimization . . . . . . . . . . . . . . . 15 2.3.1 Notations and Assumptions . . . . . . . . . . . . . . . . . . . 15 2.3.2 Minimization of Magnitude/Fluctuation of Battery Power . . 17 2.3.3 Minimization of the Power Loss . . . . . . . . . . . . . . . . . 21 2.3.4 Minimization of the Dual Objective Functions . . . . . . . . . 22 2.4 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.4.1 Computation by Solver . . . . . . . . . . . . . . . . . . . . . 24 2.4.2 Parameter Adjustment Algorithm . . . . . . . . . . . . . . . 24 2.4.3 Analysis of the Total Number of Iterations in the Algorithm . 26 2.5 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.5.1 Review of Previous Approach . . . . . . . . . . . . . . . . . . 29 2.5.2 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . 32 2.5.3 Adjustment of the Boundary Parameters in the Algorithm . . 33 2.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 2.7 Proof of 2nd constraint in P2 . . . . . . . . . . . . . . . . . . . . . . 35 3 Real-time Optimization for Power Management Systems of a Bat- tery/Supercapacitor Hybrid Energy Storage System in Electric Ve- hicles 36 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 3.2 System Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.2.1 Powertrain Model . . . . . . . . . . . . . . . . . . . . . . . . 40 3.2.2 Regenerative Braking System . . . . . . . . . . . . . . . . . . 43 3.2.3 Battery/SC Hybrid Energy Storage Systems . . . . . . . . . . 45 3.3 Power Control System for HESS . . . . . . . . . . . . . . . . . . . . 47 3.3.1 Computation of SC Reference Voltage . . . . . . . . . . . . . 49 3.3.2 Computation of the optimal SC power . . . . . . . . . . . . . 51 3.4 Simulation Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 3.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4 Robust Multi-Target Tracking Scheme based on Gaussian Mixture Probability Hypothesis Density Filter 69 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 4.2 Background and Motivation . . . . . . . . . . . . . . . . . . . . . . . 73 4.2.1 Prediction of Future Driving Profile . . . . . . . . . . . . . . 73 4.2.2 Brief Overview of The Conventional GM-PHD Filter . . . . . 74 4.2.3 Problems of the GM-PHD Filter . . . . . . . . . . . . . . . . 77 4.3 The Proposed Robust GM-PHD Filter . . . . . . . . . . . . . . . . . 83 4.3.1 Target Prediction and PHD Update Component Construction 85 4.3.2 State and Measurement Evaluation . . . . . . . . . . . . . . . 86 4.3.3 PHD Updating and Merging . . . . . . . . . . . . . . . . . . 89 4.3.4 Duplication Check . . . . . . . . . . . . . . . . . . . . . . . . 91 4.3.5 Birth Intensity Generation for the Next Iteration . . . . . . . 91 4.4 Nonlinear Gaussian Model Extension . . . . . . . . . . . . . . . . . 92 4.5 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 4.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5 Conclusion and Future Work 110Docto

Optical based statistical space objects tracking for catalogue maintenance

The number of space objects has grown substantially in the past decades due to new launches, regular mission activities, and breakup events. This has significantly affected the space environment and the development of the space industry. To ensure safe operation of space assets, Space Situational Awareness (SSA) has attracted considerable attention in recent years. One primary strategy in SSA is to establish and maintain a Space Object Catalogue (SOC) to provide timely updated data for SSA applications, e.g., conjunction analysis, collision avoidance manoeuvring. This thesis investigates three techniques for SOC maintenance, namely the tracklet association method for initial orbit determination, the multi-target tracking method for the refinement of orbital state estimation, and multi-sensor tasking method for the optimisation of sensor resources. Generally speaking, due to the limited number of optical sensors used to track the large population of space objects, the obtained observational arcs for many targets are very short. Such short arcs, which contain a small number of angular observations, are referred as tracklets. Given such limited data, typical orbit determination methods, e.g., Laplace, Gaussian, Double-R methods, may fail to produce a valid orbital solution. By contrast, tracklet association methods compare and correlate multiple tracklets across time, and following successful association, a reliable initial orbital state can be further determined for SOC maintenance. This thesis proposes an improved initial value problem optimisation method for accurate and efficient tracklet association, and a common ellipse method to distinguish false associations of tracklets from objects in the same constellation. The proposed methods are validated using real optical data collected from the Mount Stromlo Observatory, Canberra, Australia. Furthermore, another challenging task in SSA is to track multiple objects for the maintenance of a catalog. The Bayesian multi-target tracking filter addresses this issue by associating measurements to initially known or newly detected targets and simultaneously estimating the timevarying number of targets and their orbital states. In order to achieve efficient tracking of the new space objects, a novel birth model using the Boundary Value Problem (BVP) approach is proposed. The proposed BVP birth model is implemented in the Labelled Multi-Bernoulli (LMB) filter, which is an efficient multi-target tracker developed based on the Random Finite Set (RFS) theory, for improved computational efficiency of new space object tracking. Simulation results indicate that the computational efficiency of the proposed method significantly outperforms the state-of-the-art methods. Finally, as limited sensors are available for SOC maintenance, an appropriate sensor tasking scheme is essential for the optimisation of sensor resources. The optimal sensor tasking command allocates multiple sensors to take the best action and produce useful measurements for more accurate orbital state estimation. In this thesis, an analytical form is derived for the Rényi divergence of LMB RFS in which each target state density is a single Gaussian component. The obtained analytical Rényi divergence is formulated as a reward function for multi-sensor tasking, which improves the computational efficiency, especially for large-scale space object tracking. In addition, this thesis further investigates the benefits of using the analytical Rényi  divergence and various space-based and ground-based sensor networks for accurate tracking of objects in geosynchronous Earth orbit

Sonar attentive underwater navigation in structured environment

One of the fundamental requirements of a persistently Autonomous Underwater Vehicle (AUV) is a robust navigation system. The success of most complex robotic tasks depends on the accuracy of a vehicle’s navigation system. In a basic form, an AUV estimates its position using an on-board navigation sensors through Dead-Reckoning (DR). However DR navigation systems tends to drift in the long run due to accumulated measurement errors. One way of mitigating this problem require the use of Simultaneous Localization and Mapping (SLAM) by concurrently mapping external environment features. The performance of a SLAM navigation system depends on the availability of enough good features in the environment. On the contrary, a typical underwater structured environment (harbour, pier or oilfield) has a limited amount of sonar features in a limited locations, hence exploitation of good features is a key for effective underwater SLAM. This thesis develops a novel attentive sonar line feature based SLAM framework that improves the performance of a SLAM navigation by steering a multibeam sonar sensor,which is mounted on a pan and tilt unit, towards feature-rich regions of the environment. A sonar salience map is generated at each vehicle pose to identify highly informative and stable regions of the environment. Results from a simulated test and real AUV experiment show an attentive SLAM performs better than a passive counterpart by repeatedly visiting good sonar landmarks

Automated brain segmentation methods for clinical quality MRI and CT images

Alzheimer’s disease (AD) is a progressive neurodegenerative disorder associated with brain tissue loss. Accurate estimation of this loss is critical for the diagnosis, prognosis, and tracking the progression of AD. Structural magnetic resonance imaging (sMRI) and X-ray computed tomography (CT) are widely used imaging modalities that help to in vivo map brain tissue distributions. As manual image segmentations are tedious and time-consuming, automated segmentation methods are increasingly applied to head MRI and head CT images to estimate brain tissue volumes. However, existing automated methods can be applied only to images that have high spatial resolution and their accuracy on heterogeneous low-quality clinical images has not been tested. Further, automated brain tissue segmentation methods for CT are not available, although CT is more widely acquired than MRI in the clinical setting. For these reasons, large clinical imaging archives are unusable for research studies. In this work, we identify and develop automated tissue segmentation and brain volumetry methods that can be applied to clinical quality MRI and CT images. In the first project, we surveyed the current MRI methods and validated the accuracy of these methods when applied to clinical quality images. We then developed CTSeg, a tissue segmentation method for CT images, by adopting the MRI technique that exhibited the highest reliability. CTSeg is an atlas-based statistical modeling method that relies on hand-curated features and cannot be applied to images of subjects with different diseases and age groups. Advanced deep learning-based segmentation methods use hierarchical representations and learn complex features in a data-driven manner. In our final project, we develop a fully automated deep learning segmentation method that uses contextual information to segment clinical quality head CT images. The application of this method on an AD dataset revealed larger differences between brain volumes of AD and control subjects. This dissertation demonstrates the potential of applying automated methods to large clinical imaging archives to answer research questions in a variety of studies