Object Tracking

Object tracking consists in estimation of trajectory of moving objects in the sequence of images. Automation of the computer object tracking is a difficult task. Dynamics of multiple parameters changes representing features and motion of the objects, and temporary partial or full occlusion of the tracked objects have to be considered. This monograph presents the development of object tracking algorithms, methods and systems. Both, state of the art of object tracking methods and also the new trends in research are described in this book. Fourteen chapters are split into two sections. Section 1 presents new theoretical ideas whereas Section 2 presents real-life applications. Despite the variety of topics contained in this monograph it constitutes a consisted knowledge in the field of computer object tracking. The intention of editor was to follow up the very quick progress in the developing of methods as well as extension of the application

Multiple Integrated Navigation Sensors for Improving Occupancy Grid FastSLAM

An autonomous vehicle must accurately observe its location within the environment to interact with objects and accomplish its mission. When its environment is unknown, the vehicle must construct a map detailing its surroundings while using it to maintain an accurate location. Such a vehicle is faced with the circularly defined Simultaneous Localization and Mapping (SLAM) problem. However difficult, SLAM is a critical component of autonomous vehicle exploration with applications to search and rescue. To current knowledge, this research presents the first SLAM solution to integrate stereo cameras, inertial measurements, and vehicle odometry into a Multiple Integrated Navigation Sensor (MINS) path. The implementation combines the MINS path with LIDAR to observe and map the environment using the FastSLAM algorithm. In real-world tests, a mobile ground vehicle equipped with these sensors completed a 140 meter loop around indoor hallways. This SLAM solution produces a path that closes the loop and remains within 1 meter of truth, reducing the error 92% from an image-inertial navigation system and 79% from odometry FastSLAM

Target Tracking in UWB Multistatic Radars

Detection, localization and tracking of non-collaborative objects moving inside an area is of great interest to many surveillance applications. An ultra- wideband (UWB) multistatic radar is considered as a good infrastructure for such anti-intruder systems, due to the high range resolution provided by the UWB impulse-radio and the spatial diversity achieved with a multistatic configuration. Detection of targets, which are typically human beings, is a challenging task due to reflections from unwanted objects in the area, shadowing, antenna cross-talks, low transmit power, and the blind zones arised from intrinsic peculiarities of UWB multistatic radars. Hence, we propose more effective detection, localization, as well as clutter removal techniques for these systems. However, the majority of the thesis effort is devoted to the tracking phase, which is an essential part for improving the localization accuracy, predicting the target position and filling out the missed detections. Since UWB radars are not linear Gaussian systems, the widely used tracking filters, such as the Kalman filter, are not expected to provide a satisfactory performance. Thus, we propose the Bayesian filter as an appropriate candidate for UWB radars. In particular, we develop tracking algorithms based on particle filtering, which is the most common approximation of Bayesian filtering, for both single and multiple target scenarios. Also, we propose some effective detection and tracking algorithms based on image processing tools. We evaluate the performance of our proposed approaches by numerical simulations. Moreover, we provide experimental results by channel measurements for tracking a person walking in an indoor area, with the presence of a significant clutter. We discuss the existing practical issues and address them by proposing more robust algorithms

3D VISUAL TRACKING USING A SINGLE CAMERA

automated surveillance and motion based recognition. 3D tracking address the localization of moving target is the 3D space. Therefore, 3D tracking requires 3D measurement of the moving object which cannot be obtained from 2D cameras. Existing 3D tracking systems use multiple cameras for computing the depth of field and it is only used in research laboratories. Millions of surveillance cameras are installed worldwide and all of them capture 2D images. Therefore, 3D tracking cannot be performed with these cameras unless multiple cameras are installed at each location in order to compute the depth. This means installing millions of new cameras which is not a feasible solution. This work introduces a novel depth estimation method from a single 2D image using triangulation. This method computes the absolute depth of field for any object in the scene with high accuracy and short computational time. The developed method is used for performing 3D visual tracking using a single camera by providing the depth of field and ground coordinates of the moving object for each frame accurately and efficiently. Therefore, this technique can help in transforming existing 2D tracking and 2D video analytics into 3D without incurring additional costs. This makes video surveillance more efficient and increases its usage in human life. The proposed methodology uses background subtraction process for detecting a moving object in the image. Then, the newly developed depth estimation method is used for computing the 3D measurement of the moving target. Finally, the unscented Kalman filter is used for tracking the moving object given the 3D measurement obtained by the triangulation method. This system has been test and validated using several video sequences and it shows good performance in term of accuracy and computational complexity

Fusion Of Multiple Inertial Measurements Units And Its Application In Reduced Cost, Size, Weight, And Power Synthetic Aperture Radars

Position navigation and timing (PNT) is the concept of determining where an object is on the Earth (position), the destination of the object (navigation), and when the object is in these positions (timing). In autonomous applications, these three attributes are crucial to determining the control inputs required to control and move the platform through an area. Traditionally, the position information is gathered using mainly a global positioning system (GPS) which can provide positioning sufficient for most PNT applications. However, GPS navigational solutions are limited by slower update rates, limited accuracy, and can be unreliable. GPS solutions update slower due to the signal having to travel a great distance from the satellite to the receiver. Additionally, the accuracy of the GPS solution relies on the environment of the receiver and the effects caused by additional reflections that introduce ambiguity into the positional solution. As result, the positional solution can become unstable or unreliable if the ambiguities are significant and greatly impact the accuracy of the positional solution. A common solution to addressing the shortcomings of the GPS solution is to introduce an additional sensor focused on measuring the physical state of the platform. The sensors popularly used are inertial measurement units (IMU) and can help provide faster positional accuracy as the transmission time is eliminated. Furthermore, the IMU is directly measuring physical forces that contribute to the position of the platform, therefore, the ambiguities caused by additional signal reflections are also eliminated. Although the introduction of the IMU helps mitigate some of the shortcomings of GPS, the sensors introduce a slightly different set of challenges. Since the IMUs directly measure the physical forces experienced by the platform, the position is estimated using these measurements. The estimates of position utilize the previously known position and estimate the changes to the position based on the accelerations measured by the IMUs. As the IMUs intrinsically have sensor noise and errors in their measurements, the noise errors directly impact the accuracy of the position estimated. These inaccuracies are further compounded as the erroneous position estimate is now used as the basis for future position calculations. Inertial navigation systems (INS) have been developed to pair the IMUs with the GPS to overcome the challenges brought by each sensor independently. The data provided from each sensor is processed using a technique known as data fusion where the statistical likelihood of each positional solution is evaluated and used to estimate the most likely position solution given the observations from each sensor. Data fusion allows for the navigation solution to provide a positional solution at the sampling rate of the fastest sensor while also limiting the compounding errors intrinsic to using IMUs. Synthetic aperture radar (SAR) is an application that utilizes a moving radar to synthetically generate a larger aperture to create images of a target scene. The larger aperture allows for a finer spatial resolution resulting in higher quality SAR images. For synthetic aperture radar applications, the PNT solution is fundamental to producing a quality image as the range to a target is only reported by the radar. To form an image, the range to each target must be aligned over the coherent processing interval (CPI). In doing so, the energy reflected from the target as the radar is moving can be combined coherently and resolved to a pixel in the image product. In practice, the position of the radar is measured using a navigational solution utilizing a GPS and IMU. Inaccuracies in these solutions directly contribute to the image quality in a SAR system because the measured range from the radar will not agree with the calculated range to the location represented by the pixel. As a result, the final image becomes unfocused and the target will be blurred across multiple pixels. For INS systems, increasing the accuracy of the final position estimate is dependent on the accuracy of the sensors in the system. An easy way to increase the accuracy of the INS solution is to upgrade to a higher grade IMU. As a result, the errors compounded by the IMU estimations are minimized because the intrinsic noise perturbations are smaller. The trade-off is the IMU sensors increase in cost, size, weight, and power (C-SWAP) as the quality of the sensor increases. The increase in C-SWAP is a challenge of utilizing higher grade IMUs in INS navigational solutions for SAR applications. This problem is amplified when developing miniaturized SAR systems. In this dissertation, a method of leveraging the benefits of data fusion to combine multiple IMUs to produce higher accuracy INS solutions is presented. Specifically, the C-SWAP can be reduced when utilizing lower-quality IMUs. The use of lower quality IMUs presents an additional challenge of providing positional solutions at the rates required for SAR. A method of interpolating the position provided by the fusion algorithm while maintaining positional accuracy is also presented in this dissertation. The methods presented in this dissertation are successful in providing accurate positional solutions from lower C-SWAP INS. The presented methods are verified in simulations of motion paths and the results of the fusion algorithms are evaluated for accuracy. The presented methods are instrumented in both ground and flight tests and the results are compared to a 3rd party accurate position solution for an accuracy metric. Lastly, the algorithms are implemented in a miniaturized SAR system and both ground and airborne SAR tests are conducted to evaluate the effectiveness of the algorithms. In general, the designed algorithms are capable of producing positional accuracy at the rate required to focus SAR images in a miniaturized SAR system

PROGNOSTIC MODELING FOR RELIABILITY PREDICTIONS OF POWER ELECTRONIC DEVICES

The applications of semiconductor power electronic devices, including power and RF devices, in industry have stringent requirements on their reliability. Power devices are subject to various types of failure mechanisms under various stressors. Prognostics and health management (PHM) allows detecting signs of failures, providing warnings of failures in advance, and performing condition-based maintenance. There is a pressing need to develop a robust prognostic model to detect anomalous behavior and predict the lifetime of devices that can be applicable to different types of power transistors. In the present dissertation, a comprehensive prognostic model for remaining useful life (RUL) prediction of semiconductor power electronic devices is developed. The model consists of an anomaly detection module and a RUL prediction module including a non-linear system process model describing the evolution of parametric degradation, and a measurement model. The anomaly detection module uses principal component analysis (PCA) for dimensionality reduction and feature extraction, as well as k-means clustering to establish baseline clusters in the feature space. The novel singular-value-weighted distance (SVWD) is developed as the distance measure in the feature space, based on which Fisher criterion (FC) is used for anomaly probability calculation. The system process model incorporates variables concerning loading conditions and physics-of-failure of devices, and uses particle filter (PF) approach for process model training and RUL prediction. For PF methodology, a novel resampling technique, called MHA-replacement resampling, is developed to solve the particle degeneracy in classic PF techniques and sample impoverishment in traditional resampling techniques. The developed prognostic model is first implemented on IGBT modules for validation. It was reported that the module package of power transistors was susceptible to various types of fatigue-related failure modes due to coefficient of thermal expansion (CTE) mismatches under temperature/power cycles introducing thermomechanical stresses. The physics-of-failure "driving variable" is derived from Paris equation. The model is validated on several time-series IGBT module degradation data under power cycles from literature sources, based on SIR particle filter for RUL prediction with good accuracy. Then the model is implemented on GaN HEMTs, a representative of wide-bandgap semiconductor power devices. GaN HEMTs are susceptible to degradation mechanisms such as ohmic contact inter-diffusion that leads to voiding in the field plate at high temperature under RF accelerated life tests (ALTs). The time-series data of the physics-of-failure "driving variable" is obtained from diffusion computation in Mathematica with the temperature prole coming from COMSOL thermal simulation. The RUL prediction results based on SIR lter are also satisfactory for GaN HEMTs. For each type of device, the new resampling technique is validated through performance benchmarking against state-of-the-art resampling techniques. Another reliability threat for GaN HEMTs, especially in aerospace and nuclear applications, is the degradation due to radiation effect on the device performance. Gamma radiation has been found to lead to generation of defects in AlGaN/GaN layers, which form complexes acting as carrier traps, thus reducing carrier density and current. EPC GaN HEMTs are irradiated under a wide range of Gamma ray doses and critical DC characteristics are recorded before and after radiation to quantify their shifts during the irradiation. Future work needed to allow implementation of the developed prognostic model for RUL estimation is proposed

Multichannel source separation and tracking with phase differences by random sample consensus

Blind audio source separation (BASS) is a fascinating problem that has been tackled from many different angles. The use case of interest in this thesis is that of multiple moving and simultaneously-active speakers in a reverberant room. This is a common situation, for example, in social gatherings. We human beings have the remarkable ability to focus attention on a particular speaker while effectively ignoring the rest. This is referred to as the ``cocktail party effect'' and has been the holy grail of source separation for many decades. Replicating this feat in real-time with a machine is the goal of BASS. Single-channel methods attempt to identify the individual speakers from a single recording. However, with the advent of hand-held consumer electronics, techniques based on microphone array processing are becoming increasingly popular. Multichannel methods record a sound field from various locations to incorporate spatial information. If the speakers move over time, we need an algorithm capable of tracking their positions in the room. For compact arrays with 1-10 cm of separation between the microphones, this can be accomplished by applying a temporal filter on estimates of the directions-of-arrival (DOA) of the speakers. In this thesis, we review recent work on BSS with inter-channel phase difference (IPD) features and provide extensions to the case of moving speakers. It is shown that IPD features compose a noisy circular-linear dataset. This data is clustered with the RANdom SAmple Consensus (RANSAC) algorithm in the presence of strong reverberation to simultaneously localize and separate speakers. The remarkable performance of RANSAC is due to its natural tendency to reject outliers. To handle the case of non-stationary speakers, a factorial wrapped Kalman filter (FWKF) and a factorial von Mises-Fisher particle filter (FvMFPF) are proposed that track source DOAs directly on the unit circle and unit sphere, respectively. These algorithms combine directional statistics, Bayesian filtering theory, and probabilistic data association techniques to track the speakers with mixtures of directional distributions

Blind dereverberation of speech from moving and stationary speakers using sequential Monte Carlo methods

Speech signals radiated in confined spaces are subject to reverberation due to reflections of surrounding walls and obstacles. Reverberation leads to severe degradation of speech intelligibility and can be prohibitive for applications where speech is digitally recorded, such as audio conferencing or hearing aids. Dereverberation of speech is therefore an important field in speech enhancement. Driven by consumer demand, blind speech dereverberation has become a popular field in the research community and has led to many interesting approaches in the literature. However, most existing methods are dictated by their underlying models and hence suffer from assumptions that constrain the approaches to specific subproblems of blind speech dereverberation. For example, many approaches limit the dereverberation to voiced speech sounds, leading to poor results for unvoiced speech. Few approaches tackle single-sensor blind speech dereverberation, and only a very limited subset allows for dereverberation of speech from moving speakers. Therefore, the aim of this dissertation is the development of a flexible and extendible framework for blind speech dereverberation accommodating different speech sound types, single- or multiple sensor as well as stationary and moving speakers. Bayesian methods benefit from – rather than being dictated by – appropriate model choices. Therefore, the problem of blind speech dereverberation is considered from a Bayesian perspective in this thesis. A generic sequential Monte Carlo approach accommodating a multitude of models for the speech production mechanism and room transfer function is consequently derived. In this approach both the anechoic source signal and reverberant channel are estimated using their optimal estimators by means of Rao-Blackwellisation of the state-space of unknown variables. The remaining model parameters are estimated using sequential importance resampling. The proposed approach is implemented for two different speech production models for stationary speakers, demonstrating substantial reduction in reverberation for both unvoiced and voiced speech sounds. Furthermore, the channel model is extended to facilitate blind dereverberation of speech from moving speakers. Due to the structure of measurement model, single- as well as multi-microphone processing is facilitated, accommodating physically constrained scenarios where only a single sensor can be used as well as allowing for the exploitation of spatial diversity in scenarios where the physical size of microphone arrays is of no concern. This dissertation is concluded with a survey of possible directions for future research, including the use of switching Markov source models, joint target tracking and enhancement, as well as an extension to subband processing for improved computational efficiency

Time-varying frequency analysis of bat echolocation signals using Monte Carlo methods

Echolocation in bats is a subject that has received much attention over the last few decades. Bat echolocation calls have evolved over millions of years and can be regarded as well suited to the task of active target-detection. In analysing the time-frequency structure of bat calls, it is hoped that some insight can be gained into their capabilities and limitations. Most analysis of calls is performed using non-parametric techniques such as the short time Fourier transform. The resulting time-frequency distributions are often ambiguous, leading to further uncertainty in any subsequent analysis which depends on the time-frequency distribution. There is thus a need to develop a method which allows improved time-frequency characterisation of bat echolocation calls. The aim of this work is to develop a parametric approach for signal analysis, specifically taking into account the varied nature of bat echolocation calls in the signal model. A time-varying harmonic signal model with a polynomial chirp basis is used to track the instantaneous frequency components of the signal. The model is placed within a Bayesian context and a particle filter is used to implement the filter. Marginalisation of parameters is considered, leading to the development of a new marginalised particle filter (MPF) which is used to implement the algorithm. Efficient reversible jump moves are formulated for estimation of the unknown (and varying) number of frequency components and higher harmonics. The algorithm is applied to the analysis of synthetic signals and the performance is compared with an existing algorithm in the literature which relies on the Rao-Blackwellised particle filter (RBPF) for online state estimation and a jump Markov system for estimation of the unknown number of harmonic components. A comparison of the relative complexity of the RBPF and the MPF is presented. Additionally, it is shown that the MPF-based algorithm performs no worse than the RBPF, and in some cases, better, for the test signals considered. Comparisons are also presented from various reversible jump sampling schemes for estimation of the time-varying number of tones and harmonics. The algorithm is subsequently applied to the analysis of bat echolocation calls to establish the improvements obtained from the new algorithm. The calls considered are both amplitude and frequency modulated and are of varying durations. The calls are analysed using polynomial basis functions of different orders and the performance of these basis functions is compared. Inharmonicity, which is deviation of overtones away from integer multiples of the fundamental frequency, is examined in echolocation calls from several bat species. The results conclude with an application of the algorithm to the analysis of calls from the feeding buzz, a sequence of extremely short duration calls emitted at high pulse repetition frequency, where it is shown that reasonable time-frequency characterisation can be achieved for these calls