Real-time visual tracking using image processing and filtering methods

The main goal of this thesis is to develop real-time computer vision algorithms in order to detect and to track targets in uncertain complex environments purely based on a visual sensor. Two major subjects addressed by this work are: 1. The development of fast and robust image segmentation algorithms that are able to search and automatically detect targets in a given image. 2. The development of sound filtering algorithms to reduce the effects of noise in signals from the image processing. The main constraint of this research is that the algorithms should work in real-time with limited computing power on an onboard computer in an aircraft. In particular, we focus on contour tracking which tracks the outline of the target represented by contours in the image plane. This thesis is concerned with three specific categories, namely image segmentation, shape modeling, and signal filtering. We have designed image segmentation algorithms based on geometric active contours implemented via level set methods. Geometric active contours are deformable contours that automatically track the outlines of objects in images. In this approach, the contour in the image plane is represented as the zero-level set of a higher dimensional function. (One example of the higher dimensional function is a three-dimensional surface for a two-dimensional contour.) This approach handles the topological changes (e.g., merging, splitting) of the contour naturally. Although geometric active contours prevail in many fields of computer vision, they suffer from the high computational costs associated with level set methods. Therefore, simplified versions of level set methods such as fast marching methods are often used in problems of real-time visual tracking. This thesis presents the development of a fast and robust segmentation algorithm based on up-to-date extensions of level set methods and geometric active contours, namely a fast implementation of Chan-Vese's (active contour) model (FICVM). The shape prior is a useful cue in the recognition of the true target. For the contour tracker, the outline of the target can be easily disrupted by noise. In geometric active contours, to cope with deviations from the true outline of the target, a higher dimensional function is constructed based on the shape prior, and the contour tracks the outline of an object by considering the difference between the higher dimensional functions obtained from the shape prior and from a measurement in a given image. The higher dimensional function is often a distance map which requires high computational costs for construction. This thesis focuses on the extraction of shape information from only the zero-level set of the higher dimensional function. This strategy compensates for inaccuracies in the calculation of the shape difference that occur when a simplified higher dimensional function is used. This is named as contour-based shape modeling. Filtering is an essential element in tracking problems because of the presence of noise in system models and measurements. The well-known Kalman filter provides an exact solution only for problems which have linear models and Gaussian distributions (linear/Gaussian problems). For nonlinear/non-Gaussian problems, particle filters have received much attention in recent years. Particle filtering is useful in the approximation of complicated posterior probability distribution functions. However, the computational burden of particle filtering prevents it from performing at full capacity in real-time applications. This thesis concentrates on improving the processing time of particle filtering for real-time applications. In principle, we follow the particle filter in the geometric active contour framework. This thesis proposes an advanced blob tracking scheme in which a blob contains shape prior information of the target. This scheme simplifies the sampling process and quickly suggests the samples which have a high probability of being the target. Only for these samples is the contour tracking algorithm applied to obtain a more detailed state estimate. Curve evolution in the contour tracking is realized by the FICVM. The dissimilarity measure is calculated by the contour based shape modeling method and the shape prior is updated when it satisfies certain conditions. The new particle filter is applied to the problems of low contrast and severe daylight conditions, to cluttered environments, and to the appearing/disappearing target tracking. We have also demonstrated the utility of the filtering algorithm for multiple target tracking in the presence of occlusions. This thesis presents several test results from simulations and flight tests. In these tests, the proposed algorithms demonstrated promising results in varied situations of tracking.Ph.D.Committee Chair: Eric N. Johnson; Committee Co-Chair: Allen R. Tannenbaum; Committee Member: Anthony J. Calise; Committee Member: Eric Feron; Committee Member: Patricio A. Vel

Performance Analysis of Bearings-only Tracking Problems for Maneuvering Target and Heterogeneous Sensor Applications

State estimation, i.e. determining the trajectory, of a maneuvering target from noisy measurements collected by a single or multiple passive sensors (e.g. passive sonar and radar) has wide civil and military applications, for example underwater surveillance, air defence, wireless communications, and self-protection of military vehicles. These passive sensors are listening to target emitted signals without emitting signals themselves which give them concealing properties. Tactical scenarios exists where the own position shall not be revealed, e.g. for tracking submarines with passive sonar or tracking an aerial target by means of electro-optic image sensors like infrared sensors. This estimation process is widely known as bearings-only tracking. On the one hand, a challenge is the high degree of nonlinearity in the estimation process caused by the nonlinear relation of angular measurements to the Cartesian state. On the other hand, passive sensors cannot provide direct target location measurements, so bearings-only tracking suffers from poor target trajectory estimation accuracy due to marginal observability from sensor measurements. In order to achieve observability, that means to be able to estimate the complete target state, multiple passive sensor measurements must be fused. The measurements can be recorded spatially distributed by multiple dislocated sensor platforms or temporally distributed by a single, moving sensor platform. Furthermore, an extended case of bearings-only tracking is given if heterogeneous measurements from targets emitting different types of signals, are involved. With this, observability can also be achieved on a single, not necessarily moving platform. In this work, a performance bound for complex motion models, i.e. piecewisely maneuvering targets with unknown maneuver change times, by means of bearings-only measurements from a single, moving sensor platform is derived and an efficient estimator is implemented and analyzed. Furthermore, an observability analysis is carried out for targets emitting acoustic and electromagnetic signals. Here, the different signal propagation velocities can be exploited to ensure observability on a single, not necessarily moving platform. Based on the theoretical performance and observability analyses a distributed fusion system has been realized by means of heterogeneous sensors, which shall detect an event and localize a threat. This is performed by a microphone array to detect sound waves emitted by the threat as well as a radar detector that detects electromagnetic emissions from the threat. Since multiple platforms are involved to provide increased observability and also redundancy against possible breakdowns, a WiFi mobile ad hoc network is used for communications. In order to keep up the network in a breakdown OLSR (optimized link state routing) routing approach is employed

Wald Sequential Probability Ratio Test for Space Object Conjunction Assessment

This paper shows how satellite owner/operators may use sequential estimates of collision probability, along with a prior assessment of the base risk of collision, in a compound hypothesis ratio test to inform decisions concerning collision risk mitigation maneuvers. The compound hypothesis test reduces to a simple probability ratio test, which appears to be a novel result. The test satisfies tolerances related to targeted false alarm and missed detection rates. This result is independent of the method one uses to compute the probability density that one integrates to compute collision probability. A well-established test case from the literature shows that this test yields acceptable results within the constraints of a typical operational conjunction assessment decision timeline. Another example illustrates the use of the test in a practical conjunction assessment scenario based on operations of the International Space Station

Two-layer particle filter for multiple target detection and tracking

This paper deals with the detection and tracking of an unknown number of targets using a Bayesian hierarchical model with target labels. To approximate the posterior probability density function, we develop a two-layer particle filter. One deals with track initiation, and the other with track maintenance. In addition, the parallel partition method is proposed to sample the states of the surviving targets

Multiple Model Adaptive Estimator Target Tracker for Maneuvering Targets in Clutter

The task of tracking a target in the presence of measurement clutter is a two-fold problem: one of handling measurement association uncertainty (due to clutter), and poorly known or significantly varying target dynamics. Measurement association uncertainty does not allow conventional tracking algorithms (such as Kalman filters) to be implemented directly. Poorly known or varying target dynamics complicate the design of any tracking filter, and filters using only a single dynamics model can rarely handle anything beyond the most benign target maneuvers. In recent years, the Multiple Hypothesis Tracker (MHT) has gained acceptance as a means of handling targets in a measurement-clutter environment. MHT algorithms rely on Gaussian mixture representations of a target\u27s current state estimate, and the number of components within these mixtures grows exponentially with each successive sensor scan. Previous research into techniques that limit the growth of Gaussian mixture components proved that the Integral Square Error cost-function-based algorithm performs well in this role. Also, multiple-model adaptive algorithms have been shown to handle poorly known target dynamics or targets that exhibit a large range of maneuverability over time with excellent results. This research integrates the ISE mixture reduction algorithm into Multiple-Model Adaptive Estimator (MMAE) and Interacting Mixed Model (IMM) tracking algorithms. The algorithms were validated to perform well at a variety of measurement clutter densities by using a Monte Carlo simulation environment based on the C++ language. Compared to single-dynamics-model MHT trackers running against a maneuvering target, the Williams-filter-based, multiple-model algorithms exhibited superior tracking performance

Efficient characterization of labeling uncertainty in closely-spaced targets tracking

In this paper we propose a novel solution to the labeled multi-target tracking problem. The method presented is specially effective in scenarios where the targets have once moved in close proximity. When this is the case, disregarding the labeling uncertainty present in a solution (after the targets split) may lead to a wrong decision by the end user. We take a closer look at the main cause of the labeling problem. By modeling the possible crosses between the targets, we define some relevant labeled point estimates. We extend the concept of crossing objects, which is obvious in one dimension, to scenarios where the objects move in multiple dimensions. Moreover, we provide a measure of uncertainty associated to the proposed solution to tackle the labeling problem. We develop a novel, scalable and modular framework in line with it. The proposed method is applied and analyzed on the basis of one-dimensional objects and two-dimensional objects simulation experiments

Association-free Tracking of Two Closely Spaced Targets

This paper introduces a new concept for tracking closely spaced targets in Cartesian space based on position measurements corrupted with additive Gaussian noise. The basic idea is to select a special state representation that eliminates the target identity and avoids the explicit evaluation of association probabilities. One major advantage of this approach is that the resulting likelihood function for this special problem is unimodal. Hence, it is especially suitable for closely spaced targets. The resulting estimation problem can be tackled with a standard nonlinear estimator. In this work, we focus on two targets in two-dimensional Cartesian space. The Cartesian coordinates of the targets are represented by means of extreme values, i.e., minima and maxima. Simulation results demonstrate the feasibility of the new approach

National Aeronautics and Space Administration (NASA)/American Society for Engineering Education (ASEE) Summer Faculty Fellowship Program, 1989, volume 1

The 1989 Johnson Space Center (JSC) National Aeronautics and Space Administration (NASA)/American Society for Engineering Education (ASEE) Summer Faculty Fellowship Program was conducted by Texas A and M University and JSC. The 10-week program was operated under the auspices of the ASEE. The program at JSC, as well as the programs at other NASA Centers, was funded by the Office of University Affairs, NASA Headquarters, Washington, D.C. The objectives of the program, which began nationally in 1964 and at JSC in 1965, are: (1) to further the professional knowledge of qualified engineering and science faculty members; (2) to stimulate an exchange of ideas between participants and NASA; (3) to enrich and refresh the research and teaching activities of participants' institutions; and (4) to contribute to the research objective of the NASA Centers