Particle filter-based camera tracker fusing marker- and feature point-based cues

This paper presents a video-based camera tracker that combines marker-based and feature point-based cues in a particle filter framework. The framework relies on their complementary performances. Marker-based trackers can robustly recover camera position and orientation when a reference (marker) is available, but fail once the reference becomes unavailable. On the other hand, filter-based camera trackers using feature point cues can still provide predicted estimates given the previous state. However, these tend to drift and usually fail to recover when the reference reappears. Therefore, we propose a fusion where the estimate of the filter is updated from the individual measurements of each cue. More precisely, the marker-based cue is selected when the reference is available whereas the feature point-based cue is selected otherwise. Evaluations on real cases show that the fusion of these two approaches outperforms the individual tracking results

Probabilistic three-dimensional object tracking based on adaptive depth segmentation

Object tracking is one of the fundamental topics of computer vision with diverse applications. The arising challenges in tracking, i.e., cluttered scenes, occlusion, complex motion, and illumination variations have motivated utilization of depth information from 3D sensors. However, current 3D trackers are not applicable to unconstrained environments without a priori knowledge. As an important object detection module in tracking, segmentation subdivides an image into its constituent regions. Nevertheless, the existing range segmentation methods in literature are difficult to implement in real-time due to their slow performance. In this thesis, a 3D object tracking method based on adaptive depth segmentation and particle filtering is presented. In this approach, the segmentation method as the bottom-up process is combined with the particle filter as the top-down process to achieve efficient tracking results under challenging circumstances. The experimental results demonstrate the efficiency, as well as robustness of the tracking algorithm utilizing real-world range information

Particle Filters for Colour-Based Face Tracking Under Varying Illumination

Automatic human face tracking is the basis of robotic and active vision systems used for facial feature analysis, automatic surveillance, video conferencing, intelligent transportation, human-computer interaction and many other applications. Superior human face tracking will allow future safety surveillance systems which monitor drowsy drivers, or patients and elderly people at the risk of seizure or sudden falls and will perform with lower risk of failure in unexpected situations. This area has actively been researched in the current literature in an attempt to make automatic face trackers more stable in challenging real-world environments. To detect faces in video sequences, features like colour, texture, intensity, shape or motion is used. Among these feature colour has been the most popular, because of its insensitivity to orientation and size changes and fast process-ability. The challenge of colour-based face trackers, however, has been dealing with the instability of trackers in case of colour changes due to the drastic variation in environmental illumination. Probabilistic tracking and the employment of particle filters as powerful Bayesian stochastic estimators, on the other hand, is increasing in the visual tracking field thanks to their ability to handle multi-modal distributions in cluttered scenes. Traditional particle filters utilize transition prior as importance sampling function, but this can result in poor posterior sampling. The objective of this research is to investigate and propose stable face tracker capable of dealing with challenges like rapid and random motion of head, scale changes when people are moving closer or further from the camera, motion of multiple people with close skin tones in the vicinity of the model person, presence of clutter and occlusion of face. The main focus has been on investigating an efficient method to address the sensitivity of the colour-based trackers in case of gradual or drastic illumination variations. The particle filter is used to overcome the instability of face trackers due to nonlinear and random head motions. To increase the traditional particle filter\u27s sampling efficiency an improved version of the particle filter is introduced that considers the latest measurements. This improved particle filter employs a new colour-based bottom-up approach that leads particles to generate an effective proposal distribution. The colour-based bottom-up approach is a classification technique for fast skin colour segmentation. This method is independent to distribution shape and does not require excessive memory storage or exhaustive prior training. Finally, to address the adaptability of the colour-based face tracker to illumination changes, an original likelihood model is proposed based of spatial rank information that considers both the illumination invariant colour ordering of a face\u27s pixels in an image or video frame and the spatial interaction between them. The original contribution of this work lies in the unique mixture of existing and proposed components to improve colour-base recognition and tracking of faces in complex scenes, especially where drastic illumination changes occur. Experimental results of the final version of the proposed face tracker, which combines the methods developed, are provided in the last chapter of this manuscript

Latency and Distortion compensation in Augmented Environments using Electromagnetic trackers

Augmented reality (AR) systems are often used to superimpose virtual objects or information on a scene to improve situational awareness. Delays in the display system or inaccurate registration of objects destroy the sense of immersion a user experiences when using AR systems. AC electromagnetic trackers are ideally for these applications when combined with head orientation prediction to compensate for display system delays. Unfortunately, these trackers do not perform well in environments that contain conductive or ferrous materials due to magnetic field distortion without expensive calibration techniques. In our work we focus on both the prediction and distortion compensation aspects of this application, developing a “small footprint” predictive filter for display lag compensation and a simplified calibration system for AC magnetic trackers. In the first phase of our study we presented a novel method of tracking angular head velocity from quaternion orientation using an Extended Kalman Filter in both single model (DQEKF) and multiple model (MMDQ) implementations. In the second phase of our work we have developed a new method of mapping the magnetic field generated by the tracker without high precision measurement equipment. This method uses simple fixtures with multiple sensors in a rigid geometry to collect magnetic field data in the tracking volume. We have developed a new algorithm to process the collected data and generate a map of the magnetic field distortion that can be used to compensation distorted measurement data

Wireless indoor localisation within the 5G internet of radio light

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonNumerous applications can be enhanced by accurate and efficient indoor localisation using wireless sensor networks, however trade-offs often exist between these two parameters. In this thesis, realworld and simulation data is used to examine the hybrid millimeter wave and Visible Light Communications (VLC) architecture of the 5G Internet of Radio Light (IoRL) Horizon 2020 project. Consequently, relevant localisation challenges within Visible Light Positioning (VLP) and asynchronous sampling networks are identified, and more accurate and efficient solutions are developed. Currently, VLP relies strongly on the assumed Lambertian properties of light sources. However, in practice, not all lights are Lambertian. To support the widespread deployment of VLC technology in numerous environments, measurements from non-Lambertian sources are analysed to provide new insights into the limitations of existing VLP techniques. Subsequently, a novel VLP calibration technique is proposed, and results indicate a 59% accuracy improvement against existing methods. This solution enables high accuracy centimetre level VLP to be achieved with non- Lambertian sources. Asynchronous sampling of range-based measurements is known to impact localisation performance negatively. Various Asynchronous Sampling Localisation Techniques (ASLT) exist to mitigate these effects. While effective at improving positioning performance, the exact suitability of such solutions is not evident due to their additional processes, subsequent complexity, and increased costs. As such, extensive simulations are conducted to study the effectiveness of ASLT under variable sampling latencies, sensor measurement noise, and target trajectories. Findings highlight the computational demand of existing ASLT and motivate the development of a novel solution. The proposed Kalman Extrapolated Least Squares (KELS) method achieves optimal localisation performance with a significant energy reduction of over 50% when compared to current leading ASLT. The work in this thesis demonstrates both the capability for high performance VLP from non- Lambertian sources as well as the potential for energy efficient localisation for sequentially sampled range measurements.Horizon 202

Advances in top-down and bottom-up approaches to video-based camera tracking

Video-based camera tracking consists in trailing the three dimensional pose followed by a mobile camera using video as sole input. In order to estimate the pose of a camera with respect to a real scene, one or more three dimensional references are needed. Examples of such references are landmarks with known geometric shape, or objects for which a model is generated beforehand. By comparing what is seen by a camera with what is geometrically known from reality, it is possible to recover the pose of the camera that is sensing these references. In this thesis, we investigate the problem of camera tracking at two levels. Firstly, we work at the low level of feature point recognition. Feature points are used as references for tracking and we propose a method to robustly recognise them. More specifically, we introduce a rotation-discriminative region descriptor and an efficient rotation-discriminative method to match feature point descriptors. The descriptor is based on orientation gradient histograms and template intensity information. Secondly, we have worked at the higher level of camera tracking and propose a fusion of top-down (TDA) and bottom-up approaches (BUA). We combine marker-based tracking using a BUA and feature points recognised from a TDA into a particle filter. Feature points are recognised with the method described before. We take advantage of the identification of the rotation of points for tracking purposes. The goal of the fusion is to take advantage of their compensated strengths. In particular, we are interested in covering the main capabilities that a camera tracker should provide. These capabilities are automatic initialisation, automatic recovery after loss of track, and tracking beyond references known a priori. Experiments have been performed at the two levels of investigation. Firstly, tests have been conducted to evaluate the performance of the recognition method proposed. The assessment consists in a set of patches extracted from eight textured images. The images are rotated and matching is done for each patch. The results show that the method is capable of matching accurately despite the rotations. A comparison with similar techniques in the state of the art depicts the equal or even higher precision of our method with much lower computational cost. Secondly, experimental assessment of the tracking system is also conducted. The evaluation consists in four sequences with specific problematic situations namely, occlusions of the marker, illumination changes, and erratic and/or fast motion. Results show that the fusion tracker solves characteristic failure modes of the two combined approaches. A comparison with similar trackers shows competitive accuracy. In addition, the three capabilities stated earlier are fulfilled in our tracker, whereas the state of the art reveals that no other published tracker covers these three capabilities simultaneously. The camera tracking system has a potential application in the robotics domain. It has been successfully used as a man-machine interface and applied in Augmented Reality environments. In particular, the system has been used by students of the University of art and design Lausanne (ECAL) with the purpose of conceiving new interaction concepts. Moreover, in collaboration with ECAL and fabric | ch (studio for architecture & research), we have jointly developed the Augmented interactive Reality Toolkit (AiRToolkit). The system has also proved to be reliable in public events and is the basis of a game-oriented demonstrator installed in the Swiss National Museum of Audiovisual and Multimedia (Audiorama) in Montreux

Robust execution for stochastic hybrid systems

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 2007.Includes bibliographical references (p. 177-180).Unmanned systems, such as Autonomous Underwater Vehicles (AUVs), planetary rovers and space probes, have enormous potential in areas such as reconnaissance and space exploration. However the effectiveness and robustness of these systems is currently restricted by a lack of autonomy. Previous work introduced the concept of a model-based executive, which increases the level of autonomy, elevating the level at which systems are commanded. This simplifies the operator's task and leaves degrees of freedom in the plan that allow the executive to optimize resources and ensure robustness to uncertainty. Uncertainty arises due to uncertain state estimation, disturbances, model uncertainty and component failures. This thesis develops a model-based executive that reasons explicitly from a stochastic hybrid discrete-continuous system model to find the optimal course of action, while ensuring the required level of robustness to uncertainty is achieved. Our first contribution is a novel 'Particle Control' approach for robust execution of state plans with stochastic hybrid systems. We introduce the notion of chance-constrained state plan execution; this means that the executive ensures tasks in the state plan have at least a specified minimum probability of success. The minimum probabilities are specified by the operator, enabling conservatism to be traded against performance. In order to make optimal chance-constrained execution tractable, the Particle Control approach approximates the system's state distribution using samples or 'particles' and optimizes the evolution of these particles to achieve chance-constrained state plan execution. In this manner particle control solves a tractable deterministic approximation to the original stochastic problem; furthermore the approximation becomes exact as the number of particles approaches infinity.(cont.) For an important class of hybrid discrete-continuous system known as Jump Markov Linear Systems, the resulting deterministic optimization can be posed as a Mixed Integer Linear Program and solved to global optimality using efficient commercially-available solvers. Our second contribution is 'active' hybrid estimation subject to state plan constraints. Exact hybrid state estimation in stochastic hybrid systems is, in general, intractable. Tractable approximate hybrid estimation methods can lose track of the true hybrid state. In this thesis we develop an active hybrid estimation capability, which probes the system in order to reduce uncertainty in the hybrid state. This approach generates control sequences to minimize the probability of approximate hybrid estimation losing the true mode sequence, while ensuring that the state plan is satisfied subject to chance constraints. In order to make this problem tractable, we develop an analytic upper bound on the probability of losing the true mode sequence, and use a convex constraint tightening approach to approximate the chance constraints in the problem. Our final contribution is a novel hybrid model-learning approach. Specifying accurate hybrid system models is essential for accurate estimation and control, but is also extremely challenging. The hybrid executive must therefore determine hybrid system models from observed data. In this thesis we present an approximate Expectation-Maximization method for hybrid model learning; this method extends prior approaches to deal with mode transitions that depend on the continuous state.by Lars James Christopher Blackmore.Ph.D