A Multicamera System for Gesture Tracking With Three Dimensional Hand Pose Estimation

The goal of any visual tracking system is to successfully detect then follow an object of interest through a sequence of images. The difficulty of tracking an object depends on the dynamics, the motion and the characteristics of the object as well as on the environ ment. For example, tracking an articulated, self-occluding object such as a signing hand has proven to be a very difficult problem. The focus of this work is on tracking and pose estimation with applications to hand gesture interpretation. An approach that attempts to integrate the simplicity of a region tracker with single hand 3D pose estimation methods is presented. Additionally, this work delves into the pose estimation problem. This is ac complished by both analyzing hand templates composed of their morphological skeleton, and addressing the skeleton\u27s inherent instability. Ligature points along the skeleton are flagged in order to determine their effect on skeletal instabilities. Tested on real data, the analysis finds the flagging of ligature points to proportionally increase the match strength of high similarity image-template pairs by about 6%. The effectiveness of this approach is further demonstrated in a real-time multicamera hand tracking system that tracks hand gestures through three-dimensional space as well as estimate the three-dimensional pose of the hand

Stereo Vision Tracking of Multiple Objects in Complex Indoor Environments

This paper presents a novel system capable of solving the problem of tracking multiple targets in a crowded, complex and dynamic indoor environment, like those typical of mobile robot applications. The proposed solution is based on a stereo vision set in the acquisition step and a probabilistic algorithm in the obstacles position estimation process. The system obtains 3D position and speed information related to each object in the robot’s environment; then it achieves a classification between building elements (ceiling, walls, columns and so on) and the rest of items in robot surroundings. All objects in robot surroundings, both dynamic and static, are considered to be obstacles but the structure of the environment itself. A combination of a Bayesian algorithm and a deterministic clustering process is used in order to obtain a multimodal representation of speed and position of detected obstacles. Performance of the final system has been tested against state of the art proposals; test results validate the authors’ proposal. The designed algorithms and procedures provide a solution to those applications where similar multimodal data structures are found

Real-time visual tracking using image processing and filtering methods

The main goal of this thesis is to develop real-time computer vision algorithms in order to detect and to track targets in uncertain complex environments purely based on a visual sensor. Two major subjects addressed by this work are: 1. The development of fast and robust image segmentation algorithms that are able to search and automatically detect targets in a given image. 2. The development of sound filtering algorithms to reduce the effects of noise in signals from the image processing. The main constraint of this research is that the algorithms should work in real-time with limited computing power on an onboard computer in an aircraft. In particular, we focus on contour tracking which tracks the outline of the target represented by contours in the image plane. This thesis is concerned with three specific categories, namely image segmentation, shape modeling, and signal filtering. We have designed image segmentation algorithms based on geometric active contours implemented via level set methods. Geometric active contours are deformable contours that automatically track the outlines of objects in images. In this approach, the contour in the image plane is represented as the zero-level set of a higher dimensional function. (One example of the higher dimensional function is a three-dimensional surface for a two-dimensional contour.) This approach handles the topological changes (e.g., merging, splitting) of the contour naturally. Although geometric active contours prevail in many fields of computer vision, they suffer from the high computational costs associated with level set methods. Therefore, simplified versions of level set methods such as fast marching methods are often used in problems of real-time visual tracking. This thesis presents the development of a fast and robust segmentation algorithm based on up-to-date extensions of level set methods and geometric active contours, namely a fast implementation of Chan-Vese's (active contour) model (FICVM). The shape prior is a useful cue in the recognition of the true target. For the contour tracker, the outline of the target can be easily disrupted by noise. In geometric active contours, to cope with deviations from the true outline of the target, a higher dimensional function is constructed based on the shape prior, and the contour tracks the outline of an object by considering the difference between the higher dimensional functions obtained from the shape prior and from a measurement in a given image. The higher dimensional function is often a distance map which requires high computational costs for construction. This thesis focuses on the extraction of shape information from only the zero-level set of the higher dimensional function. This strategy compensates for inaccuracies in the calculation of the shape difference that occur when a simplified higher dimensional function is used. This is named as contour-based shape modeling. Filtering is an essential element in tracking problems because of the presence of noise in system models and measurements. The well-known Kalman filter provides an exact solution only for problems which have linear models and Gaussian distributions (linear/Gaussian problems). For nonlinear/non-Gaussian problems, particle filters have received much attention in recent years. Particle filtering is useful in the approximation of complicated posterior probability distribution functions. However, the computational burden of particle filtering prevents it from performing at full capacity in real-time applications. This thesis concentrates on improving the processing time of particle filtering for real-time applications. In principle, we follow the particle filter in the geometric active contour framework. This thesis proposes an advanced blob tracking scheme in which a blob contains shape prior information of the target. This scheme simplifies the sampling process and quickly suggests the samples which have a high probability of being the target. Only for these samples is the contour tracking algorithm applied to obtain a more detailed state estimate. Curve evolution in the contour tracking is realized by the FICVM. The dissimilarity measure is calculated by the contour based shape modeling method and the shape prior is updated when it satisfies certain conditions. The new particle filter is applied to the problems of low contrast and severe daylight conditions, to cluttered environments, and to the appearing/disappearing target tracking. We have also demonstrated the utility of the filtering algorithm for multiple target tracking in the presence of occlusions. This thesis presents several test results from simulations and flight tests. In these tests, the proposed algorithms demonstrated promising results in varied situations of tracking.Ph.D.Committee Chair: Eric N. Johnson; Committee Co-Chair: Allen R. Tannenbaum; Committee Member: Anthony J. Calise; Committee Member: Eric Feron; Committee Member: Patricio A. Vel

Exploring Motion Signatures for Vision-Based Tracking, Recognition and Navigation

As cameras become more and more popular in intelligent systems, algorithms and systems for understanding video data become more and more important. There is a broad range of applications, including object detection, tracking, scene understanding, and robot navigation. Besides the stationary information, video data contains rich motion information of the environment. Biological visual systems, like human and animal eyes, are very sensitive to the motion information. This inspires active research on vision-based motion analysis in recent years. The main focus of motion analysis has been on low level motion representations of pixels and image regions. However, the motion signatures can benefit a broader range of applications if further in-depth analysis techniques are developed. In this dissertation, we mainly discuss how to exploit motion signatures to solve problems in two applications: object recognition and robot navigation. First, we use bird species recognition as the application to explore motion signatures for object recognition. We begin with study of the periodic wingbeat motion of flying birds. To analyze the wing motion of a flying bird, we establish kinematics models for bird wings, and obtain wingbeat periodicity in image frames after the perspective projection. Time series of salient extremities on bird images are extracted, and the wingbeat frequency is acquired for species classification. Physical experiments show that the frequency based recognition method is robust to segmentation errors and measurement lost up to 30%. In addition to the wing motion, the body motion of the bird is also analyzed to extract the flying velocity in 3D space. An interacting multi-model approach is then designed to capture the combined object motion patterns and different environment conditions. The proposed systems and algorithms are tested in physical experiments, and the results show a false positive rate of around 20% with a low false negative rate close to zero. Second, we explore motion signatures for vision-based vehicle navigation. We discover that motion vectors (MVs) encoded in Moving Picture Experts Group (MPEG) videos provide rich information of the motion in the environment, which can be used to reconstruct the vehicle ego-motion and the structure of the scene. However, MVs suffer from high noise level. To handle the challenge, an error propagation model for MVs is first proposed. Several steps, including MV merging, plane-at-infinity elimination, and planar region extraction, are designed to further reduce noises. The extracted planes are used as landmarks in an extended Kalman filter (EKF) for simultaneous localization and mapping. Results show that the algorithm performs localization and plane mapping with a relative trajectory error below 5:1%. Exploiting the fact that MVs encodes both environment information and moving obstacles, we further propose to track moving objects at the same time of localization and mapping. This enables the two critical navigation functionalities, localization and obstacle avoidance, to be performed in a single framework. MVs are labeled as stationary or moving according to their consistency to geometric constraints. Therefore, the extracted planes are separated into moving objects and the stationary scene. Multiple EKFs are used to track the static scene and the moving objects simultaneously. In physical experiments, we show a detection rate of moving objects at 96:6% and a mean absolute localization error below 3:5 meters

Creating illusion in computer aided performance

This thesis studies the creation of illusion in computer aided performance. Illusion is created here by using deceptions, and a design framework is presented which suggests several different deception strategies which may be useful. The framework has been developed in an iterative process in tandem with the development of 3 real world performances which were used to explore deception strategies. The first case study presents a system for augmenting juggling performance. The techniques that were developed to control this system demonstrate how deception may become useful even when the core of the performance is not deceptive in any way. This is followed by a magic performance called the Cup Game, which was designed to explicitly test the strategies of deception described in the framework. The final case study is an interactive art installation which presents the illusion of a pet rock that lives in a cage. This demonstrates the usefulness of suspension of disbelief in the creation of illusions. It also demonstrates interesting social effects that are used to strengthen this suspension of disbelief. The idea of creating the impression of a false situation is inspired particularly by previous HCI work on public interaction. This work demonstrated the usefulness of hiding interface use or computer outputs from some people in a situation. The creation of deliberately ambiguous computer interfaces, which allow for a wider variety of interpretations to be made by the user has also been described. The work here goes beyond these techniques to use technology to actively create false impressions. The techniques used in this process are guided by the work of magic performers, and by psychological studies of how magic performance works. As well as artistic performance, it is envisaged that this work may prove applicable to more traditional situations. In addition to the framework itself, the development of the case studies has created several useful algorithms which have wider applications. The case studies are also useful guides for those creating performance systems, or other systems where deceptive techniques may be useful

Deleuze, Freud and the Three Syntheses

The aim of this paper is to provide a close reading of Deleuze's complex account of Freud's Beyond the Pleasure Principle in Difference and Repetition. The first part provides a reading of Beyond the Pleasure Principle itself, showing why Freud feels the need to develop a transcendental account of repetition. In the second, I show the limitations of Freud's account, drawing on the work of Weismann to argue that Freud's transcendental model mischaracterises repetition. In the third part, I show how Freud's account of the death drive is shadowed by Deleuze's own non-representational transcendental account

Towards a schizogenealogy of heretical materialism : between Bruno and Spinoza, Nietzsche, Deleuze and other philosophical recluses

The central problematic of this thesis is the formation of a philosophy of creative matter, a philosophical materialism, deriving from the work of Gilles Deleuze Fdlix and Guattari, and based substantially upon an examination of the consequences of their engagement with the philosophical tradition. I have supplemented the writers used by Deleuze and Guattari with the resources of Giordano Bruno's philosophy, as well as numerous examples and arguments from the natural sciences. Bruno is particularly important here, in that in his work and life, materialism is most tightly bound up with monism. Philosophical materialist monism can be crystallised as a sustained meditation upon one problem: that of the overcoming of dualism; and in this sense to speak of materialism is to speak of the problem of hylomorphism. The hylomorphic model, formalised by Aristotle, and operative in both philosophy and science, implies both a transcendent form that organises matter, and a dead matter, passively moulded by the imposition of that form. These ontological and epistemological assumptions have clear political and theological ramifications, contributing to an abstract diagram of State power. The critique of this model calls for a philosophy of active, self-organising matter- a necessarily heretical, materialist thought, constitutionally opposed to all transcendent powers. I In this chapter I produce a performative diagram of DeleuzeGuattari's understanding of the heterogenetic nature of the concept by examining those of drive, assemblage, multiplicity. The case used here is the linked complex of problems associated with death and entropy. These issues are posed throughout as means of indicating Deleuze and Guattari's challenge to dominant modes of philosophising. II Here I offer an elaboration of Deleuze and Guattari's relationship with cybernetics, through an outline of the work of Gilbert Simondon. The principal concepts developed here, are individuation and becoming. This is followed by extensive critiques of hylomorphism and autopoiesis. The categories of minor or nomad, and major or State, sciences, are introduced along with the related concepts of following and reproducing. III This chapter explores the oppositions between consistency and organisation; immanence and transcendence. Here I read two of Deleuze and Guattari's key concepts- intensity and incorporeal transformation- in terms of Spinoza and Schelling respectively. Symbiosis and morphogenesis are examined as examples of the minor sciences introduced in the previous chapter. The minor then poses the questions of invention and pragmatics in philosophy. IV This chapter is devoted to a critique of Manuel De Landa's reading of Deleuze and Guattari that aims to demonstrate, against his claims, the centrality of Marx to their philosophy. The chapter also elaborates upon the concepts of Geophilosophy, the machinic phylum, and machinic surplus value. V This chapter offers a set of elaborations upon the nature of the materialism produced by bringing the thought of Giordano Bruno into contact with that of Deleuze, thereby transforming both. Inverted vitalism is posed as a key marker of Deleuze's genealogy. I show the identity of metaphysics and politics, and its role in an account of materialist heresy. VI The final chapter consists of a critique of Kant's claim to being `Copernican', and Copernicus' claim to being revolutionary. It demonstrates the extent of Bruno's cosmological revolution. I use Nietzsche's `perfect nihilist' to further the ideas of invention and heresy advanced earlier, to end with a demonstration of philosophy's ever present becomings hybrid, as opposed to dominant ideas of its being in a permanent state of mourning