Space-variant motion detection for active visual target tracking

A biologically inspired approach to active visual target tracking is presented. The approach makes use of three strategies found in biological systems: space-variant sensing, a spatio-temporal frequency based model of motion detection and the alignment of sensory-motor maps. Space-variant imaging is used to create a 1-D array of elementary motion detectors (EMDs) that are tuned in such a way as to make it possible to detect motion over a wide range of velocities while still being able to detect motion precisely. The array is incorporated into an active visual tracking system. A method of analysis and design for such a tracking system is proposed. It makes use of a sensory-motor map which consists of a phase-plane plot of the continuous-time dynamics of the tracking system overlaid onto a map of the detection capabilities of the array of EMDs. This sensory-motor map is used to design a simple 1-D tracking system and several simulations show how the method can be used to control tracking performance using such metrics as overshoot and settling time. A complete 1-D active vision system is implemented and a set of simple target tracking experiments are performed to demonstrate the effectiveness of the approach

A space-variant architecture for active visual target tracking

An active visual target tracking system is an automatic feedback control system that can track a moving target by controlling the movement of a camera or sensor array. This kind of system is often used in applications such as automatic surveillance and human-computer interaction. The design of an effective target tracking system is challenging because the system should be able to precisely detect the fine movements of a target while still being able to detect a large range of target velocities. Achieving this in a computationally efficient manner is difficult with a conventional system architecture. This thesis presents an architecture for an active visual target tracking system based on the idea of space-variant motion detection. In general, space-variant imaging involves the use of a non-uniform distribution of sensing elements across a sensor array, similar to how the photoreceptors in the human eye are not evenly distributed. In the proposed architecture, space-variant imaging is used to design an array of elementary motion detectors (EMDs). The EMDs are tuned in such a way as to make it possible to detect motion both precisely and over a wide range of velocities in a computationally efficient manner. The increased ranges are achieved without additional computational costs beyond the basic mechanism of motion detection. The technique is general in that it can be used with different motion detection mechanisms and the overall space-variant structure can be varied to suit a particular application. The design of a tracking system based on a space-variant motion detection array is a difficult task. This thesis presents a method of analysis and design for such a tracking system. The method of analysis consists of superimposing a phase-plane plot of the continuous-time dynamics of the tracking system onto a map of the detection capabilities of the array of EMDs. With the help of this 'sensory-motor' plot, a simple optimization algorithm is used to design a tracking system to meet particular objectives for settling time, steady-state error and overshoot. Several simulations demonstrate the effectiveness of the method. A complete active vision system is implemented and a set of target tracking experiments are performed. Experimental results support the effectiveness of the approac

Wide-Angle Foveation for All-Purpose Use

This paper proposes a model of a wide-angle space-variant image that provides a guide for designing a fovea sensor. First, an advanced wide-angle foveated (AdWAF) model is formulated, taking all-purpose use into account. This proposed model uses both Cartesian (linear) coordinates and logarithmic coordinates in both planar projection and spherical projection. Thus, this model divides its wide-angle field of view into four areas, such that it can represent an image by various types of lenses, flexibly. The first simulation compares with other lens models, in terms of image height and resolution. The result shows that the AdWAF model can reduce image data by 13.5%, compared to a log-polar lens model, both having the same resolution in the central field of view. The AdWAF image is remapped from an actual input image by the prototype fovea lens, a wide-angle foveated (WAF) lens, using the proposed model. The second simulation compares with other foveation models used for the existing log-polar chip and vision system. The third simulation estimates a scale-invariant property by comparing with the existing fovea lens and the log-polar lens. The AdWAF model gives its planar logarithmic part a complete scale-invariant property, while the fovea lens has 7.6% error at most in its spherical logarithmic part. The fourth simulation computes optical flow in order to examine the unidirectional property when the fovea sensor by the AdWAF model moves, compared to the pinhole camera. The result obtained by using a concept of a virtual cylindrical screen indicates that the proposed model has advantages in terms of computation and application of the optical flow when the fovea sensor moves forward

A Comprehensive Performance Evaluation of Deformable Face Tracking "In-the-Wild"

Recently, technologies such as face detection, facial landmark localisation and face recognition and verification have matured enough to provide effective and efficient solutions for imagery captured under arbitrary conditions (referred to as "in-the-wild"). This is partially attributed to the fact that comprehensive "in-the-wild" benchmarks have been developed for face detection, landmark localisation and recognition/verification. A very important technology that has not been thoroughly evaluated yet is deformable face tracking "in-the-wild". Until now, the performance has mainly been assessed qualitatively by visually assessing the result of a deformable face tracking technology on short videos. In this paper, we perform the first, to the best of our knowledge, thorough evaluation of state-of-the-art deformable face tracking pipelines using the recently introduced 300VW benchmark. We evaluate many different architectures focusing mainly on the task of on-line deformable face tracking. In particular, we compare the following general strategies: (a) generic face detection plus generic facial landmark localisation, (b) generic model free tracking plus generic facial landmark localisation, as well as (c) hybrid approaches using state-of-the-art face detection, model free tracking and facial landmark localisation technologies. Our evaluation reveals future avenues for further research on the topic.Comment: E. Antonakos and P. Snape contributed equally and have joint second authorshi

Beyond standard benchmarks: Parameterizing performance evaluation in visual object tracking

Object-to-camera motion produces a variety of apparent motion patterns that significantly affect performance of short-term visual trackers. Despite being crucial for designing robust trackers, their influence is poorly explored in standard benchmarks due to weakly defined, biased and overlapping attribute annotations. In this paper we propose to go beyond pre-recorded benchmarks with post-hoc annotations by presenting an approach that utilizes omnidirectional videos to generate realistic, consistently annotated, short-term tracking scenarios with exactly parameterized motion patterns. We have created an evaluation system, constructed a fully annotated dataset of omnidirectional videos and the generators for typical motion patterns. We provide an in-depth analysis of major tracking paradigms which is complementary to the standard benchmarks and confirms the expressiveness of our evaluation approach

A Neural Model of How the Brain Computes Heading from Optic Flow in Realistic Scenes

Animals avoid obstacles and approach goals in novel cluttered environments using visual information, notably optic flow, to compute heading, or direction of travel, with respect to objects in the environment. We present a neural model of how heading is computed that describes interactions among neurons in several visual areas of the primate magnocellular pathway, from retina through V1, MT+, and MSTd. The model produces outputs which are qualitatively and quantitatively similar to human heading estimation data in response to complex natural scenes. The model estimates heading to within 1.5° in random dot or photo-realistically rendered scenes and within 3° in video streams from driving in real-world environments. Simulated rotations of less than 1 degree per second do not affect model performance, but faster simulated rotation rates deteriorate performance, as in humans. The model is part of a larger navigational system that identifies and tracks objects while navigating in cluttered environments.National Science Foundation (SBE-0354378, BCS-0235398); Office of Naval Research (N00014-01-1-0624); National-Geospatial Intelligence Agency (NMA201-01-1-2016