Stereo 3D Lip Tracking

A system is presented that tracks in 3D a person's unadorned lips, and outputs the 3D locations of the mouth corners and ten points describing the outer lip contour. This output is suitable for audio visual speech processing, 3D animation, or expression recognition. A stereo head tracker is used to track the subject's head, allowing for robust performance whilst the subject's head is moving and turning with respect to the cameras. The head pose is used in conjunction with the novel adaptable templates to generate a robust estimate of the deforming mouth corner locations. A 3D geometric model is used to generate search paths for key points on the outer lip contour which are subsequently located using adaptable templates and geometric constraints. The system is demonstrated robustly tracking the head pose and 3D mouth shape on a person speaking while moving his head. 1 Introduction Tracking the lips has a broad scope of applications across the field of human-computer interaction (HCI)..

Advancing active vision systems by improved design and control

This paper presents the mechanical hardware and control software of a novel high-performance active vision system. It is the latest in an ongoing research effort to develop real-world vision systems based on cable-drive transmissions. The head presented in this paper is the laboratory’s first fully cable-driven binocular rig, and builds on several successful aspects of previous monocular prototypes. Namely, an increased payload capacity, a more compact transmission, and a design optimised for rigidity. In addition, we have developed a simple and compact controller for real-time tracking applications. It consists of two behavioural subgroups, saccade and smooth pursuit. By using a single trapezoidal profile motion (TPM) algorithm, we show that saccade time and motion smoothness can be optimised

Active Stereo Vision System with Foveated Wide Angle Lenses

A novel active stereo vision system with apair of foveated wide angle lenses is presented. The projection curve of the lens is designed so that it facilitates active vision algorithms for motion analysis, object identification, and precise fixation. Apair of such lenses are mounted on a specially designed active stereo vision platform. It is compact and light so that it can be mounted on a mobile robot or a manipulator. A real time stereotracking system is constructed using the platform, a dual-processor servo controller, and pipelined image processors with a multiprocessor backend.

Cooperation by Observation - The Framework and Basic Task Patterns

A novel framework for multiple robot cooperation called "Cooperation by Observation" is presented. It introduces many interesting issues such as viewpoint constraint and role interchange, as well as novel concepts like "attentional structure". The framework has the potential to realize a high level of task coordination by decentralized autonomous robots allowing minimum explicit communication. Its source of power lies in an advanced capability given to each robot for recognizing other agent's actions by (primarily visual) observation. This provides rich information about the current task situation around each robot which facilitates highly-structured task coordination. The basic visuo-motor routines are described. Concrete examples and experiments using real mobile robots are also presented. 1 Introduction In order to execute complex tasks with multiple mobile robots and achieve superlinear increase of task performance to the number of robots, we need a variety of flexible and highl..