Upper Body Pose Estimation from Stereo and Hand-face Tracking

Abstract In applications such as immersive telepresence we want to extract high quality 3D models of collaborators in real time from multiview image sequences. One way to improve the quality of stereo or visual hull based models is to estimate the kinematic pose of the user first and then constrain 3D reconstruction accordingly. To serve as a preprocessing step such pose extraction must be very fast, precluding the usual generate and test techniques. We examine a method based on psychophysical evidence that known relative hand position can be used to directly compute the pose of the arm. First we explore a number of possible models for this relationship using motion capture data. We then examine how reconstruction of face and hand position as well as a patch on the torso, allow us to exploit these simple direct calculations to estimate the pose of a user in a desktop collaboration environment

Visual Perception for Manipulation and Imitation in Humanoid Robots

This thesis deals with visual perception for manipulation and imitation in humanoid robots. In particular, real-time applicable methods for object recognition and pose estimation as well as for markerless human motion capture have been developed. As only sensor a small baseline stereo camera system (approx. human eye distance) was used. An extensive experimental evaluation has been performed on simulated as well as real image data from real-world scenarios using the humanoid robot ARMAR-III