3D Scene Reconstruction by Stereo Methods for Analysis and Visualization of Sports Scenes

The 3D reconstruction of image and video scenes by stereo analysis is an important topic in computer vision research. In this talk, we first present some principles of stereo algorithms and recent developments. We then demonstrate two applications of stereo reconstruction for the analysis and visualization of human movement: (a) We employ depth maps derived from sport scenes for novel view synthesis, and (b) we show how stereo processing can be used for expressive visualization of human motion in a comic-like style

The Global Patch Collider

Abstract This paper proposes a novel extremely efficient, fullyparallelizable, task-specific algorithm for the computation of global point-wise correspondences in images and videos. Our algorithm, the Global Patch Collider, is based on detecting unique collisions between image points using a collection of learned tree structures that act as conditional hash functions. In contrast to conventional approaches that rely on pairwise distance computation, our algorithm isolates distinctive pixel pairs that hit the same leaf during traversal through multiple learned tree structures. The split functions stored at the intermediate nodes of the trees are trained to ensure that only visually similar patches or their geometric or photometric transformed versions fall into the same leaf node. The matching process involves passing all pixel positions in the images under analysis through the tree structures. We then compute matches by isolating points that uniquely collide with each other ie. fell in the same empty leaf in multiple trees. Our algorithm is linear in the number of pixels but can be made constant time on a parallel computation architecture as the tree traversal for individual image points is decoupled. We demonstrate the efficacy of our method by using it to perform optical flow matching and stereo matching on some challenging benchmarks. Experimental results show that not only is our method extremely computationally efficient, but it is also able to match or outperform state of the art methods that are much more complex

Fusion4D: Real-time Performance Capture of Challenging Scenes

We contribute a new pipeline for live multi-view performance capture, generating temporally coherent high-quality reconstructions in real-time. Our algorithm supports both incremental reconstruction, improving the surface estimation over time, as well as parameterizing the nonrigid scene motion. Our approach is highly robust to both large frame-to-frame motion and topology changes, allowing us to reconstruct extremely challenging scenes. We demonstrate advantages over related real-time techniques that either deform an online generated template or continually fuse depth data nonrigidly into a single reference model. Finally, we show geometric reconstruction results on par with offline methods which require orders of magnitude more processing time and many more RGBD cameras

Region-Based Optical Flow Estimation with Treatment of Occlusions

The estimation of optical flow plays a key-role in several computer vision problems, including motion detection and segmentation, frame interpolation, three-dimensional scene reconstruction, robot navigation, video shot detection, mosaicking and video compression. In this work we propose a new algorithm for computing a dense optical flow field between two or more images of a video sequence, which tackles the inherent problems of conventional optical flow algorithms. These algorithms usually show a bad performance in regions of low texture as well as near motion boundaries. We try to overcome these problems by segmenting the reference frame into regions of homogeneous color. The color segmentation incorporates the assumption that the motion inside regions of homogeneous color varies smoothly and motion discontinuities coincide with the borders of those regions. The affine motion model is used to describe the motion inside a segment. To initializ

Improving Color Modeling for Alpha Matting

This paper addresses the problem of extracting an alpha matte from a single photograph given a user-defined trimap. A crucial part of this task is the color modeling step where for each pixel the optimal alpha value, together with its confidence, is estimated individually. This forms the data term of the objective function. It comprises of three steps: (i) Collecting a candidate set of potential fore- and background colors; (ii) Selecting high confidence samples from the candidate set; (iii) Estimating a sparsity prior to remove blurry artifacts. We introduce novel ideas for each of these steps and show that our approach considerably improves over state-of-the-art techniques by evaluating it on a large database of 54 images with known high-quality ground truth.