Bleyer M.

Cagniart C.

Dou M.

Gall J.

Hernández C.

Kolb A.

Li H.

Liao M.

Mitra N. J.

Oikonomidis I.

Pradeep V.

Sorkine O.

Taylor J.

Waschbüsch M.

Weise T.


            We present a combined hardware and software solution for markerless reconstruction of non-rigidly deforming physical objects with arbitrary shape in
            real-time
            . Our system uses a single self-contained stereo camera unit built from off-the-shelf components and consumer graphics hardware to generate spatio-temporally coherent 3D models at 30 Hz. A new stereo matching algorithm estimates real-time RGB-D data. We start by scanning a smooth template model of the subject as they move rigidly. This geometric surface prior avoids strong scene assumptions, such as a kinematic human skeleton or a parametric shape model. Next, a novel GPU pipeline performs non-rigid registration of live RGB-D data to the smooth template using an extended non-linear as-rigid-as-possible (ARAP) framework. High-frequency details are fused onto the final mesh using a linear deformation model. The system is an order of magnitude faster than state-of-the-art methods, while matching the quality and robustness of many offline algorithms. We show precise real-time reconstructions of diverse scenes, including: large deformations of users' heads, hands, and upper bodies; fine-scale wrinkles and folds of skin and clothing; and non-rigid interactions performed by users on flexible objects such as toys. We demonstrate how acquired models can be used for many interactive scenarios, including re-texturing, online performance capture and preview, and real-time shape and motion re-targeting.
          </jats:p

Michael Zollhöfer

Matthias Nießner

Shahram Izadi

Christoph Rehmann

Christopher Zach

Matthew Fisher

Chenglei Wu

Andrew Fitzgibbon

Charles Loop

Christian Theobalt

Marc Stamminger

Crossref

ACM Transactions on Graphics

Real-time non-rigid reconstruction using an RGB-D camera

http://www.cs.jhu.edu/%7Emisha/ReadingSeminar/Papers/Zollhoefer14.pdf

Real-time non-rigid reconstruction using an RGB-D camera

Abstract

Similar works

Full text

Available Versions

Crossref