PhDReconstructing the 3D geometry of scenes based on monocular image sequences is
a long-standing problem in computer vision. Structure from motion (SfM) aims at a
data-driven approach without requiring a priori models of the scene. When the scene is
rigid, SfM is a well understood problem with solutions widely used in industry. However,
if the scene is non-rigid, monocular reconstruction without additional information
is an ill-posed problem and no satisfactory solution has yet been found.
Current non-rigid SfM (NRSfM) methods typically aim at modelling deformable
motion globally. Additionally, most of these methods focus on cases where deformable
motion is seen as small variations from a mean shape. In turn, these methods fail at
reconstructing highly deformable objects such as a flag waving in the wind. Additionally,
reconstructions typically consist of low detail, sparse point-cloud representation
of objects.
In this thesis we aim at reconstructing highly deformable surfaces by modelling
them locally. In line with a recent trend in NRSfM, we propose a piecewise approach
which reconstructs local overlapping regions independently. These reconstructions are
merged into a global object by imposing 3D consistency of the overlapping regions.
We propose our own local model – the Quadratic Deformation model – and show
how patch division and reconstruction can be formulated in a principled approach by
alternating at minimizing a single geometric cost – the image re-projection error of
the reconstruction. Moreover, we extend our approach to dense NRSfM, where reconstructions
are preformed at the pixel level, improving the detail of state of the art
reconstructions.
Finally we show how our principled approach can be used to perform simultaneous
segmentation and reconstruction of articulated motion, recovering meaningful
segments which provide a coarse 3D skeleton of the object.Fundacao para a Ciencia e a Tecnologia (FCT)
under Doctoral Grant SFRH/BD/70312/2010; European Research Council
under ERC Starting Grant agreement 204871-HUMANI
