Camera Tracking Using A Dense 3D Model

Abstract In this thesis we present a method for tracking a depth sensor and building a 3D model in real-time. The tracking will be done using the current estimated 3D model. We will show that this approach gives more accurate results and is more robust than the well-known KinectFusion approach. It will also be shown how to colourise the 3D model in real-time. In this work we will study different error metrics such as the projective point-to-point metric and projective point-to-plane metric. Also different norms will be evaluated to find out which gives the best result. We will also show how to represent a 3D model by using a so called signed distance function

A Non-Convex Relaxation for Fixed-Rank Approximation

This paper considers the problem of finding a low rank matrix from observations of linear combinations of its elements. It is well known that if the problem fulfills a restricted isometry property (RIP), convex relaxations using the nuclear norm typically work well and come with theoretical performance guarantees. On the other hand these formulations suffer from a shrinking bias that can severely degrade the solution in the presence of noise. In this theoretical paper we study an alternative non-convex relaxation that in contrast to the nuclear norm does not penalize the leading singular values and thereby avoids this bias. We show that despite its non-convexity the proposed formulation will in many cases have a single local minimizer if a RIP holds. Our numerical tests show that our approach typically converges to a better solution than nuclear norm based alternatives even in cases when the RIP does not hold

Real-Time Camera Tracking and 3D Reconstruction Using Signed Distance Functions

The ability to quickly acquire 3D models is an essential capability needed in many disciplines including robotics, computer vision, geodesy, and architecture. In this paper we present a novel method for real-time camera tracking and 3D reconstruction of static indoor environments using an RGB-D sensor. We show that by representing the geometry with a signed distance function (SDF), the camera pose can be efficiently estimated by directly minimizing the error of the depth images on the SDF. As the SDF contains the distances to the surface for each voxel, the pose optimization can be carried out extremely fast. By iteratively estimating the camera poses and integrating the RGB-D data in the voxel grid, a detailed reconstruction of an indoor environment can be achieved. We present reconstructions of several rooms using a hand-held sensor and from onboard an autonomous quadrocopter. Our extensive evaluation on publicly available benchmark data shows that our approach is more accurate and robust than the iterated closest point algorithm (ICP) used by KinectFusion, and yields often a comparable accuracy at much higher speed to feature-based bundle adjustment methods such as RGB-D SLAM for up to medium-sized scenes

Dense Tracking and Mapping with a Quadrocopter

In this paper, we present an approach for acquiring textured 3D models of room-sized indoor spaces using a quadrocopter. Such room models are for example useful for architects and interior designers as well as for factory planners and construction man- agers. The model is internally represented by a signed distance function (SDF) and the SDF is used to directly track the camera with respect to the model. Our solution enables accurate position control of the quadrocopter, so that it can automatically follow a pre-defined flight pattern. Our system provides live feedback of the acquired 3D model to the user. The final model consisting of a textured 3D triangle mesh can be saved in several standard CAD file formats

Optimization Methods for 3D Reconstruction : Depth Sensors, Distance Functions and Low-Rank Models

This thesis explores methods for estimating 3D models using depth sensors andfinding low-rank approximations of matrices. In the first part we focus on how toestimate the movement of a depth camera and creating a 3D model of the scene.Given an accurate estimation of the camera position, we can produce dense 3Dmodels using the images obtained from the camera. We present algorithms thatare both accurate, robust and in addition, fast enough for online 3D reconstructionin real-time. The frame rate varies between about 5-20 Hz. It is shown inexperiments that these algorithms are viable for several different applications suchas autonomous quadrocopter navigation and object reconstruction.In the second part we study the problem of finding a low-rank approximationof a given matrix. This has several applications in computer vision such as rigidand non-rigid Structure from Motion, denoising, photometric stereo and so on.Two convex relaxations which take both the rank function and a data term intoaccount are introduced and analyzed together with a non-convex relaxation. It isshown that these methods often avoid shrinkage bias and give better results thanthe common heuristic of replacing the rank function with the nuclear norm

Robust Camera Tracking by Combining Color and Depth Measurements

One of the major research areas in computer vision is scene reconstruction from image streams. The advent of RGB-D cameras, such as the Microsoft Kinect, has lead to new possibilities for performing accurate and dense 3D reconstruction. There are already well-working algorithms to acquire 3D models from depth sensors, both for large and small scale scenes. However, these methods often break down when the scene geometry is not so informative, for example, in the case of planar surfaces. Similarly, standard image-based methods fail for texture-less scenes. We combine both color and depth measurements from an RGB-D sensor to simultaneously reconstruct both the camera motion and the scene geometry in a robust manner. Experiments on real data show that we can accurately reconstruct large-scale 3D scenes despite many planar surfaces

CopyMe3D: Scanning and Printing Persons in 3D

Abstract. In this paper, we describe a novel approach to create 3D miniatures of persons using a Kinect sensor and a 3D color printer. To achieve this, we acquire color and depth images while the person is rotating on a swivel chair. We represent the model with a signed distance function which is updated and visualized as the images are captured for immediate feedback. Our approach automatically fills small holes that stem from self-occlusions. To optimize the model for 3D printing, we extract a watertight but hollow shell to minimize the production costs. In extensive experiments, we evaluate the quality of the obtained models as a function of the rotation speed, the non-rigid deformations of a person during recording, the camera pose, and the resulting self-occlusions. Finally, we present a large number of reconstructions and fabricated figures to demonstrate the validity of our approach.

Minimizing the maximal rank

In computer vision, many problems can be formulated as finding a low rank approximation of a given matrix. Ideally, if all elements of the measurement matrix are available, this is easily solved in the L2-norm using factorization. However, in practice this is rarely the case. Lately, this problem has been addressed using different approaches, one is to replace the rank term by the convex nuclear norm, another is to derive the convex envelope of the rank term plus a data term. In the latter case, matrices are divided into sub-matrices and the envelope is computed for each subblock individually. In this paper a new convex envelope is derived which takes all sub-matrices into account simultaneously. This leads to a simpler formulation, using only one parameter to control the trade-of between rank and data fit, for applications where one seeks low rank approximations of multiple matrices with the same rank. We show in this paper how our general framework can be used for manifold denoising of several images at once, as well as just denoising one image. Experimental comparisons show that our method achieves results similar to state-of-the-art approaches while being applicable for other problems such as linear shape model estimation

Direct Camera Pose Tracking and Mapping With Signed Distance Functions

Abstract—In many areas, the ability to create accurate 3D models is of great interest, for example, in computer vision, robotics, architecture, and augmented reality. In this paper we show how a textured indoor environment can be reconstructed in 3D using an RGB-D camera. Real-time performance can be achieved using a GPU. We show how the camera pose can be estimated directly using the geometry that we represent as a signed distance function (SDF). Since the SDF contains information about the distance to the surface, it defines an error-metric which is minimized to estimate the pose of the camera. By iteratively estimating the camera pose and integrating the new depth images into the model, the 3D reconstruction is computed on the fly. We present several examples of 3D reconstructions made from a handheld and robot-mounted depth sensor, including detailed reconstructions from medium-sized rooms with almost drift-free pose estimation. Furthermore, we demonstrate that our algorithm is robust enough for 3D reconstruction using data recorded from a quadrocopter, making it potentially useful for navigation applications. I