A Joint Intensity and Depth Co-Sparse Analysis Model for Depth Map Super-Resolution

High-resolution depth maps can be inferred from low-resolution depth measurements and an additional high-resolution intensity image of the same scene. To that end, we introduce a bimodal co-sparse analysis model, which is able to capture the interdependency of registered intensity and depth information. This model is based on the assumption that the co-supports of corresponding bimodal image structures are aligned when computed by a suitable pair of analysis operators. No analytic form of such operators exist and we propose a method for learning them from a set of registered training signals. This learning process is done offline and returns a bimodal analysis operator that is universally applicable to natural scenes. We use this to exploit the bimodal co-sparse analysis model as a prior for solving inverse problems, which leads to an efficient algorithm for depth map super-resolution.Comment: 13 pages, 4 figure

Deformable Shape Completion with Graph Convolutional Autoencoders

The availability of affordable and portable depth sensors has made scanning objects and people simpler than ever. However, dealing with occlusions and missing parts is still a significant challenge. The problem of reconstructing a (possibly non-rigidly moving) 3D object from a single or multiple partial scans has received increasing attention in recent years. In this work, we propose a novel learning-based method for the completion of partial shapes. Unlike the majority of existing approaches, our method focuses on objects that can undergo non-rigid deformations. The core of our method is a variational autoencoder with graph convolutional operations that learns a latent space for complete realistic shapes. At inference, we optimize to find the representation in this latent space that best fits the generated shape to the known partial input. The completed shape exhibits a realistic appearance on the unknown part. We show promising results towards the completion of synthetic and real scans of human body and face meshes exhibiting different styles of articulation and partiality.Comment: CVPR 201

Xtru3D: Single-View 3D Object Reconstruction from Color and Depth Data

D object reconstruction from single image has been a noticeable research trend in recent years. The most common method is to rely on symmetries of real-life objects, but these are hard to compute in practice. However, a large class of everyday objects, especially when manufactured, can be generated by extruding a 2D shape through an extrusion axis. This paper proposes to exploit this property to acquire 3D object models using a single RGB+Depth image, such as those provided by available low-cost range cameras. It estimates the hidden parts by exploiting the geometrical properties of everyday objects, and both depth and color information are combined to refine the model of the object of interest. Experimental results on a set of 12 common objects are shown to demonstrate not only the effectiveness and simplicity of our approach, but also its applicability for tasks such as robotic grasping.The research leading to these results has been funded by the HANDLE European project (FP7/2007-2013) under grant agreement ICT 231640-http://www.handle-project.eu.Publicad

Real-Time Object Removal in Augmented Reality

Diminished reality, as a sub-topic of augmented reality where digital information is overlaid on an environment, is the perceived removal of an object from an environment. Previous approaches to diminished reality used digital replacement techniques, inpainting, and multi-view homographies. However, few used a virtual representation of the real environment, limiting their domains to planar environments. This thesis provides a framework to achieve real-time diminished reality on an augmented reality headset. Using state-of-the-art hardware, we combine a virtual representation of the real environment with inpainting to remove existing objects from complex environments. Our work is found to be competitive with previous results, with a similar qualitative outcome under the limitations of available technology. Additionally, by implementing new texturing algorithms, a more detailed representation of the real environment is achieved