Deformable Shape Completion with Graph Convolutional Autoencoders

The availability of affordable and portable depth sensors has made scanning objects and people simpler than ever. However, dealing with occlusions and missing parts is still a significant challenge. The problem of reconstructing a (possibly non-rigidly moving) 3D object from a single or multiple partial scans has received increasing attention in recent years. In this work, we propose a novel learning-based method for the completion of partial shapes. Unlike the majority of existing approaches, our method focuses on objects that can undergo non-rigid deformations. The core of our method is a variational autoencoder with graph convolutional operations that learns a latent space for complete realistic shapes. At inference, we optimize to find the representation in this latent space that best fits the generated shape to the known partial input. The completed shape exhibits a realistic appearance on the unknown part. We show promising results towards the completion of synthetic and real scans of human body and face meshes exhibiting different styles of articulation and partiality.Comment: CVPR 201

Multiple depth maps integration for 3D reconstruction using geodesic graph cuts

Depth images, in particular depth maps estimated from stereo vision, may have a substantial amount of outliers and result in inaccurate 3D modelling and reconstruction. To address this challenging issue, in this paper, a graph-cut based multiple depth maps integration approach is proposed to obtain smooth and watertight surfaces. First, confidence maps for the depth images are estimated to suppress noise, based on which reliable patches covering the object surface are determined. These patches are then exploited to estimate the path weight for 3D geodesic distance computation, where an adaptive regional term is introduced to deal with the “shorter-cuts” problem caused by the effect of the minimal surface bias. Finally, the adaptive regional term and the boundary term constructed using patches are combined in the graph-cut framework for more accurate and smoother 3D modelling. We demonstrate the superior performance of our algorithm on the well-known Middlebury multi-view database and additionally on real-world multiple depth images captured by Kinect. The experimental results have shown that our method is able to preserve the object protrusions and details while maintaining surface smoothness

Neural Process Reconstruction from Sparse User Scribbles

We present a novel semi-automatic method for segmenting neural processes in large, highly anisotropic EM (electron microscopy) image stacks. Our method takes advantage of sparse scribble annotations provided by the user to guide a 3D variational segmentation model, thereby allowing our method to globally optimally enforce 3D geometric constraints on the segmentation. Moreover, we leverage a novel algorithm for propagating segmentation constraints through the image stack via optimal volumetric pathways, thereby allowing our method to compute highly accurate 3D segmentations from very sparse user input. We evaluate our method by reconstructing 16 neural processes in a 1024×1024×50 nanometer-scale EM image stack of a mouse hippocampus. We demonstrate that, on average, our method is 68% more accurate than previous state-of-the-art semi-automatic methods.Engineering and Applied Science

A graph-spectral approach to shape-from-shading

In this paper, we explore how graph-spectral methods can be used to develop a new shape-from-shading algorithm. We characterize the field of surface normals using a weight matrix whose elements are computed from the sectional curvature between different image locations and penalize large changes in surface normal direction. Modeling the blocks of the weight matrix as distinct surface patches, we use a graph seriation method to find a surface integration path that maximizes the sum of curvature-dependent weights and that can be used for the purposes of height reconstruction. To smooth the reconstructed surface, we fit quadrics to the height data for each patch. The smoothed surface normal directions are updated ensuring compliance with Lambert's law. The processes of height recovery and surface normal adjustment are interleaved and iterated until a stable surface is obtained. We provide results on synthetic and real-world imagery

Methodology for automatic recovering of 3D partitions from unstitched faces of non-manifold CAD models

Data exchanges between different software are currently used in industry to speed up the preparation of digital prototypes for Finite Element Analysis (FEA). Unfortunately, due to data loss, the yield of the transfer of manifold models rarely reaches 1. In the case of non-manifold models, the transfer results are even less satisfactory. This is particularly true for partitioned 3D models: during the data transfer based on the well-known exchange formats, all 3D partitions are generally lost. Partitions are mainly used for preparing mesh models required for advanced FEA: mapped meshing, material separation, definition of specific boundary conditions, etc. This paper sets up a methodology to automatically recover 3D partitions from exported non-manifold CAD models in order to increase the yield of the data exchange. Our fully automatic approach is based on three steps. First, starting from a set of potentially disconnected faces, the CAD model is stitched. Then, the shells used to create the 3D partitions are recovered using an iterative propagation strategy which starts from the so-called manifold vertices. Finally, using the identified closed shells, the 3D partitions can be reconstructed. The proposed methodology has been validated on academic as well as industrial examples.This work has been carried out under a research contract between the Research and Development Direction of the EDF Group and the Arts et Métiers ParisTech Aix-en-Provence

Discrete Bulk Reconstruction

According to the AdS/CFT correspondence, the geometries of certain spacetimes are fully determined by quantum states that live on their boundaries -- indeed, by the von Neumann entropies of portions of those boundary states. This work investigates to what extent the geometries can be reconstructed from the entropies in polynomial time. Bouland, Fefferman, and Vazirani (2019) argued that the AdS/CFT map can be exponentially complex if one wants to reconstruct regions such as the interiors of black holes. Our main result provides a sort of converse: we show that, in the special case of a single 1D boundary, if the input data consists of a list of entropies of contiguous boundary regions, and if the entropies satisfy a single inequality called Strong Subadditivity, then we can construct a graph model for the bulk in linear time. Moreover, the bulk graph is planar, it has $O(N^2)$ vertices (the information-theoretic minimum), and it's ``universal,'' with only the edge weights depending on the specific entropies in question. From a combinatorial perspective, our problem boils down to an ``inverse'' of the famous min-cut problem: rather than being given a graph and asked to find a min-cut, here we're given the values of min-cuts separating various sets of vertices, and need to find a weighted undirected graph consistent with those values. Our solution to this problem relies on the notion of a ``bulkless'' graph, which might be of independent interest for AdS/CFT. We also make initial progress on the case of multiple 1D boundaries -- where the boundaries could be connected via wormholes -- including an upper bound of $O(N^4)$ vertices whenever a planar bulk graph exists (thus putting the problem into the complexity class $\mathsf{NP}$).Comment: 41 pages, 18 figures. Comments welcomed! v2: new corollaries 2.3 and 4.5 with more explicit discussions of computability, additional references and discussio