Carving from Ray-Tracing Constraints: IRT-Carving

We present a new algorithm for improving an available (conservative) estimate of the shape of an object using constraints from ray-tracing. In particular, we exploit incoherences between the lit portions of the object - detected on a set of acquired images - and the shadows that the current estimate casts on itself. Whenever a contradiction is found the current estimate is modified in order to remove the inconsistency. Sufficient conditions for the correctness of the algorithm and a discussion of their validity are provided. Finally, we describe a simple implementation of the method and present some preliminary experimental results from computer simulations

Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Images

Recovering the 3D representation of an object from single-view or multi-view RGB images by deep neural networks has attracted increasing attention in the past few years. Several mainstream works (e.g., 3D-R2N2) use recurrent neural networks (RNNs) to fuse multiple feature maps extracted from input images sequentially. However, when given the same set of input images with different orders, RNN-based approaches are unable to produce consistent reconstruction results. Moreover, due to long-term memory loss, RNNs cannot fully exploit input images to refine reconstruction results. To solve these problems, we propose a novel framework for single-view and multi-view 3D reconstruction, named Pix2Vox. By using a well-designed encoder-decoder, it generates a coarse 3D volume from each input image. Then, a context-aware fusion module is introduced to adaptively select high-quality reconstructions for each part (e.g., table legs) from different coarse 3D volumes to obtain a fused 3D volume. Finally, a refiner further refines the fused 3D volume to generate the final output. Experimental results on the ShapeNet and Pix3D benchmarks indicate that the proposed Pix2Vox outperforms state-of-the-arts by a large margin. Furthermore, the proposed method is 24 times faster than 3D-R2N2 in terms of backward inference time. The experiments on ShapeNet unseen 3D categories have shown the superior generalization abilities of our method.Comment: ICCV 201

DeformNet: Free-Form Deformation Network for 3D Shape Reconstruction from a Single Image

3D reconstruction from a single image is a key problem in multiple applications ranging from robotic manipulation to augmented reality. Prior methods have tackled this problem through generative models which predict 3D reconstructions as voxels or point clouds. However, these methods can be computationally expensive and miss fine details. We introduce a new differentiable layer for 3D data deformation and use it in DeformNet to learn a model for 3D reconstruction-through-deformation. DeformNet takes an image input, searches the nearest shape template from a database, and deforms the template to match the query image. We evaluate our approach on the ShapeNet dataset and show that - (a) the Free-Form Deformation layer is a powerful new building block for Deep Learning models that manipulate 3D data (b) DeformNet uses this FFD layer combined with shape retrieval for smooth and detail-preserving 3D reconstruction of qualitatively plausible point clouds with respect to a single query image (c) compared to other state-of-the-art 3D reconstruction methods, DeformNet quantitatively matches or outperforms their benchmarks by significant margins. For more information, visit: https://deformnet-site.github.io/DeformNet-website/ .Comment: 11 pages, 9 figures, NIP

Few-Shot Single-View 3-D Object Reconstruction with Compositional Priors

The impressive performance of deep convolutional neural networks in single-view 3D reconstruction suggests that these models perform non-trivial reasoning about the 3D structure of the output space. However, recent work has challenged this belief, showing that complex encoder-decoder architectures perform similarly to nearest-neighbor baselines or simple linear decoder models that exploit large amounts of per category data in standard benchmarks. On the other hand settings where 3D shape must be inferred for new categories with few examples are more natural and require models that generalize about shapes. In this work we demonstrate experimentally that naive baselines do not apply when the goal is to learn to reconstruct novel objects using very few examples, and that in a \emph{few-shot} learning setting, the network must learn concepts that can be applied to new categories, avoiding rote memorization. To address deficiencies in existing approaches to this problem, we propose three approaches that efficiently integrate a class prior into a 3D reconstruction model, allowing to account for intra-class variability and imposing an implicit compositional structure that the model should learn. Experiments on the popular ShapeNet database demonstrate that our method significantly outperform existing baselines on this task in the few-shot setting

Weakly supervised 3D Reconstruction with Adversarial Constraint

Supervised 3D reconstruction has witnessed a significant progress through the use of deep neural networks. However, this increase in performance requires large scale annotations of 2D/3D data. In this paper, we explore inexpensive 2D supervision as an alternative for expensive 3D CAD annotation. Specifically, we use foreground masks as weak supervision through a raytrace pooling layer that enables perspective projection and backpropagation. Additionally, since the 3D reconstruction from masks is an ill posed problem, we propose to constrain the 3D reconstruction to the manifold of unlabeled realistic 3D shapes that match mask observations. We demonstrate that learning a log-barrier solution to this constrained optimization problem resembles the GAN objective, enabling the use of existing tools for training GANs. We evaluate and analyze the manifold constrained reconstruction on various datasets for single and multi-view reconstruction of both synthetic and real images

Surface Reconstruction and Evolution from Multiple Views

Applications like 3D Telepresence necessitate faithful 3D surface reconstruction of the object and 3D data compression in both spatial and temporal domains. This makes us feel immersed in virtual environments there by making 3D Telepresence a powerful tool in many applications. Hence 3D surface reconstruction and 3D compression are two challenging problems which are addressed in this thesis

Ubiquitous Positioning: A Taxonomy for Location Determination on Mobile Navigation System

The location determination in obstructed area can be very challenging especially if Global Positioning System are blocked. Users will find it difficult to navigate directly on-site in such condition, especially indoor car park lot or obstructed environment. Sometimes, it needs to combine with other sensors and positioning methods in order to determine the location with more intelligent, reliable and ubiquity. By using ubiquitous positioning in mobile navigation system, it is a promising ubiquitous location technique in a mobile phone since as it is a familiar personal electronic device for many people. However, as research on ubiquitous positioning systems goes beyond basic methods there is an increasing need for better comparison of proposed ubiquitous positioning systems. System developers are also lacking of good frameworks for understanding different options during building ubiquitous positioning systems. This paper proposes taxonomy to address both of these problems. The proposed taxonomy has been constructed from a literature study of papers and articles on positioning estimation that can be used to determine location everywhere on mobile navigation system. For researchers the taxonomy can also be used as an aid for scoping out future research in the area of ubiquitous positioning.Comment: 15 Pages, 3 figure

An Investigation of Object Shadows Utilization In 3D Shape Re-Construction Using Inexpensive Equipment

An approach for automatic 3D object re-construction using its shadow ispresented. The approach investigates the use of information inherited by thegenerated object shadows to re-construct the object geometry. An algorithm isdeveloped that make use of object height information for the directions associatedwith the incident light and the generated object shadows, hence, acquired heightfeatures represents the object features that have actually obstructed the incidentlight. The technique is tested using objects of different shapes. Close to realmeasurements are gained and the overall accuracy of the system is found to bewithin 0.75 mm using the adopted imaging hardware and setup. Obtained resultsconfirmed the validity of the proposed approach