Fast Back-Projection for Non-Line of Sight Reconstruction

Recent works have demonstrated non-line of sight (NLOS) reconstruction by using the time-resolved signal frommultiply scattered light. These works combine ultrafast imaging systems with computation, which back-projects the recorded space-time signal to build a probabilistic map of the hidden geometry. Unfortunately, this computation is slow, becoming a bottleneck as the imaging technology improves. In this work, we propose a new back-projection technique for NLOS reconstruction, which is up to a thousand times faster than previous work, with almost no quality loss. We base on the observation that the hidden geometry probability map can be built as the intersection of the three-bounce space-time manifolds defined by the light illuminating the hidden geometry and the visible point receiving the scattered light from such hidden geometry. This allows us to pose the reconstruction of the hidden geometry as the voxelization of these space-time manifolds, which has lower theoretic complexity and is easily implementable in the GPU. We demonstrate the efficiency and quality of our technique compared against previous methods in both captured and synthetic dat

Hybrid Bayesian Eigenobjects: Combining Linear Subspace and Deep Network Methods for 3D Robot Vision

We introduce Hybrid Bayesian Eigenobjects (HBEOs), a novel representation for 3D objects designed to allow a robot to jointly estimate the pose, class, and full 3D geometry of a novel object observed from a single viewpoint in a single practical framework. By combining both linear subspace methods and deep convolutional prediction, HBEOs efficiently learn nonlinear object representations without directly regressing into high-dimensional space. HBEOs also remove the onerous and generally impractical necessity of input data voxelization prior to inference. We experimentally evaluate the suitability of HBEOs to the challenging task of joint pose, class, and shape inference on novel objects and show that, compared to preceding work, HBEOs offer dramatically improved performance in all three tasks along with several orders of magnitude faster runtime performance.Comment: To appear in the International Conference on Intelligent Robots (IROS) - Madrid, 201

Study and evaluation of correlation techniques for automotive sensor data

openI sistemi di guida completamente autonomi sono dotati di avanzati sensori in grado di rilevare e riconoscere rapidamente oggetti nell'ambiente circostante. Tuttavia, la generazione di enormi volumi di dati da parte di questi sensori può rappresentare una sfida considerevole per le tecnologie di comunicazione standard. In questa tesi, ci proponiamo di esaminare e convalidare diversi metodi per valutare la correlazione dei dati automobilistici utilizzando un dataset sintetico, al fine di determinare il modo più efficace per trasmettere tali dati. La nostra ricerca esplora vari metodi di correlazione, tra cui il coefficiente di correlazione di Pearson, la Chamfer Distance, l'algoritmo Iterative Closest Point (ICP) e la Normal Distribution Transform (NDT). Inoltre, sviluppiamo una nuova funzione di punteggio che combina la Chamfer Distance e la correlazione al fine di individuare cambiamenti significativi nei dati automobilistici. La nostra indagine si basa su un dataset sintetico e considera contesti sia statici che dinamici. I nostri risultati mettono in evidenza l'importante interazione tra correlazione e Chamfer Distance in ambienti dinamici e dimostrano che i dati automobilistici sono intrinsecamente correlati, pur richiedendo una certa ridondanza per garantire l'accuratezza, specialmente nelle aree critiche in cui sono essenziali i requisiti di sicurezza.Fully autonomous driving systems are equipped with sensors able to guarantee fast detection and recognition of sensitive objects in the environment. However, the resulting huge volumes of data generated from those sensors may be challenging to handle for standard communication technologies. Along these lines, in this thesis we test and validate different methods to evaluate correlation of automotive data on a synthetic dataset, to decide the most convenient way of transmitting data. The study investigates correlation methods such as Pearson’s correlation coefficient, the Chamfer Distance, the Iterative Closest Point (ICP), and the Normal Distribution Transform (NDT) algorithms. We also develop a new score function combining Chamfer Distance and correlation to detect significant changes in the automotive data. The research utilizes a synthetic dataset and examines both static and dynamic contexts. Our findings highlight the interplay between correlation and Chamfer Distance in dynamic environments, and demonstrate that automotive data are highly correlated, even though redundancy is needed to guarantee accuracy, especially in critical areas where safety requirements are particularly critical

Unsupervised Object-Centric Voxelization for Dynamic Scene Understanding

Understanding the compositional dynamics of multiple objects in unsupervised visual environments is challenging, and existing object-centric representation learning methods often ignore 3D consistency in scene decomposition. We propose DynaVol, an inverse graphics approach that learns object-centric volumetric representations in a neural rendering framework. DynaVol maintains time-varying 3D voxel grids that explicitly represent the probability of each spatial location belonging to different objects, and decouple temporal dynamics and spatial information by learning a canonical-space deformation field. To optimize the volumetric features, we embed them into a fully differentiable neural network, binding them to object-centric global features and then driving a compositional NeRF for scene reconstruction. DynaVol outperforms existing methods in novel view synthesis and unsupervised scene decomposition and allows for the editing of dynamic scenes, such as adding, deleting, replacing objects, and modifying their trajectories

Dense 3D Object Reconstruction from a Single Depth View

In this paper, we propose a novel approach, 3D-RecGAN++, which reconstructs the complete 3D structure of a given object from a single arbitrary depth view using generative adversarial networks. Unlike existing work which typically requires multiple views of the same object or class labels to recover the full 3D geometry, the proposed 3D-RecGAN++ only takes the voxel grid representation of a depth view of the object as input, and is able to generate the complete 3D occupancy grid with a high resolution of 256^3 by recovering the occluded/missing regions. The key idea is to combine the generative capabilities of autoencoders and the conditional Generative Adversarial Networks (GAN) framework, to infer accurate and fine-grained 3D structures of objects in high-dimensional voxel space. Extensive experiments on large synthetic datasets and real-world Kinect datasets show that the proposed 3D-RecGAN++ significantly outperforms the state of the art in single view 3D object reconstruction, and is able to reconstruct unseen types of objects.Comment: TPAMI 2018. Code and data are available at: https://github.com/Yang7879/3D-RecGAN-extended. This article extends from arXiv:1708.0796

3D-PhysNet: Learning the Intuitive Physics of Non-Rigid Object Deformations

The ability to interact and understand the environment is a fundamental prerequisite for a wide range of applications from robotics to augmented reality. In particular, predicting how deformable objects will react to applied forces in real time is a significant challenge. This is further confounded by the fact that shape information about encountered objects in the real world is often impaired by occlusions, noise and missing regions e.g. a robot manipulating an object will only be able to observe a partial view of the entire solid. In this work we present a framework, 3D-PhysNet, which is able to predict how a three-dimensional solid will deform under an applied force using intuitive physics modelling. In particular, we propose a new method to encode the physical properties of the material and the applied force, enabling generalisation over materials. The key is to combine deep variational autoencoders with adversarial training, conditioned on the applied force and the material properties. We further propose a cascaded architecture that takes a single 2.5D depth view of the object and predicts its deformation. Training data is provided by a physics simulator. The network is fast enough to be used in real-time applications from partial views. Experimental results show the viability and the generalisation properties of the proposed architecture.Comment: in IJCAI 201