OctNetFusion: Learning Depth Fusion from Data

In this paper, we present a learning based approach to depth fusion, i.e., dense 3D reconstruction from multiple depth images. The most common approach to depth fusion is based on averaging truncated signed distance functions, which was originally proposed by Curless and Levoy in 1996. While this method is simple and provides great results, it is not able to reconstruct (partially) occluded surfaces and requires a large number frames to filter out sensor noise and outliers. Motivated by the availability of large 3D model repositories and recent advances in deep learning, we present a novel 3D CNN architecture that learns to predict an implicit surface representation from the input depth maps. Our learning based method significantly outperforms the traditional volumetric fusion approach in terms of noise reduction and outlier suppression. By learning the structure of real world 3D objects and scenes, our approach is further able to reconstruct occluded regions and to fill in gaps in the reconstruction. We demonstrate that our learning based approach outperforms both vanilla TSDF fusion as well as TV-L1 fusion on the task of volumetric fusion. Further, we demonstrate state-of-the-art 3D shape completion results.Comment: 3DV 2017, https://github.com/griegler/octnetfusio

Dynamic Adaptive Point Cloud Streaming

High-quality point clouds have recently gained interest as an emerging form of representing immersive 3D graphics. Unfortunately, these 3D media are bulky and severely bandwidth intensive, which makes it difficult for streaming to resource-limited and mobile devices. This has called researchers to propose efficient and adaptive approaches for streaming of high-quality point clouds. In this paper, we run a pilot study towards dynamic adaptive point cloud streaming, and extend the concept of dynamic adaptive streaming over HTTP (DASH) towards DASH-PC, a dynamic adaptive bandwidth-efficient and view-aware point cloud streaming system. DASH-PC can tackle the huge bandwidth demands of dense point cloud streaming while at the same time can semantically link to human visual acuity to maintain high visual quality when needed. In order to describe the various quality representations, we propose multiple thinning approaches to spatially sub-sample point clouds in the 3D space, and design a DASH Media Presentation Description manifest specific for point cloud streaming. Our initial evaluations show that we can achieve significant bandwidth and performance improvement on dense point cloud streaming with minor negative quality impacts compared to the baseline scenario when no adaptations is applied.Comment: 6 pages, 23rd ACM Packet Video (PV'18) Workshop, June 12--15, 2018, Amsterdam, Netherland

Geometric transformations in octrees using shears

Existent algorithms to perform geometric transformations on octrees can be classified in two families: inverse transformation and address computation ones. Those in the inverse transformation family essentially resample the target octree from the source one, and are able to cope with all the affine transformations. Those in the address computation family only deal with translations, but are commonly accepted as faster than the former ones for they do no intersection tests, but directly calculate the transformed address of each black node in the source tree. This work introduces a new translation algorithm that shows to perform better than previous one when very small displacements are involved. This property is particularly useful in applications such as simulation, robotics or computer animation.Postprint (published version

Highly efficient computer oriented octree data structure and neighbors search in 3D GIS spatial

Three Dimensional (3D) have given new perspective in various field such as urban planning, hydrology, infrastructure modeling, geology etc due to its capability of handling real world object in more realistic manners, rather than two-dimensional (2D) approach. However, implementation of 3D spatial analysis in the real world has proven difficult due to the complexity of algorithm, computational power and time consuming. Existing GIS system enables 2D and two-and-a-half dimensional (2.5D) spatial datasets, but less capable of supporting 3D data structures. Recent development in Octree see more effort to improve weakness of octree in finding neighbor node by using various address encoding scheme with specific rule to eliminate the need of tree traversal. This paper proposed a new method to speed up neighbor searching and eliminating the needs of complex operation to extract spatial information from octree by preserving 3D spatial information directly from Octree data structure. This new method able to achieve O(1) complexity and utilizing Bit Manipulation Instruction 2 (BMI2) to speedup address encoding, extraction and voxel search 700% compared with generic implementation