81 research outputs found
Pointwise Convolutional Neural Networks
Deep learning with 3D data such as reconstructed point clouds and CAD models
has received great research interests recently. However, the capability of
using point clouds with convolutional neural network has been so far not fully
explored. In this paper, we present a convolutional neural network for semantic
segmentation and object recognition with 3D point clouds. At the core of our
network is pointwise convolution, a new convolution operator that can be
applied at each point of a point cloud. Our fully convolutional network design,
while being surprisingly simple to implement, can yield competitive accuracy in
both semantic segmentation and object recognition task.Comment: 10 pages, 6 figures, 10 tables. Paper accepted to CVPR 201
ShellNet: Efficient Point Cloud Convolutional Neural Networks using Concentric Shells Statistics
Deep learning with 3D data has progressed significantly since the
introduction of convolutional neural networks that can handle point order
ambiguity in point cloud data. While being able to achieve good accuracies in
various scene understanding tasks, previous methods often have low training
speed and complex network architecture. In this paper, we address these
problems by proposing an efficient end-to-end permutation invariant convolution
for point cloud deep learning. Our simple yet effective convolution operator
named ShellConv uses statistics from concentric spherical shells to define
representative features and resolve the point order ambiguity, allowing
traditional convolution to perform on such features. Based on ShellConv we
further build an efficient neural network named ShellNet to directly consume
the point clouds with larger receptive fields while maintaining less layers. We
demonstrate the efficacy of ShellNet by producing state-of-the-art results on
object classification, object part segmentation, and semantic scene
segmentation while keeping the network very fast to train.Comment: International Conference on Computer Vision (ICCV) 2019 Ora
Locally Stylized Neural Radiance Fields
In recent years, there has been increasing interest in applying stylization
on 3D scenes from a reference style image, in particular onto neural radiance
fields (NeRF). While performing stylization directly on NeRF guarantees
appearance consistency over arbitrary novel views, it is a challenging problem
to guide the transfer of patterns from the style image onto different parts of
the NeRF scene. In this work, we propose a stylization framework for NeRF based
on local style transfer. In particular, we use a hash-grid encoding to learn
the embedding of the appearance and geometry components, and show that the
mapping defined by the hash table allows us to control the stylization to a
certain extent. Stylization is then achieved by optimizing the appearance
branch while keeping the geometry branch fixed. To support local style
transfer, we propose a new loss function that utilizes a segmentation network
and bipartite matching to establish region correspondences between the style
image and the content images obtained from volume rendering. Our experiments
show that our method yields plausible stylization results with novel view
synthesis while having flexible controllability via manipulating and
customizing the region correspondences.Comment: ICCV 202
Rotation Invariant Convolutions for 3D Point Clouds Deep Learning
Recent progresses in 3D deep learning has shown that it is possible to design
special convolution operators to consume point cloud data. However, a typical
drawback is that rotation invariance is often not guaranteed, resulting in
networks being trained with data augmented with rotations. In this paper, we
introduce a novel convolution operator for point clouds that achieves rotation
invariance. Our core idea is to use low-level rotation invariant geometric
features such as distances and angles to design a convolution operator for
point cloud learning. The well-known point ordering problem is also addressed
by a binning approach seamlessly built into the convolution. This convolution
operator then serves as the basic building block of a neural network that is
robust to point clouds under 6DoF transformations such as translation and
rotation. Our experiment shows that our method performs with high accuracy in
common scene understanding tasks such as object classification and
segmentation. Compared to previous works, most importantly, our method is able
to generalize and achieve consistent results across different scenarios in
which training and testing can contain arbitrary rotations.Comment: International Conference on 3D Vision (3DV) 201
Unbounded High Dynamic Range Photography Using a Modulo Camera
This paper presents a novel framework to extend the dynamic range of images called Unbounded High Dynamic Range (UHDR) photography with a modulo camera. A modulo camera could theoretically take unbounded radiance levels by keeping only the least significant bits. We show that with limited bit depth, very high radiance levels can be recovered from a single modulus image with our newly proposed unwrapping algorithm for natural images. We can also obtain an HDR image with details equally well preserved for all radiance levels by merging the least number of modulus images. Synthetic experiment and experiment with a real modulo camera show the effectiveness of the proposed approach.United States. Dept. of Defense. Assistant Secretary of Defense for Research & Engineering (United States. Air Force Contract FA8721-05-C-0002)SUTD-MIT (Joint Doctoral Fellowship
- …