We propose a weakly supervised semantic segmentation method for point clouds
that predicts "per-point" labels from just "whole-scene" annotations while
achieving the performance of recent fully supervised approaches. Our core idea
is to propagate the scene-level labels to each point in the point cloud by
creating pseudo labels in a conservative way. Specifically, we over-segment
point cloud features via unsupervised clustering and associate scene-level
labels with clusters through bipartite matching, thus propagating scene labels
only to the most relevant clusters, leaving the rest to be guided solely via
unsupervised clustering. We empirically demonstrate that over-segmentation and
bipartite assignment plays a crucial role. We evaluate our method on ScanNet
and S3DIS datasets, outperforming state of the art, and demonstrate that we can
achieve results comparable to fully supervised methods.Comment: The first two authors contributed equally; Project website:
https://densify-your-labels.github.io