When performing localization and mapping, working at the level of structure
can be advantageous in terms of robustness to environmental changes and
differences in illumination. This paper presents SegMap: a map representation
solution to the localization and mapping problem based on the extraction of
segments in 3D point clouds. In addition to facilitating the computationally
intensive task of processing 3D point clouds, working at the level of segments
addresses the data compression requirements of real-time single- and
multi-robot systems. While current methods extract descriptors for the single
task of localization, SegMap leverages a data-driven descriptor in order to
extract meaningful features that can also be used for reconstructing a dense 3D
map of the environment and for extracting semantic information. This is
particularly interesting for navigation tasks and for providing visual feedback
to end-users such as robot operators, for example in search and rescue
scenarios. These capabilities are demonstrated in multiple urban driving and
search and rescue experiments. Our method leads to an increase of area under
the ROC curve of 28.3% over current state of the art using eigenvalue based
features. We also obtain very similar reconstruction capabilities to a model
specifically trained for this task. The SegMap implementation will be made
available open-source along with easy to run demonstrations at
www.github.com/ethz-asl/segmap. A video demonstration is available at
https://youtu.be/CMk4w4eRobg