Collecting 3D object datasets involves a large amount of manual work and is
time consuming. Getting complete models of objects either requires a 3D scanner
that covers all the surfaces of an object or one needs to rotate it to
completely observe it. We present a system that incrementally builds a database
of objects as a mobile agent traverses a scene. Our approach requires no prior
knowledge of the shapes present in the scene. Object-like segments are
extracted from a global segmentation map, which is built online using the input
of segmented RGB-D images. These segments are stored in a database, matched
among each other, and merged with other previously observed instances. This
allows us to create and improve object models on the fly and to use these
merged models to reconstruct also unobserved parts of the scene. The database
contains each (potentially merged) object model only once, together with a set
of poses where it was observed. We evaluate our pipeline with one public
dataset, and on a newly created Google Tango dataset containing four indoor
scenes with some of the objects appearing multiple times, both within and
across scenes