Reconstructing partially visible models using stereo vision, structured light, and the g2o framework

ABSTRACT This paper describes a framework for model-based 3D reconstruction of vines and trellising for a robot equipped with stereo cameras and structured light. In each frame, high-level 2D features, and a sparse set of 3D structured light points are found. Detected features are matched to 3D model components, and the g2o optimisation framework is used to estimate both the model's structure and the camera's trajectory. The system is demonstrated reconstructing the trellising present in images of vines, together with the camera's trajectory, over a 12m track consisting of 360 sets of frames. The estimated model is structurally correct and is almost complete, and the estimated trajectory drifts by just 4%. Future work will extend the framework to reconstruct the more complex structure of the vines