Image-to-Geometry Registration on Mobile Devices – Concepts, Challenges and Applications

Registering natural photos to existing 3D surface models, particularly on low-power mobile devices, gathers increasing attention to a variety of application domains. The paper discusses up-to-date computation insights of the technique, condensing available literature and knowledge obtained from experiments across multiple research groups. Challenges like smartphone camera calibration or the sensor-based estimation of location and orientation are current research subjects, for which new data and experimental results are presented. Moreover, computing-related, practical challenges (e.g. device variability) are detailed to increase the technological understanding and reasoning on the limits of mobile devices. An overview of running projects utilising image-to-geometry registration methods shows the potential for mobile devices to, amongst others, improve flood hazard mitigation and hydrocarbon exploration with crowdsourced data