3D LiDARs and 2D cameras are increasingly being used alongside each other in
sensor rigs for perception tasks. Before these sensors can be used to gather
meaningful data, however, their extrinsics (and intrinsics) need to be
accurately calibrated, as the performance of the sensor rig is extremely
sensitive to these calibration parameters. A vast majority of existing
calibration techniques require significant amounts of data and/or calibration
targets and human effort, severely impacting their applicability in large-scale
production systems. We address this gap with CalibNet: a self-supervised deep
network capable of automatically estimating the 6-DoF rigid body transformation
between a 3D LiDAR and a 2D camera in real-time. CalibNet alleviates the need
for calibration targets, thereby resulting in significant savings in
calibration efforts. During training, the network only takes as input a LiDAR
point cloud, the corresponding monocular image, and the camera calibration
matrix K. At train time, we do not impose direct supervision (i.e., we do not
directly regress to the calibration parameters, for example). Instead, we train
the network to predict calibration parameters that maximize the geometric and
photometric consistency of the input images and point clouds. CalibNet learns
to iteratively solve the underlying geometric problem and accurately predicts
extrinsic calibration parameters for a wide range of mis-calibrations, without
requiring retraining or domain adaptation. The project page is hosted at
https://epiception.github.io/CalibNetComment: Appeared in the proccedings of the IEEE International Conference on
Intelligent Robots and Systems (IROS) 201