Night-to-day: Online image-to-image translation for object detection within autonomous driving by night

Abstract

Object detectors are central to autonomous driving and widely used in driver assistance systems. Object detectors are trained on a finite amount of data, within a specific domain. This hampers detection performance when applying object detectors to samples from other domains during inference, an effect known as domain gap. Domain gap is a concern for data-driven applications, evoking repetitive retraining of networks when the applications unfold into other domains. With object detectors that have been trained on day images only, domain gap can be clearly observed in object detection by night. Training object detectors on night images is critical due to the enormous effort required to generate an adequate amount of diversely labeled data, and existing data sets often tend to overfit specific domain characteristics. This work proposes, for the first time, adapting domains by online image-to-image translation to expand an object detector's domain of operation. The domain gap is decreased without additional labeling effort, and without having to retrain the object detector, while unfolding into the target domain. The approach follows the concept of domain adaptation, shifting the samples of the target domain into the domain known to the object detector (source domain). Firstly, the UNIT network is trained for domain adaptation and subsequently cast into an online domain adaptation module, which narrows down the domain gap. Domain adaptation capabilities are evaluated qualitatively by displaying translated samples, as well as by visualizing the domain shift through the 2D tSNE algorithm. We quantitatively benchmark the influence of the domain adaptation on a state-of-the-art object detector, and on a retrained object detector, with respect to mean average precision, mean recall and the resulting F1 score. Applying online domain adaptation, our approach achieves a F1 score improvement of 5.27 %, within object detection by night. The evaluation is executed on the BDD100K benchmark data set