110 research outputs found
Box-level Segmentation Supervised Deep Neural Networks for Accurate and Real-time Multispectral Pedestrian Detection
Effective fusion of complementary information captured by multi-modal sensors
(visible and infrared cameras) enables robust pedestrian detection under
various surveillance situations (e.g. daytime and nighttime). In this paper, we
present a novel box-level segmentation supervised learning framework for
accurate and real-time multispectral pedestrian detection by incorporating
features extracted in visible and infrared channels. Specifically, our method
takes pairs of aligned visible and infrared images with easily obtained
bounding box annotations as input and estimates accurate prediction maps to
highlight the existence of pedestrians. It offers two major advantages over the
existing anchor box based multispectral detection methods. Firstly, it
overcomes the hyperparameter setting problem occurred during the training phase
of anchor box based detectors and can obtain more accurate detection results,
especially for small and occluded pedestrian instances. Secondly, it is capable
of generating accurate detection results using small-size input images, leading
to improvement of computational efficiency for real-time autonomous driving
applications. Experimental results on KAIST multispectral dataset show that our
proposed method outperforms state-of-the-art approaches in terms of both
accuracy and speed
Unsupervised Domain Adaptation for Multispectral Pedestrian Detection
Multimodal information (e.g., visible and thermal) can generate robust
pedestrian detections to facilitate around-the-clock computer vision
applications, such as autonomous driving and video surveillance. However, it
still remains a crucial challenge to train a reliable detector working well in
different multispectral pedestrian datasets without manual annotations. In this
paper, we propose a novel unsupervised domain adaptation framework for
multispectral pedestrian detection, by iteratively generating pseudo
annotations and updating the parameters of our designed multispectral
pedestrian detector on target domain. Pseudo annotations are generated using
the detector trained on source domain, and then updated by fixing the
parameters of detector and minimizing the cross entropy loss without
back-propagation. Training labels are generated using the pseudo annotations by
considering the characteristics of similarity and complementarity between
well-aligned visible and infrared image pairs. The parameters of detector are
updated using the generated labels by minimizing our defined multi-detection
loss function with back-propagation. The optimal parameters of detector can be
obtained after iteratively updating the pseudo annotations and parameters.
Experimental results show that our proposed unsupervised multimodal domain
adaptation method achieves significantly higher detection performance than the
approach without domain adaptation, and is competitive with the supervised
multispectral pedestrian detectors
TFDet: Target-aware Fusion for RGB-T Pedestrian Detection
Pedestrian detection plays a critical role in computer vision as it
contributes to ensuring traffic safety. Existing methods that rely solely on
RGB images suffer from performance degradation under low-light conditions due
to the lack of useful information. To address this issue, recent multispectral
detection approaches have combined thermal images to provide complementary
information and have obtained enhanced performances. Nevertheless, few
approaches focus on the negative effects of false positives caused by noisy
fused feature maps. Different from them, we comprehensively analyze the impacts
of false positives on the detection performance and find that enhancing feature
contrast can significantly reduce these false positives. In this paper, we
propose a novel target-aware fusion strategy for multispectral pedestrian
detection, named TFDet. Our fusion strategy highlights the pedestrian-related
features while suppressing unrelated ones, resulting in more discriminative
fused features. TFDet achieves state-of-the-art performance on both KAIST and
LLVIP benchmarks, with an efficiency comparable to the previous
state-of-the-art counterpart. Importantly, TFDet performs remarkably well even
under low-light conditions, which is a significant advancement for ensuring
road safety. The code will be made publicly available at
\url{https://github.com/XueZ-phd/TFDet.git}
- …