Multispectral Deep Neural Networks for Pedestrian Detection

Multispectral pedestrian detection is essential for around-the-clock applications, e.g., surveillance and autonomous driving. We deeply analyze Faster R-CNN for multispectral pedestrian detection task and then model it into a convolutional network (ConvNet) fusion problem. Further, we discover that ConvNet-based pedestrian detectors trained by color or thermal images separately provide complementary information in discriminating human instances. Thus there is a large potential to improve pedestrian detection by using color and thermal images in DNNs simultaneously. We carefully design four ConvNet fusion architectures that integrate two-branch ConvNets on different DNNs stages, all of which yield better performance compared with the baseline detector. Our experimental results on KAIST pedestrian benchmark show that the Halfway Fusion model that performs fusion on the middle-level convolutional features outperforms the baseline method by 11% and yields a missing rate 3.5% lower than the other proposed architectures.Comment: 13 pages, 8 figures, BMVC 2016 ora

Response to Wang and Luo

This article is a response to Wang and Luo

Deep GrabCut for Object Selection

Most previous bounding-box-based segmentation methods assume the bounding box tightly covers the object of interest. However it is common that a rectangle input could be too large or too small. In this paper, we propose a novel segmentation approach that uses a rectangle as a soft constraint by transforming it into an Euclidean distance map. A convolutional encoder-decoder network is trained end-to-end by concatenating images with these distance maps as inputs and predicting the object masks as outputs. Our approach gets accurate segmentation results given sloppy rectangles while being general for both interactive segmentation and instance segmentation. We show our network extends to curve-based input without retraining. We further apply our network to instance-level semantic segmentation and resolve any overlap using a conditional random field. Experiments on benchmark datasets demonstrate the effectiveness of the proposed approaches.Comment: BMVC 201

Alternative approach to all-angle negative refraction in two-dimensional photonic crystals

We show that with an appropriate surface modification, a slab of photonic crystal can be made to allow wave transmission within the band gap. Furthermore, negative refraction and all-angle-negative-refraction (AANR) can be achieved by this surface modification in frequency windows that were not realized before in two-dimensional photonic crystals [C. Luo et al, Phys. Rev. B 65, 201104 (2002)]. This approach to AANR leads to new applications in flat lens imaging. Previous flat lens using photonic crystals requires object-image distance u+v less than or equal to the lens thickness d, u+v d. Our approach can be used to design flat lens with u+v=sd with s>>1, thus being able to image large and/or far away objects. Our results are confirmed by FDTD simulations.Comment: 5 pages, 9 eps figs in RevTex forma