POL-LWIR Vehicle Detection: Convolutional Neural Networks Meet Polarised Infrared Sensors

For vehicle autonomy, driver assistance and situational awareness, it is necessary to operate at day and night, and in all weather conditions. In particular, long wave infrared (LWIR) sensors that receive predominantly emitted radiation have the capability to operate at night as well as during the day. In this work, we employ a polarised LWIR (POL-LWIR) camera to acquire data from a mobile vehicle, to compare and contrast four different convolutional neural network (CNN) configurations to detect other vehicles in video sequences. We evaluate two distinct and promising approaches, two-stage detection (Faster-RCNN) and one-stage detection (SSD), in four different configurations. We also employ two different image decompositions: the first based on the polarisation ellipse and the second on the Stokes parameters themselves. To evaluate our approach, the experimental trials were quantified by mean average precision (mAP) and processing time, showing a clear trade-off between the two factors. For example, the best mAP result of 80.94% was achieved using Faster-RCNN, but at a frame rate of 6.4 fps. In contrast, MobileNet SSD achieved only 64.51% mAP, but at 53.4 fps.Comment: Computer Vision and Pattern Recognition Workshop 201

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Deep-learning based single object tracker for night surveillance

Tracking an object in night surveillance video is a challenging task as the quality of the captured image is normally poor with low brightness and contrast. The task becomes harder for a small object as fewer features are apparent. Traditional approach is based on improving the image quality before tracking is performed. In this paper, a single object tracking algorithm based on deep-learning approach is proposed to exploit its outstanding capability of modelling object’s appearance even during night. The algorithm uses pre-trained convolutional neural networks coupled with fully connected layers, which are trained online during the tracking so that it is able to cater for appearance changes as the object moves around. Various learning hyperparameters for the optimization function, learning rate and ratio of training samples are tested to find optimal setup for tracking in night scenarios. Fourteen night surveillance videos are collected for validation purpose, which are captured from three viewing angles. The results show that the best accuracy is obtained by using Adam optimizer with learning rate of 0.00075 and sampling ratio of 2:1 for positive and negative training data. This algorithm is suitable to be implemented in higher level surveillance applications such as abnormal behavioral recognition