Analytical Study of Deep Learning Methods for Road Condition Assessment

Automated pavement distress recognition is a key step in smart infrastructure assessment. Advances in deep learning and computer vision have improved the automated recognition of pavement distresses in road surface images. This task, however, remains challenging due to the high variations in road objects and pavement types, variety of lighting condition, low contrast, and background noises in pavement images. In this dissertation, we propose novel deep learning algorithms for image-based road condition assessment to tackle current challenges in detection, classification and segmentation of pavement images. Motivated by the need for classifying a wide range of objects in road monitoring, this dissertation introduces a Multi-Scale Convolution Neural Network (MCNN) for multi-class classification of pavement images. MCNN improves the classification performance by encoding contextual information through multi-scale input tiles. Then, an Attention-Based Multi-Scale CNN (A+MCNN) is proposed to further improve the classification results through a novel mid-fusion strategy for combining multi-scale features extracted from multi-scale input tiles. An attention module is designed as an adaptive fusion strategy to generate importance scores and integrate multi-scale features based on how informative they are to the classification task. Finally, Dual Attention CNN (DACNN) is introduced to improve the performance of multi-class classification using both intensity and range images collected with 3D laser imaging devices. DACNN integrates information in intensity and range images to enhance distinct features improving the objects classification in noisy images under various illumination conditions. The standard road condition assessment includes determining not only the type of defects but also the severity of detects. In this regard, a pavement crack segmentation algorithm, CrackSegmenter, is proposed to detect crack at pixel level. The CrackSegmenter leverages residual blocks, attention blocks, Atrous Spatial Pyramid Pooling (ASSP), and squeeze and excitation blocks to improve segmentation performance in pavement crack images

Multi-scale 3D Convolution Network for Video Based Person Re-Identification

This paper proposes a two-stream convolution network to extract spatial and temporal cues for video based person Re-Identification (ReID). A temporal stream in this network is constructed by inserting several Multi-scale 3D (M3D) convolution layers into a 2D CNN network. The resulting M3D convolution network introduces a fraction of parameters into the 2D CNN, but gains the ability of multi-scale temporal feature learning. With this compact architecture, M3D convolution network is also more efficient and easier to optimize than existing 3D convolution networks. The temporal stream further involves Residual Attention Layers (RAL) to refine the temporal features. By jointly learning spatial-temporal attention masks in a residual manner, RAL identifies the discriminative spatial regions and temporal cues. The other stream in our network is implemented with a 2D CNN for spatial feature extraction. The spatial and temporal features from two streams are finally fused for the video based person ReID. Evaluations on three widely used benchmarks datasets, i.e., MARS, PRID2011, and iLIDS-VID demonstrate the substantial advantages of our method over existing 3D convolution networks and state-of-art methods.Comment: AAAI, 201

Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin