A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Neural Architecture Search for Image Segmentation and Classification

Deep learning (DL) is a class of machine learning algorithms that relies on deep neural networks (DNNs) for computations. Unlike traditional machine learning algorithms, DL can learn from raw data directly and effectively. Hence, DL has been successfully applied to tackle many real-world problems. When applying DL to a given problem, the primary task is designing the optimum DNN. This task relies heavily on human expertise, is time-consuming, and requires many trial-and-error experiments. This thesis aims to automate the laborious task of designing the optimum DNN by exploring the neural architecture search (NAS) approach. Here, we propose two new NAS algorithms for two real-world problems: pedestrian lane detection for assistive navigation and hyperspectral image segmentation for biosecurity scanning. Additionally, we also introduce a new dataset-agnostic predictor of neural network performance, which can be used to speed-up NAS algorithms that require the evaluation of candidate DNNs

Proceedings of the 2021 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory

2021, the annual joint workshop of the Fraunhofer IOSB and KIT IES was hosted at the IOSB in Karlsruhe. For a week from the 2nd to the 6th July the doctoral students extensive reports on the status of their research. The results and ideas presented at the workshop are collected in this book in the form of detailed technical reports

Proceedings of the 2021 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory

2021, the annual joint workshop of the Fraunhofer IOSB and KIT IES was hosted at the IOSB in Karlsruhe. For a week from the 2nd to the 6th July the doctoral students extensive reports on the status of their research. The results and ideas presented at the workshop are collected in this book in the form of detailed technical reports

WeAbDeepCNN: Weighted Average Model and ASSCA based Two Level Fusion Scheme For Multi-Focus Images

Fusion of images is a strategy that merges various moderately focused images or non-focused images of a single scene to generate a fully focused, clear and sharp image. The goal of this research is to discover the focused regions and further combination of focused regions of different source images into solitary image. However, there exist several issues in image fusion that involves contrast reduction, block artifacts, and artificial edges. To solve this issue, a two level fusion scheme has been devised, which involves weighted average model along with Atom Search Sine Cosine algorithm-based Deep Convolutional Neural Network (ASSCA-based Deep CNN) and may be abbreviated as “WeAbDeepCNN” i.e. weighted average model and ASSCA based Deep CNN. In the study two images are fed to initial fusion module, which is performed using weighted average model. The fusion score are generated whose values are determined in an optimal manner. Thus, final fusion is performed using proposed ASSCA-based Deep CNN. The Deep CNN training is carried out with proposed ASSCA, which is devised by combining Sine Cosine Algorithm, abbreviated as SCA, as well as atom search optimization (ASO). The proposed ASSCA-based Deep CNN offers improved performance in contrast to current state of the art techniques with a highest value 1.52 of mutual information (MI), with a highest value of 32.55 dB of maximum Peak Signal to Noise Ratio i.e. PSNR as well as  value of 7.59 of Minimum Root Mean Square Error (RMSE)

WeAbDeepCNN: Weighted Average Model and ASSCA based Two Level Fusion Scheme For Multi-Focus Images

905-914Fusion of images is a strategy that merges various moderately focused images or non-focused images of a single scene to generate a fully focused, clear and sharp image. The goal of this research is to discover the focused regions and further combination of focused regions of different source images into solitary image. However, there exist several issues in image fusion that involves contrast reduction, block artifacts, and artificial edges. To solve this issue, a two level fusion scheme has been devised, which involves weighted average model along with Atom Search Sine Cosine algorithm-based Deep Convolutional Neural Network (ASSCA-based Deep CNN) and may be abbreviated as “WeAbDeepCNN” i.e. weighted average model and ASSCA based Deep CNN. In the study two images are fed to initial fusion module, which is performed using weighted average model. The fusion score are generated whose values are determined in an optimal manner. Thus, final fusion is performed using proposed ASSCA-based Deep CNN. The Deep CNN training is carried out with proposed ASSCA, which is devised by combining Sine Cosine Algorithm, abbreviated as SCA, as well as atom search optimization (ASO). The proposed ASSCA-based Deep CNN offers improved performance in contrast to current state of the art techniques with a highest value 1.52 of mutual information (MI), with a highest value of 32.55 dB of maximum Peak Signal to Noise Ratio i.e. PSNR as well as value of 7.59 of Minimum Root Mean Square Error (RMSE)

Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks

International audienceThis work investigates the use of deep fully convolutional neural networks (DFCNN) for pixel-wise scene labeling of Earth Observation images. Especially, we train a variant of the SegNet architecture on remote sensing data over an urban area and study different strategies for performing accurate semantic segmentation. Our contributions are the following: 1) we transfer efficiently a DFCNN from generic everyday images to remote sensing images; 2) we introduce a multi-kernel convolutional layer for fast aggregation of predictions at multiple scales; 3) we perform data fusion from heterogeneous sensors (optical and laser) using residual correction. Our framework improves state-of-the-art accuracy on the ISPRS Vaihingen 2D Semantic Labeling dataset

Advances in Computational Intelligence Applications in the Mining Industry

This book captures advancements in the applications of computational intelligence (artificial intelligence, machine learning, etc.) to problems in the mineral and mining industries. The papers present the state of the art in four broad categories: mine operations, mine planning, mine safety, and advances in the sciences, primarily in image processing applications. Authors in the book include both researchers and industry practitioners