A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

Selection of the key earth observation sensors and platforms focusing on applications for Polar Regions in the scope of Copernicus system 2020-2030

An optimal payload selection conducted in the frame of the H2020 ONION project (id 687490) is presented based on the ability to cover the observation needs of the Copernicus system in the time period 2020–2030. Payload selection is constrained by the variables that can be measured, the power consumption, and weight of the instrument, and the required accuracy and spatial resolution (horizontal or vertical). It involved 20 measurements with observation gaps according to the user requirements that were detected in the top 10 use cases in the scope of Copernicus space infrastructure, 9 potential applied technologies, and 39 available commercial platforms. Additional Earth Observation (EO) infrastructures are proposed to reduce measurements gaps, based on a weighting system that assigned high relevance for measurements associated to Marine for Weather Forecast over Polar Regions. This study concludes with a rank and mapping of the potential technologies and the suitable commercial platforms to cover most of the requirements of the top ten use cases, analyzing the Marine for Weather Forecast, Sea Ice Monitoring, Fishing Pressure, and Agriculture and Forestry: Hydric stress as the priority use cases.Peer ReviewedPostprint (published version

Optimal Exploitation of the Sentinel-2 Spectral Capabilities for Crop Leaf Area Index Mapping

The continuously increasing demand of accurate quantitative high quality information on land surface properties will be faced by a new generation of environmental Earth observation (EO) missions. One current example, associated with a high potential to contribute to those demands, is the multi-spectral ESA Sentinel-2 (S2) system. The present study focuses on the evaluation of spectral information content needed for crop leaf area index (LAI) mapping in view of the future sensors. Data from a field campaign were used to determine the optimal spectral sampling from available S2 bands applying inversion of a radiative transfer model (PROSAIL) with look-up table (LUT) and artificial neural network (ANN) approaches. Overall LAI estimation performance of the proposed LUT approach (LUTN₅₀) was comparable in terms of retrieval performances with a tested and approved ANN method. Employing seven- and eight-band combinations, the LUTN₅₀ approach obtained LAI RMSE of 0.53 and normalized LAI RMSE of 0.12, which was comparable to the results of the ANN. However, the LUTN50 method showed a higher robustness and insensitivity to different band settings. Most frequently selected wavebands were located in near infrared and red edge spectral regions. In conclusion, our results emphasize the potential benefits of the Sentinel-2 mission for agricultural applications

Drones: Innovative Technology for Use in Precision Pest Management.

Arthropod pest outbreaks are unpredictable and not uniformly distributed within fields. Early outbreak detection and treatment application are inherent to effective pest management, allowing management decisions to be implemented before pests are well-established and crop losses accrue. Pest monitoring is time-consuming and may be hampered by lack of reliable or cost-effective sampling techniques. Thus, we argue that an important research challenge associated with enhanced sustainability of pest management in modern agriculture is developing and promoting improved crop monitoring procedures. Biotic stress, such as herbivory by arthropod pests, elicits physiological defense responses in plants, leading to changes in leaf reflectance. Advanced imaging technologies can detect such changes, and can, therefore, be used as noninvasive crop monitoring methods. Furthermore, novel methods of treatment precision application are required. Both sensing and actuation technologies can be mounted on equipment moving through fields (e.g., irrigation equipment), on (un)manned driving vehicles, and on small drones. In this review, we focus specifically on use of small unmanned aerial robots, or small drones, in agricultural systems. Acquired and processed canopy reflectance data obtained with sensing drones could potentially be transmitted as a digital map to guide a second type of drone, actuation drones, to deliver solutions to the identified pest hotspots, such as precision releases of natural enemies and/or precision-sprays of pesticides. We emphasize how sustainable pest management in 21st-century agriculture will depend heavily on novel technologies, and how this trend will lead to a growing need for multi-disciplinary research collaborations between agronomists, ecologists, software programmers, and engineers

Estimation of leaf area index and its sunlit portion from DSCOVR EPIC data: theoretical basis

This paper presents the theoretical basis of the algorithm designed for the generation of leaf area index and diurnal course of its sunlit portion from NASA's Earth Polychromatic Imaging Camera (EPIC) onboard NOAA's Deep Space Climate Observatory (DSCOVR). The Look-up-Table (LUT) approach implemented in the MODIS operational LAI/FPAR algorithm is adopted. The LUT, which is the heart of the approach, has been significantly modified. First, its parameterization incorporates the canopy hot spot phenomenon and recent advances in the theory of canopy spectral invariants. This allows more accurate decoupling of the structural and radiometric components of the measured Bidirectional Reflectance Factor (BRF), improves scaling properties of the LUT and consequently simplifies adjustments of the algorithm for data spatial resolution and spectral band compositions. Second, the stochastic radiative transfer equations are used to generate the LUT for all biome types. The equations naturally account for radiative effects of the three-dimensional canopy structure on the BRF and allow for an accurate discrimination between sunlit and shaded leaf areas. Third, the LUT entries are measurable, i.e., they can be independently derived from both below canopy measurements of the transmitted and above canopy measurements of reflected radiation fields. This feature makes possible direct validation of the LUT, facilitates identification of its deficiencies and development of refinements. Analyses of field data on canopy structure and leaf optics collected at 18 sites in the Hyytiälä forest in southern boreal zone in Finland and hyperspectral images acquired by the EO-1 Hyperion sensor support the theoretical basis.Shared Services Center NAS

Remote sensing image fusion on 3D scenarios: A review of applications for agriculture and forestry

Three-dimensional (3D) image mapping of real-world scenarios has a great potential to provide the user with a more accurate scene understanding. This will enable, among others, unsupervised automatic sampling of meaningful material classes from the target area for adaptive semi-supervised deep learning techniques. This path is already being taken by the recent and fast-developing research in computational fields, however, some issues related to computationally expensive processes in the integration of multi-source sensing data remain. Recent studies focused on Earth observation and characterization are enhanced by the proliferation of Unmanned Aerial Vehicles (UAV) and sensors able to capture massive datasets with a high spatial resolution. In this scope, many approaches have been presented for 3D modeling, remote sensing, image processing and mapping, and multi-source data fusion. This survey aims to present a summary of previous work according to the most relevant contributions for the reconstruction and analysis of 3D models of real scenarios using multispectral, thermal and hyperspectral imagery. Surveyed applications are focused on agriculture and forestry since these fields concentrate most applications and are widely studied. Many challenges are currently being overcome by recent methods based on the reconstruction of multi-sensorial 3D scenarios. In parallel, the processing of large image datasets has recently been accelerated by General-Purpose Graphics Processing Unit (GPGPU) approaches that are also summarized in this work. Finally, as a conclusion, some open issues and future research directions are presented.European Commission 1381202-GEU PYC20-RE-005-UJA IEG-2021Junta de Andalucia 1381202-GEU PYC20-RE-005-UJA IEG-2021Instituto de Estudios GiennesesEuropean CommissionSpanish Government UIDB/04033/2020DATI-Digital Agriculture TechnologiesPortuguese Foundation for Science and Technology 1381202-GEU FPU19/0010

Estimating the crop leaf area index using hyperspectral remote sensing

AbstractThe leaf area index (LAI) is an important vegetation parameter, which is used widely in many applications. Remote sensing techniques are known to be effective but inexpensive methods for estimating the LAI of crop canopies. During the last two decades, hyperspectral remote sensing has been employed increasingly for crop LAI estimation, which requires unique technical procedures compared with conventional multispectral data, such as denoising and dimension reduction. Thus, we provide a comprehensive and intensive overview of crop LAI estimation based on hyperspectral remote sensing techniques. First, we compare hyperspectral data and multispectral data by highlighting their potential and limitations in LAI estimation. Second, we categorize the approaches used for crop LAI estimation based on hyperspectral data into three types: approaches based on statistical models, physical models (i.e., canopy reflectance models), and hybrid inversions. We summarize and evaluate the theoretical basis and different methods employed by these approaches (e.g., the characteristic parameters of LAI, regression methods for constructing statistical predictive models, commonly applied physical models, and inversion strategies for physical models). Thus, numerous models and inversion strategies are organized in a clear conceptual framework. Moreover, we highlight the technical difficulties that may hinder crop LAI estimation, such as the “curse of dimensionality” and the ill-posed problem. Finally, we discuss the prospects for future research based on the previous studies described in this review