A systematic review of the use of Deep Learning in Satellite Imagery for Agriculture

Agricultural research is essential for increasing food production to meet the requirements of an increasing population in the coming decades. Recently, satellite technology has been improving rapidly and deep learning has seen much success in generic computer vision tasks and many application areas which presents an important opportunity to improve analysis of agricultural land. Here we present a systematic review of 150 studies to find the current uses of deep learning on satellite imagery for agricultural research. Although we identify 5 categories of agricultural monitoring tasks, the majority of the research interest is in crop segmentation and yield prediction. We found that, when used, modern deep learning methods consistently outperformed traditional machine learning across most tasks; the only exception was that Long Short-Term Memory (LSTM) Recurrent Neural Networks did not consistently outperform Random Forests (RF) for yield prediction. The reviewed studies have largely adopted methodologies from generic computer vision, except for one major omission: benchmark datasets are not utilised to evaluate models across studies, making it difficult to compare results. Additionally, some studies have specifically utilised the extra spectral resolution available in satellite imagery, but other divergent properties of satellite images - such as the hugely different scales of spatial patterns - are not being taken advantage of in the reviewed studies.Comment: 25 pages, 2 figures and lots of large tables. Supplementary materials section included here in main pd

Rice crop detection using LSTM, Bi-LSTM, and machine learning models from sentinel-1 time series

The Synthetic Aperture Radar (SAR) time series allows describing the rice phenological cycle by the backscattering time signature. Therefore, the advent of the Copernicus Sentinel-1 program expands studies of radar data (C-band) for rice monitoring at regional scales, due to the high temporal resolution and free data distribution. Recurrent Neural Network (RNN) model has reached state-of-the-art in the pattern recognition of time-sequenced data, obtaining a significant advantage at crop classification on the remote sensing images. One of the most used approaches in the RNN model is the Long Short-Term Memory (LSTM) model and its improvements, such as Bidirectional LSTM (Bi-LSTM). Bi-LSTM models are more effective as their output depends on the previous and the next segment, in contrast to the unidirectional LSTM models. The present research aims to map rice crops from Sentinel-1 time series (band C) using LSTM and Bi-LSTM models in West Rio Grande do Sul (Brazil). We compared the results with traditional Machine Learning techniques: Support Vector Machines (SVM), Random Forest (RF), k-Nearest Neighbors (k-NN), and Normal Bayes (NB). The developed methodology can be subdivided into the following steps: (a) acquisition of the Sentinel time series over two years; (b) data pre-processing and minimizing noise from 3D spatial-temporal filters and smoothing with Savitzky-Golay filter; (c) time series classification procedures; (d) accuracy analysis and comparison among the methods. The results show high overall accuracy and Kappa (>97% for all methods and metrics). Bi-LSTM was the best model, presenting statistical differences in the McNemar test with a significance of 0.05. However, LSTM and Traditional Machine Learning models also achieved high accuracy values. The study establishes an adequate methodology for mapping the rice crops in West Rio Grande do Sul

M3Fusion: A Deep Learning Architecture for Multi-{Scale/Modal/Temporal} satellite data fusion

Modern Earth Observation systems provide sensing data at different temporal and spatial resolutions. Among optical sensors, today the Sentinel-2 program supplies high-resolution temporal (every 5 days) and high spatial resolution (10m) images that can be useful to monitor land cover dynamics. On the other hand, Very High Spatial Resolution images (VHSR) are still an essential tool to figure out land cover mapping characterized by fine spatial patterns. Understand how to efficiently leverage these complementary sources of information together to deal with land cover mapping is still challenging. With the aim to tackle land cover mapping through the fusion of multi-temporal High Spatial Resolution and Very High Spatial Resolution satellite images, we propose an End-to-End Deep Learning framework, named M3Fusion, able to leverage simultaneously the temporal knowledge contained in time series data as well as the fine spatial information available in VHSR information. Experiments carried out on the Reunion Island study area asses the quality of our proposal considering both quantitative and qualitative aspects

Análise de inundações e classificação da cobertura vegetal no bioma amazônico usando séries temporais sentinel-1 SAR e técnicas de deep learning

Tese (doutorado) — Universidade de Brasília, Instituto de Ciências Humanas, Departamento de Geografia, Programa de Pós-Graduação em Geografia, 2022.Os recursos hídricos e os estudos fenológicos florestais são extremamente importantes para a compreensão de diversos fenômenos naturais como as mudanças climáticas, dinâmica hidrogeomorfológica, condicionamento ambiental e gestão dos recursos. Inserida na dinâmica hídrica, estão presentes as áres inundáveis que estão intrinsecamente ligadas à manuntenção da biota e da fauna nos biomas brasileiros. Nesse contexto, os produtos derivados de sensoriamento remoto têm sido bastante utilizados para a análise e monitoramento de áreas inundáveis, mapeamento de uso e ocupação da terra e dinâmica fenológica devido à sua importância ambiental. As imagens de radar de abertura sintética (SAR) são produtos potenciais por não apresentar interferências atmosféricas, entretanto, necessitam de diversos tratamentos iniciais, definidos de pré-processamento para assim ser possível obter uma melhor extração de informações de uma determinada área. Nesse sentido, essa pesquisa teve como objetivo aplicar as técnicas de deep learning utilizando algoritmos de processamento de séries temporais de imagens de satélite baseados em redes neurais para extração e identificação de áreas inundáveis, corpos hídricos e fenologias florestais em áreas de cerrado, floresta amazônica, mangues, cultivos agrícolas e várzea. O presente estudo foi dividido em três capítulos principais: (a) análises métricas e estatísticas de filtragens espaciais em imagem Sentinel-1 SAR da Amazônia Central, Brasil; (b) análise de série temporal Sentinel-1 SAR em inundações na Amazônia Central; e (c) classificação fenológica de floresta, mangues, cerrado e vegetação alagada do bioma Amazônia por meio de comparação dos modelos LSTM, Bi-LSTM, GRU, Bi-GRU e modelos de aprendizagem de máquina baseados em séries temporais do satélite Sentinel-1. As etapas metodológicas foram distintas para cada capítulo e todos apresentaram precisão e altos valores métricos para mensuração e análise dos corpos hídricos, inundação e fenologias florestais. Dentre os métodos de filtragem analisados na imagem SAR, o filtro Lee com janela 3 × 3 apresentou os melhores desempenhos na redução do ruído speckle (MSE igual a 1,88 e MAE igual a 1,638) e baixo valor de distorção de contraste na polarização VH. Entretanto, para a polarização VV, mensuraram-se diferentes resultados para análise da redução do ruído speckle, onde o filtro Frost com janela 3 × 3 apresentou o melhor desempenho, com baixo valor para as métricas em geral (MSE igual a 1,2 e MAE igual a 6,28) e também um baixo valor de distorção de contraste. Por apresentar os melhores valores estatísticos, o filtro de mediana com janela 11 × 11 nas polarizações VH e VV pode ser utilizado como uma técnica de filtragem alternativa na imagem Sentinel-1 nas duas polarizações. As áreas de inundação mensuradas nas polarizações VH e VV apresentaram uma forte correlação e sem significância estatística entre as amostras, presumindo que se pode utilizar as duas polarizações para obtenção do pulso de inundação e mapeamento da dinâmica das áreas inundáveis na região. Pelo fato de não haver imagens Sentinel-1 anteriores ao ano de 2016, quando os eventos extremos de LMEO foram superiores a 100%, não foi possível delimitar a LMEO por meio de dados SAR. Algumas áreas ao longo da costa e rios apresentam assinaturas temporais de retroespalhamento que evidenciam transições entre ambientes terrestres e áreas cobertas por água. A variação temporal do retroespalhamento de valores mais altos para mais baixos indica erosão e inundação progressiva, enquanto o inverso indica aumento terrestre. O modelo Bi-GRU apresentou a maior acurácia geral, precisão, recall e F-score tanto na polarização individual como na polarização combinada VV+VH. A combinação entre as polarizações forneceu os melhores resultados na classificação e a polarização VH obteve melhores resultados quando comparado à polarização VV. O presente estudo atestou o procedimento metodológico adequado para mensurar as áreas de corpos hídricos e seu pulso de inundação como também obteve a classificação de fenologias com alta precisão na Amazônia Central por meio de deep learning advindas de série temporal de imagens Sentinel-1 SAR.Water resources and forest phenological studies are extremely important for the understanding of various natural phenomena, such as climate variation, hydrogeomorphological dynamics, environmental conditioning, and resource management. In this context, products derived from remote sensing have been widely used for the analysis and monitoring of flooding areas, land use and occupation mapping, and phenological dynamics due to their environmental importance. Synthetic aperture radar (SAR) images are potential products as they do not present atmospheric interference, however, they require several initial treatments, defined as pre-processing, so that it is possible to obtain a better extraction of information from a certain area. In this sense, this research aimed to apply deep learning techniques using algorithms based on neural networks for the extraction and identification of flooding areas, water bodies, and forest phenologies such as cerrado, Amazon forest, mangroves, agricultural crops, and floodplain through time series of remote sensing images. This study was divided into three main chapters: (a) metric and statistical analysis of spatial filtering in Sentinel-1 SAR images of Central Amazonia, Brazil; (b) Sentinel-1 SAR time series analysis in flooding areas of Central Amazon; and (c) phenological classification of forest, mangroves, savannas, and two flooded vegetation of the Amazon biome by comparing LSTM, Bi-LSTM, GRU, Bi-GRU, and machine learning models from Sentinel-1 time series. The methodological steps were different for each chapter and all presented precision and high metric values for measurement and analysis of water bodies, flooding and forest phenologies. Among the filtering methods analyzed in the SAR image, the Lee filter with 3 × 3 window presented the best performance in reducing speckle noise (MSE of 1.88 and MAE of 1.638) and low value of contrast distortion in the VH polarization. However, for the VV polarization, different results were measured for the analysis of the sepeckle noise reduction, where the Frost filter with 3 × 3 window presented the best performance, with a low value for the metrics in general (MSE of 1.2 and MAE of 6.28) and also a low contrast distortion value. Statistical values derived from the median filter with 11 × 11 window in the VH and VV polarizations can be used as an alternative filtering technique in the Sentinel-1 SAR image in both polarizations. The flooding areas measured in the VH and VV polarizations showed a strong correlation and no statistical significance between the samples, assuming that both polarizations can be used to obtain the flood pulse and mapping the dynamics of the flooded areas in the region. Because there are no Sentinel1 SAR images prior to 2016 when extreme LMEO events were greater than 100%, it was not possible to delimit the LMEO through SAR data. Some areas along the coast and rivers show temporal backscatter signatures with transitions between terrestrial environments and areas covered by water. The temporal variation of backscatter from higher to lower values indicates erosion and progressive flooding, while the inverse indicates terrestrial increase. The Bi-GRU model showed the highest overall accuracy, precision, recall, and F-score in both separate polarization and combined VV+VH polarization. The combination between the polarizations provided the best results in the classification and the VH polarization obtained better results when compared to the VV polarization. This study attested an adequate methodological procedure to measure the areas of water bodies and their flood pulse, as well as obtaining the classification of phenologies with high precision in the Central Amazon by means of deep learning applied to the time series of Sentinel-1 SAR images

Using LUCAS survey and Recurrent Neural Networks to produce LCLU classification based on a Satellite Image time series of Sentinel-2

Dissertation presented as the partial requirement for obtaining a Master's degree in Information Management, specialization in Knowledge Management and Business IntelligenceThe need of timely and accurate information for the territory has increased over the years, making Land Cover Land Use (LCLU) mapping one of the most common application of remote sensing. Recently, the advances in satellite technology and the open access policies for remote sensing data increased the interest in exploring satellite image time series. In addition, the attention of researchers has shifted from standard machine learning algorithms (e.g., Support Vector Machines and Random Forest) to Recurrent Neural Networks due to their ability of exploiting sequential information. However, acquiring reference data to train these algorithms is still a hurdle. This study aims to evaluate the capability of a Gated Recurrent Unit in performing pixel-level LCLU classification of a satellite image time series, using Sentinel-2 imagery and having the LUCAS survey as reference data. To assess the performance of our model we compared it to state-of-the-art classifiers (SVM and RF). Due to the unbalance nature of the LUCAS survey, we applied oversampling to this dataset to increase the performance of our models, testing three different oversampling techniques. The results attained showed that Recurrent Neural Networks did not outperform the other state-of-the-art algorithms, when trained with a limited number of sampling units, and that oversampling the LUCAS survey increased the performance of all the classifiers. Finally, we were able to demonstrate that it is possible to produce LCLU classification of satellite image time series using only open-source data by using Sentinel-2 imagery and the LUCAS survey as refence data

Advancements in Multi-temporal Remote Sensing Data Analysis Techniques for Precision Agriculture

L'abstract è presente nell'allegato / the abstract is in the attachmen