Multitemporal segmentation of Sentinel-2 images in an agricultural intensification region in Brazil.

ABSTRACT: With the recent evolution in the sensor's spatial resolution, such as the MultiSpectral Imager (MSI) of the Sentinel-2 mission, the need to use segmentation techniques in satellite images has increased. Although the advantages of image segmentation to delineate agricultural fields in images are already known, the literature shows that it is rarely used to consider temporal changes in highly managed regions with the intensification of agricultural activities. Therefore, this work aimed to evaluate a multitemporal segmentation method based on the coefficient of variation of spectral bands and vegetation indices obtained from Sentinel-2 images, considering two agricultural years (2018-2019 and 2019-2020) in an area with agricultural intensification. Images of the coefficient of variation represented the spectro-temporal dynamics within the study area. These images were also used to apply an edge detection filter (Sobel) to verify their performance. The region-based algorithm Watershed Segmentation (WS) was used in the segmentation process. Subsequently, to assess the quality of the segmentation results produced, the metrics Potential Segmentation Error (PSE), Number-of-Segments Ratio (NSR), and Euclidean Distance 2 (ED2) were calculated from manually delineated reference objects. The segmentation achieved its best performance when applied to the unfiltered coefficient of variation images of spectral bands with an ED2 equal to 7.289 and 2.529 for 2018-2019 and 2019-2020, respectively. There was a tendency for the WS algorithm to produce over-segmentation in the study area; however, its use proved to be effective in identifying objects in a dynamic area with the intensification of agricultural activities.Edition of proceedings of the 2022 edition of the XXIVth ISPRS Congress, Nice, France

Mapping smallholder cashew plantations to inform sustainable tree crop expansion in Benin

Cashews are grown by over 3 million smallholders in more than 40 countries worldwide as a principal source of income. As the third largest cashew producer in Africa, Benin has nearly 200,000 smallholder cashew growers contributing 15% of the country's national export earnings. However, a lack of information on where and how cashew trees grow across the country hinders decision-making that could support increased cashew production and poverty alleviation. By leveraging 2.4-m Planet Basemaps and 0.5-m aerial imagery, newly developed deep learning algorithms, and large-scale ground truth datasets, we successfully produced the first national map of cashew in Benin and characterized the expansion of cashew plantations between 2015 and 2021. In particular, we developed a SpatioTemporal Classification with Attention (STCA) model to map the distribution of cashew plantations, which can fully capture texture information from discriminative time steps during a growing season. We further developed a Clustering Augmented Self-supervised Temporal Classification (CASTC) model to distinguish high-density versus low-density cashew plantations by automatic feature extraction and optimized clustering. Results show that the STCA model has an overall accuracy over 85% and the CASTC model achieved an overall accuracy of 76%. We found that the cashew area in Benin almost doubled from 2015 to 2021 with 60% of new plantation development coming from cropland or fallow land, while encroachment of cashew plantations into protected areas has increased by 55%. Only half of cashew plantations were high-density in 2021, suggesting high potential for intensification. Our study illustrates the power of combining high-resolution remote sensing imagery and state-of-the-art deep learning algorithms to better understand tree crops in the heterogeneous smallholder landscape

Geotechnical Characterisation of Coal Spoil Piles Using High-Resolution Optical and Multispectral Data: A Machine Learning Approach

Geotechnical characterisation of spoil piles has traditionally relied on the expertise of field specialists, which can be both hazardous and time-consuming. Although unmanned aerial vehicles (UAV) show promise as a remote sensing tool in various applications; accurately segmenting and classifying very high-resolution remote sensing images of heterogeneous terrains, such as mining spoil piles with irregular morphologies, presents significant challenges. The proposed method adopts a robust approach that combines morphology-based segmentation, as well as spectral, textural, structural, and statistical feature extraction techniques to overcome the difficulties associated with spoil pile characterisation. Additionally, it incorporates minimum redundancy maximum relevance (mRMR) based feature selection and machine learning-based classification. This automated characterisation will serve as a proactive tool for dump stability assessment, providing crucial data for improved stability models and contributing to a greener and more responsible mining industry

Image-based time series representations for satellite images classification

Orientador: Ricardo da Silva TorresTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: A classificação de imagens de sensoriamento remoto por pixel com base no perfil temporal desempenha um papel importante em várias aplicações, tais como: reconhecimento de culturas, estudos fenológicos e monitoramento de mudanças na cobertura do solo. Avanços sensores captura de imagem aumentaram a necessidade de criação de metodologias para analisar o perfil temporal das informações coletadas. Nós investigamos dados coletados em dois tipos de sensores: (i) sensores em plataformas orbitais, esse tipo de imagem sofre interferências de nuvens e fatores atmosféricos; e (ii) sensores fixados em campo, mais especificamente, uma câmera digital no alto de uma torre, cujas imagens capturadas podem conter dezenas de espécies, dificultando a identificação de padrões de interesse. Devido às particularidades dos dados detectados remotamente, torna-se custoso enviar a imagem capturada pelo sensor diretamente para métodos de aprendizado de máquina sem realizar um pré-processamento. Para algumas aplicações de sensoriamento remoto, comumente não se utiliza as imagens brutas oriundas dos sensores, mas os índices de vegetação extraídos das regiões de interesse ao longo do tempo. Assim, o perfil temporal pode ser caracterizado como uma série de observações dos índices de vegetação dos pixels de interesse. Métodos baseados em aprendizado profundo obtiveram bons resultados em aplicações de sensoriamento remoto relacionadas à classificação de imagens. Contudo, em consequência da natureza dos dados, nem sempre é possível realizar o treinamento adequado das redes de aprendizado profundo pela limitação causada por dados faltantes. Entretanto, podemos nos beneficiar de redes previamente treinadas para detecção de objetos para extrair características e padrões de imagens. O problema alvo deste trabalho é classificar séries temporais extraídas de imagens de sensoriamento remoto representando as características temporais como imagens 2D. Este trabalho investiga abordagens que codificam séries temporais como representação de imagem para propor metodologias de classificação binária e multiclasse no contexto de sensoriamento remoto, se beneficiando de redes extratoras de características profundas. Os experimentos conduzidos para classificação binária foram realizados em dados de satélite para identificar plantações de eucalipto. Os resultados superaram métodos baseline propostos recentemente. Os experimentos realizados para classificação multiclasse focaram em imagens capturadas com câmera digital para detectar o padrão fenológico de regiões de interesse. Os resultados mostram que a acurácia aumenta se consideramos conjuntos de pixelsAbstract: Pixelwise remote sensing image classification based on temporal profile plays an important role in several applications, such as crop recognition, phenological studies, and land cover change monitoring. Advances in image capture sensors have increased the need for methodologies to analyze the temporal profile of collected information. We investigate data collected by two types of sensors: (i) sensors on orbital platforms, this type of image suffers from cloud interference and atmospheric factors; and (ii) field-mounted sensors, in particular, a digital camera on top of a tower, where captured images may contain dozens of species, making it difficult to identify patterns of interest. Due to the particularities of remotely detected data, it is prohibitive to send sensor captured images directly to machine learning methods without preprocessing. In some remote sensing applications, it is not commonly used the raw images from the sensors, but the vegetation indices extracted from regions of interest over time. Thus, the temporal profile can be characterized as a series of observations of vegetative indices of pixels of interest. Deep learning methods have been successfully in remote sensing applications related to image classification. However, due to the nature of the data, it is not always possible to properly train deep learning networks because of the lack of enough labeled data. However, we can benefit from previously trained 2D object detection networks to extract features and patterns from images. The target problem of this work is to classify remote sensing images, based on pixel time series represented as 2D representations. This work investigates approaches that encode time series into image representations to propose binary and multiclass classification methodologies in the context of remote sensing, taking advantage of data-driven feature extractor approaches. The experiments conducted for binary classification were performed on satellite data to identify eucalyptus plantations. The results surpassed the ones of recently proposed baseline methods. The experiments performed for multiclass classification focused on detecting regions of interest within images captured by a digital camera. The results show that the accuracy increases if we consider a set of pixelsDoutoradoCiência da ComputaçãoDoutora em Ciência da ComputaçãoCAPE

Mapping of multitemporal rice (Oryza sativa L.) growth stages using remote sensing with multi-sensor and machine learning : a thesis dissertation presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Earth Science at Massey University, Manawatū, New Zealand

Figure 2.1 is adapted and re-used under a Creative Commons Attribution 4.0 International (CC BY 4.0) license.Rice (Oryza Sativa) plays a pivotal role in food security for Asian countries, especially in Indonesia. Due to the increasing pressure of environmental changes, such as land use and climate, rice cultivation areas need to be monitored regularly and spatially to ensure sustainable rice production. Moreover, timely information of rice growth stages (RGS) can lead to more efficient of inputs distribution from water, seed, fertilizer, and pesticide. One of the efficient solutions for regularly mapping the rice crop is using Earth observation satellites. Moreover, the increasing availability of open access satellite images such as Landsat-8, Sentinel-1, and Sentinel-2 provides ample opportunities to map continuous and high-resolution rice growth stages with greater accuracy. The majority of the literature has focused on mapping rice area, cropping patterns and relied mainly on the phenology of vegetation. However, the mapping process of RGS was difficult to assess the accuracy, time-consuming, and depended on only one sensor. In this work, we discuss the use of machine learning algorithms (MLA) for mapping paddy RGS with multiple remote sensing data in near-real-time. The study area was Java Island, which is the primary rice producer in Indonesia. This study has investigated: (1) the mapping of RGS using Landsat-8 imagery and different MLAs, and their rigorous performance was evaluated by conducting a multitemporal analysis; (2) the temporal consistency of predicting RGS using Sentinel-2, MOD13Q1, and Sentinel-1 data; (3) evaluating the correlation of local statistics data and paddy RGS using Sentinel-2, PROBA-V, and Sentinel-1 with MLAs. The ground truth datasets were collected from multi-year web camera data (2014-2016) and three months of the field campaign in different regions of Java (2018). The study considered the RGS in the analysis to be vegetative, reproductive, ripening, bare land, and flooding, and MLAs such as support vector machines (SVMs), random forest (RF), and artificial neural network (ANN) were used. The temporal consistency matrix was used to compare the classification maps within three sensor datasets (Landsat-8 OLI, Sentinel-2, and Sentinel-2, MOD13Q1, Sentinel-1) and in four periods (5, 10, 15, 16 days). Moreover, the result of the RGS map was also compared with monthly data from local statistics within each sub-district using cross-correlation analysis. The result from the analysis shows that SVM with a radial base function outperformed the RF and ANN and proved to be a robust method for small-size datasets (< 1,000 points). Compared to Sentinel-2, Landsat-8 OLI gives less accuracy due to the lack of a red-edge band and larger pixel size (30 x 30 m). Integration of Sentinel-2, MOD13Q1, and Sentinel-1 improved the classification performance and increased the temporal availability of cloud-free maps. The integration of PROBA-V and Sentinel-1 improved the classification accuracy from the Landsat-8 result, consistent with the monthly rice planting area statistics at the sub-district level. The western area of Java has the highest accuracy and consistency since the cropping pattern only relied on rice cultivation. In contrast, less accuracy was noticed in the eastern area because of upland rice cultivation due to limited irrigation facilities and mixed cropping. In addition, the cultivation of shallots to the north of Nganjuk Regency interferes with the model predictions because the cultivation of shallots resembles the vegetative phase due to the water banks. One future research idea is the auto-detection of the cropping index in the complex landscape to be able to use it for mapping RGS on a global scale. Detection of the rice area and RGS using Google Earth Engine (GEE) can be an action plan to disseminate the information quickly on a planetary scale. Our results show that the multitemporal Sentinel-1 combined with RF can detect rice areas with high accuracy (>91%). Similarly, accurate RGS maps can be detected by integrating multiple remote sensing (Sentinel-2, Landsat-8 OLI, and MOD13Q1) data with acceptable accuracy (76.4%), with high temporal frequency and lower cloud interference (every 16 days). Overall, this study shows that remote sensing combined with the machine learning methodology can deliver information on RGS in a timely fashion, which is easy to scale up and consistent both in time and space and matches the local statistics. This thesis is also in line with the existing rice monitoring projects such as Crop Monitor, Crop Watch, AMIS, and Sen4Agri to support disseminating information over a large area. To sum up, the proposed workflow and detailed map provide a more accurate method and information in near real-time for stakeholders, such as governmental agencies against the existing mapping method. This method can be introduced to provide accurate information to rice farmers promptly with sufficient inputs such as irrigation, seeds, and fertilisers for ensuring national food security from the shifting planting time due to climate change