Object-based classification of grasslands from high resolution satellite image time series using gaussian mean map kernels

This paper deals with the classification of grasslands using high resolution satellite image time series. Grasslands considered in this work are semi-natural elements in fragmented landscapes, i.e., they are heterogeneous and small elements. The first contribution of this study is to account for grassland heterogeneity while working at the object level by modeling its pixels distributions by a Gaussian distribution. To measure the similarity between two grasslands, a new kernel is proposed as a second contribution: the a-Gaussian mean kernel. It allows one to weight the influence of the covariance matrix when comparing two Gaussian distributions. This kernel is introduced in support vector machines for the supervised classification of grasslands from southwest France. A dense intra-annual multispectral time series of the Formosat-2 satellite is used for the classification of grasslands’ management practices, while an inter-annual NDVI time series of Formosat-2 is used for old and young grasslands’ discrimination. Results are compared to other existing pixel- and object-based approaches in terms of classification accuracy and processing time. The proposed method is shown to be a good compromise between processing speed and classification accuracy. It can adapt to the classification constraints, and it encompasses several similarity measures known in the literature. It is appropriate for the classification of small and heterogeneous objects such as grasslands

Suivi écologique des prairies semi-naturelles : analyse statistique de séries temporelles denses d'images satellite à haute résolution spatiale

ID ProdINRA 415874Grasslands are a significant source of biodiversity in farmed landscapes that is important to monitor. New generation satellites such as Sentinel-2 offer new opportunities for grassland’s monitoring thanks to their combined high spatial and temporal resolutions. Conversely, the new type of data provided by these sensors involves big data and high dimensional issues because of the increasing number of pixels to process and the large number of spectro-temporal variables. This thesis explores the potential of the new generation satellites to monitor biodiversity and factors that influence biodiversity in semi-natural grasslands. Tools suitable for the statistical analysis of grasslands using dense satellite image time series (SITS) with high spatial resolution are provided. First, we show that the spectro-temporal response of grasslands is characterized by its variability within and among the grasslands. Then, for the statistical analysis, grasslands are modeled at the object level to be consistent with ecological models that represent grasslands at the field scale. We propose to model the distribution of pixels in a grassland by a Gaussian distribution. Following this modeling, similarity measures between two Gaussian distributions robust to the high dimension are developed for the classification of grasslands using dense SITS: the High-Dimensional Kullback-Leibler Divergence and the α-Gaussian Mean Kernel. The latter out-performs conventional methods used with Support Vector Machines for the classification of grasslands according to their management practices and to their age. Finally, indicators of grassland biodiversity issued from dense SITS are proposed through spectro-temporal heterogeneity measures derived from the unsupervised clustering of grasslands. Their correlation with the Shannon index is significant but low. The results suggest that the spectro-temporal variations measured from SITS at a spatial resolution of 10 meters covering the period when the practices occur are more related to the intensity of management practices than to the species diversity. Therefore, although the spatial and spectral properties of Sentinel-2 seem limited to assess the species diversity in grasslands directly, this satellite should make possible the continuous monitoring of factors influencing biodiversity in grasslands. In this thesis, we provided methods that account for the heterogeneity within grasslands and enable the use of all the spectral andtemporal information provided by new generation satellites.Les prairies représentent une source importante de biodiversité dans les paysages agricoles qu’il est important de surveiller. Les satellites de nouvelle génération tels que Sentinel-2 offrent de nouvelles opportunités pour le suivi des prairies grâce à leurs hautes résolutions spatiale et temporelle combinées. Cependant, le nouveau type de données fourni par ces satellites implique des problèmes liés au big data et à la grande dimension des données en raison du nombre croissant de pixels à traiter et du nombre élevé de variables spectro-temporelles. Cette thèse explore le potentiel des satellites de nouvelle génération pour le suivi de la biodiversité et des facteurs qui influencent la biodiversité dans les prairies semi-naturelles. Des outils adaptés à l’analyse statistique des prairies à partir de séries temporelles d’images satellites (STIS) denses à haute résolution spatiale sont proposés. Tout d’abord, nous montrons que la réponse spectro-temporelle des prairies est caractérisée par sa variabilité au sein des prairies et parmi les prairies. Puis, pour les analyses statistiques, les prairies sont modélisées à l’échelle de l’objet pour être cohérent avec les modèles écologiques qui représentent les prairies à l’échelle de la parcelle. Nous proposons de modéliser la distribution des pixels dans une prairie par une loi gaussienne. A partir de cette modélisation, des mesures de similarité entre deux lois gaussiennes robustes à la grande dimension sont développées pour la classification des prairies en utilisant des STIS denses: High-Dimensional Kullback-Leibler Divergence et α-Gaussian Mean Kernel. Cette dernière est plus performante que les méthodes conventionnelles utilisées avec les machines à vecteur de support (SVM) pour la classification du mode de gestion et de l’âge des prairies. Enfin, des indicateurs de biodiversité des prairies issus de STIS denses sont proposés à travers des mesures d’hétérogénéité spectro-temporelle dérivées du clustering non supervisé des prairies. Leur corrélation avec l’indice de Shannon est significative mais faible. Les résultats suggèrent que les variations spectro-temporelles mesurées à partir de STIS à 10 mètres de résolution spatiale et qui couvrent la période où ont lieu les pratiques agricoles sont plus liées à l’intensité des pratiques qu’à la diversité en espèces. Ainsi, bien que les propriétés spatiales et temporelles de Sentinel-2 semblent limitées pour estimer directement la diversité en espèces des prairies, ce satellite devrait permettre le suivi continu des facteurs influençant la biodiversité dans les prairies. Dans cette thèse, nous avons proposé des méthodes qui prennent en compte l’hétérogénéité au sein des prairies et qui permettent l’utilisation de toute l’information spectrale et temporelle fournie par les satellites de nouvelle génération

Ecological monitoring of semi-natural grasslands : statistical analysis of dense satellite image time series with high spatial resolution

Grasslands are a significant source of biodiversity in farmed landscapes that is important to monitor. New generation satellites such as Sentinel-2 offer new opportunities for grassland’s monitoring thanks to their combined high spatial and temporal resolutions. Conversely, the new type of data provided by these sensors involves big data and high dimensional issues because of the increasing number of pixels to process and the large number of spectro-temporal variables. This thesis explores the potential of the new generation satellites to monitor biodiversity and factors that influence biodiversity in semi-natural grasslands. Tools suitable for the statistical analysis of grasslands using dense satellite image time series (SITS) with high spatial resolution are provided. First, we show that the spectro-temporal response of grasslands is characterized by its variability within and among the grasslands. Then, for the statistical analysis, grasslands are modeled at the object level to be consistent with ecological models that represent grasslands at the field scale. We propose to model the distribution of pixels in a grassland by a Gaussian distribution. Following this modeling, similarity measures between two Gaussian distributions robust to the high dimension are developed for the lassification of grasslands using dense SITS: the High-Dimensional Kullback-Leibler Divergence and the -Gaussian Mean Kernel. The latter outperforms conventional methods used with Support Vector Machines for the classification of grasslands according to their management practices and to their age. Finally, indicators of grassland biodiversity issued from dense SITS are proposed through spectro-temporal heterogeneity measures derived from the unsupervised clustering of grasslands. Their correlation with the Shannon index is significant but low. The results suggest that the spectro-temporal variations measured from SITS at a spatial resolution of 10 meters covering the period when the practices occur are more related to the intensity of management practices than to the species diversity. Therefore, although the spatial and spectral properties of Sentinel-2 seem limited to assess the species diversity in grasslands directly, this satellite should make possible the continuous monitoring of factors influencing biodiversity in grasslands. In this thesis, we provided methods that account for the heterogeneity within grasslands and enable the use of all the spectral and temporal information provided by new generation satellites

Sentinel-2 Remote Sensed Image Classification with Patchwise Trained ConvNets for Grassland Habitat Discrimination

The present study focuses on the use of Convolutional Neural Networks (CNN or ConvNet) to classify a multi-seasonal dataset of Sentinel-2 images to discriminate four grassland habitats in the “Murgia Alta” protected site. To this end, we compared two approaches differing only by the first layer machinery, which, in one case, is instantiated as a fully-connected layer and, in the other case, results in a ConvNet equipped with kernels covering the whole input (wide-kernel ConvNet). A patchwise approach, tessellating training reference data in square patches, was adopted. Besides assessing the effectiveness of ConvNets with patched multispectral data, we analyzed how the information needed for classification spreads to patterns over convex sets of pixels. Our results show that: (a) with an F1-score of around 97% (5 x 5 patch size), ConvNets provides an excellent tool for patch-based pattern recognition with multispectral input data without requiring special feature extraction; (b) the information spreads over the limit of a single pixel: the performance of the network increases until 5 x 5 patch sizes are used and then ConvNet performance starts decreasing

Exploring issues of balanced versus imbalanced samples in mapping grass community in the telperion reserve using high resolution images and selected machine learning algorithms

ABSTRACT Accurate vegetation mapping is essential for a number of reasons, one of which is for conservation purposes. The main objective of this research was to map different grass communities in the game reserve using RapidEye and Sentinel-2 MSI images and machine learning classifiers [support vector machine (SVM) and Random forest (RF)] to test the impacts of balanced and imbalance training data on the performance and the accuracy of Support Vector Machine and Random forest in mapping the grass communities and test the sensitivities of pixel resolution to balanced and imbalance training data in image classification. The imbalanced and balanced data sets were obtained through field data collection. The results show RF and SVM are producing a high overall accuracy for Sentinel-2 imagery for both the balanced and imbalanced data set. The RF classifier has yielded an overall accuracy of 79.45% and kappa of 74.38% and an overall accuracy of 76.19% and kappa of 73.21% using imbalanced and balanced training data respectively. The SVM classifier yielded an overall accuracy of 82.54% and kappa of 80.36% and an overall accuracy of 82.21% and a kappa of 78.33% using imbalanced and balanced training data respectively. For the RapidEye imagery, RF and SVM algorithm produced overall accuracy affected by a balanced data set leading to reduced accuracy. The RF algorithm had an overall accuracy that dropped by 6% (from 63.24% to 57.94%) while the SVM dropped by 7% (from 57.31% to 50.79%). The results thereby show that the imbalanced data set is a better option when looking at the image classification of vegetation species than the balanced data set. The study recommends the implementation of ways of handling misclassification among the different grass species to improve classification for future research. Further research can be carried out on other types of high resolution multispectral imagery using different advanced algorithms on different training size samples.EM201

Remote Sensing for Monitoring the Mountaintop Mining Landscape: Applications for Land Cover Mapping at the Individual Mine Complex Scale

The aim of this dissertation was to investigate the potential for mapping land cover associated with mountaintop mining in Southern West Virginia using high spatial resolution aerial- and satellite-based multispectral imagery, as well as light detection and ranging (LiDAR) elevation data and terrain derivatives. The following research themes were explored: comparing aerial- and satellite-based imagery, combining data sets of multiple dates and types, incorporating measures of texture, using nonparametric, machine learning classification algorithms, and employing a geographical object-based image analysis (GEOBIA) framework. This research is presented as four interrelated manuscripts.;In a comparison of aerial National Agriculture Imagery Program (NAIP) orthophotography and satellite-based RapidEye data, the aerial imagery was found to provide statistically less accurate classifications of land cover. These lower accuracies are most likely due to inconsistent viewing geometry and radiometric normalization associated with the aerial imagery. Nevertheless, NAIP orthophotography has many characteristics that make it useful for surface mine mapping and monitoring, including its availability for multiple years, a general lack of cloud cover, contiguous coverage of large areas, ease of availability, and low cost. The lower accuracies of the NAIP classifications were somewhat remediated by decreasing the spatial resolution and reducing the number of classes mapped.;Combining LiDAR with multispectral imagery statistically improved the classification of mining and mine reclamation land cover in comparison to only using multispectral data for both pixel-based and GEOBIA classification. This suggests that the reduced spectral resolution of high spatial resolution data can be combated by incorporating data from another sensor.;Generally, the support vector machines (SVM) algorithm provided higher classification accuracies in comparison to random forests (RF) and boosted classification and regression trees (CART) for both pixel-based and GEOBIA classification. It also outperformed k-nearest neighbor, the algorithm commonly used for GEOBIA classification. However, optimizing user-defined parameters for the SVM algorithm tends to be more complex in comparison to the other algorithms. In particular, RF has fewer parameters, and the program seems robust regarding the parameter settings. RF also offers measures to assess model performance, such as estimates of variable importance and overall accuracy.;Textural measures were found to be of marginal value for pixel-based classification. For GEOBIA, neither measures of texture nor object-specific geometry improved the classification accuracy. Notably, the incorporation of additional information from LiDAR provided a greater improvement in classification accuracy then deriving complex textural and geometric measures.;Pre- and post-mining terrain data classified using GEOBIA and machine learning algorithms resulted in significantly more accurate differentiation of mine-reclaimed and non-mining grasslands than was possible with spectral data. The combination of pre- and post-mining terrain data or just pre-mining data generally outperformed post-mining data. Elevation change data were shown to be of particular value, as were terrain shape parameters. GEOBIA was a valuable tool for combining data collected using different sensors and gridded at variable cell sizes, and machine learning algorithms were particularly useful for incorporating the ancillary data derived from the digital elevation models (DEMs), since these most likely would not have met the basic assumptions of multivariate normality required for parametric classifiers.;Collectively, this research suggests that high spatial resolution remotely sensed data are valuable for mapping and monitoring surface mining and mine reclamation, especially when elevation and spectral data are combined. Machine learning algorithms and GEOBIA are useful for integrating such diverse data

Lokaalstatistikute kasutamine rohumaade ja metsade kaugseires

Väitekirja elektrooniline versioon ei sisalda publikatsiooneKäesolev doktoritöö analüüsib lokaalstatistikute kasutamist rohumaade ja metsade kaugseires. Töö esimene osa käsitleb rohumaade monitoorimist tehisava-radari (synthetic aperture radar (SAR)) abil ning teine osa metsade kaugseiret kasutades optilisi sensoreid. Analüüsides rohumaade niitmise ja C- laineala tehisava-radari interferomeetrilise koherentsuse seoseid leiti, et selle parameetri kasutamisel on potentsiaali niitmise tuvastamise algoritmide ja rakenduste väljaarendamiseks. Tulemused näitavad, et pärast niitmist on VH ja VV polarisatsiooni 12-päeva interferomeetrilise koherentsuse mediaan väärtused statistiliselt oluliselt kõrgemad võrreldes niitmise eelse olukorraga. Koherentsus on seda kõrgem, mida väiksem on ajaline vahe niitmise ja pärast seda üles võetud esimese interferomeetrilise mõõtmise vahel. Hommikune kaste, sademed, põllutööde teostamine, näiteks külv või kündmine, kõrgelt niitmine ja kiire rohu kasv pärast niitmist vähendavad koherentsust ja raskendavad niitmise sündmuste eristamist. Selleks, et eelpoolnimetatud mõjusid leevendada tuleks tulevikus uurida 6-päeva koherentsuse ja niitmise sündmuste vahelisi seoseid. Käesolevas doktoritöös esitatud tulemused loovad siiski tugeva aluse edasisteks uuringuteks ja arendusteks eesmärgiga võtta C-laineala tehisava-radari andmed niitmise tuvastamisel ka praktikas kasutusele. Lisaks näidati, et ortofotodel põhinevate metsa kaugseire hinnangute andmisel on abi lokaalstatistikute kasutamisest. Analüüsides kaugseire hinnangut riigimetsa takseerandmete (national forest inventory) kohta leiti, et näidistel põhinev järeldamine (case-based reasoning (CBR)) sobib hästi selliste kaugseire ülesannete empiirilisteks lahendusteks, kus sisendandmetena on kasutatavad väga paljud erinevad andmeallikad. Leiti, et klasteranalüüsi saab kasutada kaugseire tunnuste eelvaliku meetodina. Võrreldes erinevaid tekstuuri statistikuid näidati, et lokaalselt arvutatud keskväärtus on kõige väärtuslikum tunnus. Järeldati, et nii statistiliste kui ka struktuursete lokaalstatistikute kasutamisega saab lisada pikslipõhistele kaugseire hinnangutele olulist andmestikku.This thesis studies approaches for remote sensing of grasslands and forests based on local statistics. The first part of the thesis focuses on monitoring of grasslands with SAR and the second part to monitoring of forests with optical sensors. It is shown that there is potential to develop mowing detection algorithms and applications using C-band SAR temporal interferometric coherence. The results demonstrate that after a mowing event, median VH and VV polarisation 12-day interferometric coherence values are statistically significantly higher than those from before the event. The sooner after the mowing event the first interferometric acquisition is taken, the higher the coherence. Morning dew, precipitation, farming activities, such as sowing or ploughing, high residual straws after the cut and rapid growth of grass are causing the coherence to decrease and impede the distinction of a mowing event. In the future, six-day interferometric coherence should also be analysed in relation to mowing events to alleviate some of these factors. Nevertheless, the results presented in this thesis offer a strong basis for further research and development activities towards the practical use of spaceborne C-band SAR data for mowing detection. Further, it was shown that local statistics can be useful for estimation of forest parameters from ortophotos and they could also provide helpful ancillary information to conduct a photo-interpretation tasks over forested areas. It was demonstrated that cluster analysis can be used as pre-selection method for the reduction of remote sensing features. Additionally, it was shown that case-based reasoning (a machine learning method) is well suited for empirical solutions of remote sensing tasks where there are many different data sources available. It was concluded that the use of local statistics adds valuable data to pixel-based remote sensing estimations

Knowledge-Based Classification of Grassland Ecosystem Based on Multi-Temporal WorldView-2 Data and FAO-LCCS Taxonomy

Grassland ecosystems can provide a variety of services for humans, such as carbon storage, food production, crop pollination and pest regulation. However, grasslands are today one of the most endangered ecosystems due to land use change, agricultural intensification, land abandonment as well as climate change. The present study explores the performance of a knowledge-driven GEOgraphic-Object—based Image Analysis (GEOBIA) learning scheme to classify Very High Resolution(VHR)imagesfornaturalgrasslandecosystemmapping. Theclassificationwasappliedto a Natura 2000 protected area in Southern Italy. The Food and Agricultural Organization Land Cover Classification System (FAO-LCCS) hierarchical scheme was instantiated in the learning phase of the algorithm. Four multi-temporal WorldView-2 (WV-2) images were classified by combining plant phenology and agricultural practices rules with prior-image spectral knowledge. Drawing on this knowledge, spectral bands and entropy features from one single date (Post Peak of Biomass) were firstly used for multiple-scale image segmentation into Small Objects (SO) and Large Objects (LO). Thereafter, SO were labelled by considering spectral and context-sensitive features from the whole multi-seasonal data set available together with ancillary data. Lastly, the labelled SO were overlaid to LO segments and, in turn, the latter were labelled by adopting FAO-LCCS criteria about the SOs presence dominance in each LO. Ground reference samples were used only for validating the SO and LO output maps. The knowledge driven GEOBIA classifier for SO classification obtained an OA value of 97.35% with an error of 0.04. For LO classification the value was 75.09% with an error of 0.70. At SO scale, grasslands ecosystem was classified with 92.6%, 99.9% and 96.1% of User’s, Producer’s Accuracy and F1-score, respectively. The findings reported indicate that the knowledge-driven approach not only can be applied for (semi)natural grasslands ecosystem mapping in vast and not accessible areas but can also reduce the costs of ground truth data acquisition. The approach used may provide different level of details (small and large objects in the scene) but also indicates how to design and validate local conservation policies

Machine Learning for Informed Representation Learning

The way we view reality and reason about the processes surrounding us is intimately connected to our perception and the representations we form about our observations and experiences. The popularity of machine learning and deep learning techniques in that regard stems from their ability to form useful representations by learning from large sets of observations. Typical application examples include image recognition or language processing for which artificial neural networks are powerful tools to extract regularity patterns or relevant statistics. In this thesis, we leverage and further develop this representation learning capability to address relevant but challenging real-world problems in geoscience and chemistry, to learn representations in an informed manner relevant to the task at hand, and reason about representation learning in neural networks, in general. Firstly, we develop an approach for efficient and scalable semantic segmentation of degraded soil in alpine grasslands in remotely-sensed images based on convolutional neural networks. To this end, we consider different grassland erosion phenomena in several Swiss valleys. We find that we are able to monitor soil degradation consistent with state-of-the-art methods in geoscience and can improve detection of affected areas. Furthermore, our approach provides a scalable method for large-scale analysis which is infeasible with established methods. Secondly, we address the question of how to identify suitable latent representations to enable generation of novel objects with selected properties. For this, we introduce a new deep generative model in the context of manifold learning and disentanglement. Our model improves targeted generation of novel objects by making use of property cycle consistency in property-relevant and property-invariant latent subspaces. We demonstrate the improvements on the generation of molecules with desired physical or chemical properties. Furthermore, we show that our model facilitates interpretability and exploration of the latent representation. Thirdly, in the context of recent advances in deep learning theory and the neural tangent kernel, we empirically investigate the learning of feature representations in standard convolutional neural networks and corresponding random feature models given by the linearisation of the neural networks. We find that performance differences between standard and linearised networks generally increase with the difficulty of the task but decrease with the considered width or over-parametrisation of these networks. Our results indicate interesting implications for feature learning and random feature models as well as the generalisation performance of highly over-parametrised neural networks. In summary, we employ and study feature learning in neural networks and review how we may use informed representation learning for challenging tasks

Enhancing temporal series of Sentinel-2 and Sentinel-3 data products: from classical regression to deep learning approach

Treball de Final de Màster Universitari Erasmus Mundus en Tecnologia Geoespacial (Pla de 2013). Codi: SIW013. Curs acadèmic 2020-2021The free and open availability of satellite images covering global extent in recent days provides many novel opportunities for global monitoring of the earth’s surface. Sentinel-2 (S2) and Sentinel-3 (S3) satellite missions capture mid to high resolution imagery with frequent revisit and show data synergy as they both focus on land and ocean observational needs. Specifically, the high temporal resolution of S3 (1-2 day revisit) presents potential in filling the data gaps in S2 (5 day revisit) vegetation products. In this scenario, this study assesses the feasibility of using Sentinel-3 images for Sentinel-2 vegetation products estimation using machine learning (ML) and deep learning (DL) approaches. This study employs four state of the art ML regression algorithms, linear regression, ridge regression, Support Vector Regression (SVR) and Random Forest Regression (RFR) and two DL network architectures with different depth and complexities, Multi-Layer Perceptron (MLP) and Convolutional Neural Network (CNN) to predict the S2 NDVI and SAVI maps from the S3 spectral bands information. A paired S2/S3 dataset is prepared for the study area covering one S2 tile in Extremadura, Spain. The results demonstrate that all the DL architectures except pixel-wise MLP outperformed the ML models with the 3D CNN performing the best. The best performing 3D CNN architecture obtained remarkable mean squared error (MSE) of 0.00198 for NDVI and 0.00282 for SAVI while the best performing ML algorithms were patch-wise RFR with MSE of 0.0035 in case of NDVI and patchwise SVR with MSE of 0.00586 for SAVI. The models and the dataset prepared for this study will be useful for further research that focus on capitalizing the free and open availability of Sentinel-2 and Sentinel-3 imagery as well as new and advanced technologies to provide better vegetation monitoring capabilities for our planet