Spatial distribution and temporal variation of tropical mountaintop vegetation through images obtained by drones

Modern UAS (Unmanned Aerial Vehicles) or just drones have emerged with the primary goal of producing maps and imagery with extremely high spatial resolution. The refined information provides a good opportunity to quantify the distribution of vegetation across heterogeneous landscapes, revealing an important strategy for biodiversity conservation. We investigate whether computer vision and machine learning techniques (Object-Based Image Analysis—OBIA method, associated with Random Forest classifier) are effective to classify heterogeneous vegetation arising from ultrahigh-resolution data generated by UAS images. We focus our fieldwork in a highly diverse, seasonally dry, complex mountaintop vegetation system, the campo rupestre or rupestrian grassland, located at Serra do Cipó, Espinhaço Range, Southeastern Brazil. According to our results, all classifications received general accuracy above 0.95, indicating that the methodological approach enabled the identification of subtle variations in species composition, the capture of detailed vegetation and landscape features, and the recognition of vegetation types’ phenophases. Therefore, our study demonstrated that the machine learning approach and combination between OBIA method and Random Forest classifier, generated extremely high accuracy classification, reducing the misclassified pixels, and providing valuable data for the classification of complex vegetation systems such as the campo rupestre mountaintop grassland

Spatial distribution and temporal variation of tropical mountaintop vegetation through images obtained by drones

Modern UAS (Unmanned Aerial Vehicles) or just drones have emerged with the primary goal of producing maps and imagery with extremely high spatial resolution. The refined information provides a good opportunity to quantify the distribution of vegetation across heterogeneous landscapes, revealing an important strategy for biodiversity conservation. We investigate whether computer vision and machine learning techniques (Object-Based Image Analysis—OBIA method, associated with Random Forest classifier) are effective to classify heterogeneous vegetation arising from ultrahigh-resolution data generated by UAS images. We focus our fieldwork in a highly diverse, seasonally dry, complex mountaintop vegetation system, the campo rupestre or rupestrian grassland, located at Serra do Cipó, Espinhaço Range, Southeastern Brazil. According to our results, all classifications received general accuracy above 0.95, indicating that the methodological approach enabled the identification of subtle variations in species composition, the capture of detailed vegetation and landscape features, and the recognition of vegetation types’ phenophases. Therefore, our study demonstrated that the machine learning approach and combination between OBIA method and Random Forest classifier, generated extremely high accuracy classification, reducing the misclassified pixels, and providing valuable data for the classification of complex vegetation systems such as the campo rupestre mountaintop grassland

Mapping and monitoring forest remnants : a multiscale analysis of spatio-temporal data

KEYWORDS : Landsat, time series, machine learning, semideciduous Atlantic forest, Brazil, wavelet transforms, classification, change detectionForests play a major role in important global matters such as carbon cycle, climate change, and biodiversity. Besides, forests also influence soil and water dynamics with major consequences for ecological relations and decision-making. One basic requirement to quantify and model these processes is the availability of accurate maps of forest cover. Data acquisition and analysis at appropriate scales is the keystone to achieve the mapping accuracy needed for development and reliable use of ecological models.The current and upcoming production of high-resolution data sets plus the ever-increasing time series that have been collected since the seventieth must be effectively explored. Missing values and distortions further complicate the analysis of this data set. Thus, integration and proper analysis is of utmost importance for environmental research. New conceptual models in environmental sciences, like the perception of multiple scales, require the development of effective implementation techniques.This thesis presents new methodologies to map and monitor forests on large, highly fragmented areas with complex land use patterns. The use of temporal information is extensively explored to distinguish natural forests from other land cover types that are spectrally similar. In chapter 4, novel schemes based on multiscale wavelet analysis are introduced, which enabled an effective preprocessing of long time series of Landsat data and improved its applicability on environmental assessment.In chapter 5, the produced time series as well as other information on spectral and spatial characteristics were used to classify forested areas in an experiment relating a number of combinations of attribute features. Feature sets were defined based on expert knowledge and on data mining techniques to be input to traditional and machine learning algorithms for pattern recognition, viz . maximum likelihood, univariate and multivariate decision trees, and neural networks. The results showed that maximum likelihood classification using temporal texture descriptors as extracted with wavelet transforms was most accurate to classify the semideciduous Atlantic forest in the study area.In chapter 6, a multiscale approach to digital change detection was developed to deal with multisensor and noisy remotely sensed images. Changes were extracted according to size classes minimising the effects of geometric and radiometric misregistration.Finally, in chapter 7, an automated procedure for GIS updating based on feature extraction, segmentation and classification was developed to monitor the remnants of semideciduos Atlantic forest. The procedure showed significant improvements over post classification comparison and direct multidate classification based on artificial neural networks.</p

Visualização de padrões temporais cíclicos em estudos de fenologia

Orientadores: Ricardo da Silva Torres, Leonor Patrícia Cerdeira MorellatoTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Em diversas aplicações, grandes volumes de dados multidimensionais têm sido gerados continuamente ao longo do tempo. Uma abordagem adequada para lidar com estas coleções consiste no uso de métodos de visualização de informação, a partir dos quais padrões de interesse podem ser identificados, possibilitando o entendimento de fenômenos temporais complexos. De fato, em diversos domínios, o desenvolvimento de ferramentas adequadas para apoiar análises complexas, por exemplo, aquelas baseadas na identificação de padrões de mudanças ou correlações existentes entre múltiplas variáveis ao longo do tempo é de suma importância. Em estudos de fenologia, por exemplo, especialistas observam as mudanças que ocorrem ao longo da vida de plantas e animais e investigam qual é a relação entre essas mudanças com variáveis ambientais. Neste cenário, especialistas em fenologia cada vez mais precisam de ferramentas para, adequadamente, visualizar séries temporais longas, com muitas variáveis e de diferentes tipos (por exemplo, texto e imagem), assim como identificar padrões temporais cíclicos. Embora diversas abordagens tenham sido propostas para visualizar dados que variam ao longo do tempo, muitas não são apropriadas ou aplicáveis para dados de fenologia, porque não são capazes de: (i) lidar com séries temporais longas, com muitas variáveis de diferentes tipos de dados e de uma ou mais dimensões; e (ii) permitir a identificação de padrões temporais cíclicos e drivers ambientais associados. Este trabalho aborda essas questões a partir da proposta de duas novas abordagens para apoiar a análise e visualização de dados temporais multidimensionais. Nossa primeira proposta combina estruturas visuais radiais com ritmos visuais. As estruturas radiais são usadas para fornecer informação contextual sobre fenômenos cíclicos, enquanto que o ritmo visual é usado para sumarizar séries temporais longas em representações compactas. Nós desenvolvemos, avaliamos e validamos nossa proposta com especialistas em fenologia em tarefas relacionadas à visualização de dados de observação direta da fenologia de plantas em nível tanto de indivíduos quanto de espécies. Nós também validamos a proposta usando dados temporais relacionados a imagens obtidas de sistemas de monitoramento de vegetação próxima à superfície. Nossa segunda abordagem é uma nova representação baseada em imagem, chamada Change Frequency Heatmap (CFH), usada para codificar mudanças temporais de dados numéricos multivariados. O método calcula histogramas de padrões de mudanças observados em sucessivos instantes de tempo. Nós validamos o uso do CFH a partir da criação de uma ferramenta de caracterização de mudanças no ciclo de vida de plantas de múltiplos indivíduos e espécies ao longo do tempo. Nós demonstramos o potencial do CFH para ajudar na identificação visual de padrões de mudanças temporais complexas, especialmente na identificação de variações entre indivíduos em estudos relacionados à fenologia de plantasAbstract: In several applications, large volumes of multidimensional data have been generated continuously over time. One suitable approach for handling those collections in a meaningful way consists in the use of information visualization methods, based on which patterns of interest can be identified, triggering the understanding of complex temporal phenomena. In fact, in several domains, the development of appropriate tools for supporting complex analysis based, for example, on the identification of change patterns in temporal data or existing correlations, over time, among multiple variables, is of paramount importance. In phenology studies, for instance, phenologists observe changes in the development of plants and animals throughout their lives and investigate what is the relationship between these changes with environmental changes. Therefore, phenologists increasingly need tools for visualizing appropriately long-term series with many variables of different data types, as well as for identifying cyclical temporal patterns. Although several approaches have been proposed to visualize data varying over time, most of them are not appropriate or applicable to phenology data, because they are not able: (i) to handle long-term series with many variables of different data types and one or more dimensions and (ii) to support the identification of cyclical temporal patterns and associated environmental drivers. This work addresses these shortcomings by presenting two new approaches to support the analysis and visualization of multidimensional temporal data. Our first proposal to visualize phenological data combines radial visual structures along with visual rhythms. Radial visual structures are used to provide contextual insights regarding cyclical phenomena, while the visual rhythm encoding is used to summarize long-term time series into compact representations. We developed, evaluated, and validated our proposal with phenology experts using plant phenology direct observational data both at individuals and species levels. Also we validated the proposal using image-related temporal data obtained from near-surface vegetation monitoring systems. Our second approach is a novel image-based representation, named Change Frequency Heatmap (CFH), used to encode temporal changes of multivariate numerical data. The method computes histograms of change patterns observed at successive timestamps. We validated the use of CFHs through the creation of a temporal change characterization tool to support complex plant phenology analysis, concerning the characterization of plant life cycle changes of multiple individuals and species over time. We demonstrated the potential of CFH to support visual identification of complex temporal change patterns, especially to decipher interindividual variations in plant phenologyDoutoradoCiência da ComputaçãoDoutora em Ciência da Computação162312/2015-62013/501550-0CNPQCAPESFAPES

Aplicações de modelos de deep learning para monitoramento ambiental e agrícola no Brasil

Tese (doutorado) — Universidade de Brasília, Instituto de Ciências Humanas, Departamento de Geografia, Programa de Pós-Graduação em Geografia, 2022.Algoritmos do novo campo de aprendizado de máquina conhecido como Deep Learning têm se popularizado recentemente, mostrando resultados superiores a modelos tradicionais em métodos de classificação e regressão. O histórico de sua utilização no campo do sensoriamento remoto ainda é breve, porém eles têm mostrado resultados similarmente superiores em processos como a classificação de uso e cobertura da terra e detecção de mudança. Esta tese teve como objetivo o desenvolvimento de metodologias utilizando estes algoritmos com um enfoque no monitoramento de alvos críticos no Brasil por via de imagens de satélite a fim de buscar modelos de alta precisão e acurácia para substituir metodologias utilizadas atualmente. Ao longo de seu desenvolvimento, foram produzidos três artigos onde foi avaliado o uso destes algoritmos para a detecção de três alvos distintos: (a) áreas queimadas no Cerrado brasileiro, (b) áreas desmatadas na região da Amazônia e (c) plantios de arroz no sul do Brasil. Apesar do objetivo similar na produção dos artigos, procurou-se distinguir suficientemente suas metodologias a fim de expandir o espaço metodológico conhecido para fornecer uma base teórica para facilitar e incentivar a adoção destes algoritmos em contexto nacional. O primeiro artigo avaliou diferentes dimensões de amostras para a classificação de áreas queimadas em imagens Landsat-8. O segundo artigo avaliou a utilização de séries temporais binárias de imagens Landsat para a detecção de novas áreas desmatadas entre os anos de 2017, 2018 e 2019. O último artigo utilizou imagens de radar Sentinel-1 (SAR) em uma série temporal contínua para a delimitação dos plantios de arroz no Rio Grande do Sul. Modelos similares foram utilizados em todos os artigos, porém certos modelos foram exclusivos a cada publicação, produzindo diferentes resultados. De maneira geral, os resultados encontrados mostram que algoritmos de Deep Learning são não só viáveis para detecção destes alvos mas também oferecem desempenho superior a métodos existentes na literatura, representando uma alternativa altamente eficiente para classificação e detecção de mudança dos alvos avaliados.Algorithms belonging to the new field of machine learning called Deep Learning have been gaining popularity recently, showing superior results when compared to traditional classification and regression methods. The history of their use in the field of remote sensing is not long, however they have been showing similarly superior results in processes such as land use classification and change detection. This thesis had as its objective the development of methodologies using these algorithms with a focus on monitoring critical targets in Brazil through satellite imagery in order to find high accuracy and precision models to substitute methods used currently. Through the development of this thesis, articles were produced evaluating their use for the detection of three distinct targets: (a) burnt areas in the Brazilian Cerrado, (b) deforested areas in the Amazon region and (c) rice fields in the south of Brazil. Despite the similar objective in the production of these articles, the methodologies in each of them was made sufficiently distinct in order to expand the methodological space known. The first article evaluated the use of differently sized samples to classify burnt areas in Landsat-8 imagery. The second article evaluated the use of binary Landsat time series to detect new deforested areas between the years of 2017, 2018 and 2019. The last article used continuous radar Sentinel-1 (SAR) time series to map rice fields in the state of Rio Grande do Sul. Similar models were used in all articles, however certain models were exclusive to each one. In general, the results show that not only are the Deep Learning models viable but also offer better results in comparison to other existing methods, representing an efficient alternative when it comes to the classification and change detection of the targets evaluated

Eficácia de medidas de similaridade para a classificação de séries temporais associadas ao comportamento fenológico de plantas

Orientadores: Luiz Camolesi Júnior, Ricardo da Silva TorresDissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de TecnologiaResumo: Fenologia é o estudo de fenômenos naturais periódicos e sua relação com o clima. Nos últimos anos, tem se apresentado relevante como o indicador mais simples e confiável dos efeitos das mudanças climáticas em plantas e animais. É nesse contexto que se destaca o e-phenology, um projeto multidisciplinar envolvendo pesquisas na área de computação e fenologia. Suas principais características são: o uso de novas tecnologias de monitoramento ambiental, o fornecimento de modelos, métodos e algoritmos para apoiar o gerenciamento, a integração e a análise remota de dados de fenologia, além da criação de um protocolo para um programa de monitoramento de fenologia. Do ponto de vista da computação, as pesquisas científicas buscam modelos, ferramentas e técnicas baseadas em processamento de imagem, extraindo e indexando características de imagens associadas a diferentes tipos de vegetação, além de se concentrar no gerenciamento e mineração de dados e no processamento de séries temporais. Diante desse cenário, esse trabalho especificamente, tem como objetivo investigar a eficácia de medidas de similaridade para a classificação de séries temporais sobre fenômenos fenológicos caracterizados por vetores de características extraídos de imagens de vegetação. Os cálculos foram realizados considerando regiões de imagens de vegetação e foram considerados diferentes critérios de avaliação: espécies de planta, hora do dia e canais de cor. Os resultados obtidos oferecem algumas possibilidades de análise, porém na visão geral, a medida de distância Edit Distance with Real Penalty (ERP) apresentou o índice de acerto mais alto com 29,90%. Adicionalmente, resultados obtidos mostram que as primeiras horas do dia e no final da tarde, provavelmente devido à luminosidade, apresentam os índices de acerto mais altos para todas as visões de análiseAbstract: Phenology is the study of periodic natural phenomena and their relationship to climate. In recent years, it has gained importance as the more simple and reliable indicator of effects of climate changes on plants and animals. In this context, we emphasizes the e-phenology, a multidisciplinary research project in computer science and phenology. Its main characteristics are: The use of new technologies for environmental monitoring, providing models, methods and algorithms to support management, integration and remote analysis of data on phenology, and the creation a protocol for a program to monitoring phenology. From the computer science point of view, the e-phenology project has been dedicated to creating models, tools and techniques based on image processing algorithms, extracting and indexing image features associated with different types of vegetation, and implementing data mining algorithms for processing time series. This project has as main goal to investigate the effectiveness of similarity measures for the classification of time series associated with phenological phenomena characterized by feature vectors extracted from images. Conducted experiments considered different regions containing individuals of different species and considering different criteria such as: plant species, time of day and color channels. Obtained results show that the Edit Distance with Real Penalty (ERP) distance measure yields the highest accuracy. Additionally, the analyzes show that in the early morning and late afternoon, probably due to light conditions, it can be observed the highest accuracy rates for all views analysisMestradoTecnologia e InovaçãoMestre em Tecnologi

Mapping a Brazilian deforestation frontier using multi-temporal TerraSAR-X data and supervised machine learning

Satellite remote sensing enables a repeated survey of the earth’s surface. With machine learning it is possible to recognize complex patterns from extensive data sets. Using methods from machine learning, remote sensing images are utilized to derive large scale land use and land cover (LULC) maps, carrying discrete information on the human management of land and intact primary forests, as well as change processes. Such information is particularly relevant in little developed regions, and areas which are undergoing transformation. Therefore, satellite remote sensing is generally the preferred method for generating LULC products within tropical regions, and particularly useful to assist tracking of change processes with regard to deforestation or land management. The Amazon is the largest area of continuous tropical forest in the world, and of substantial importance with regard to biodiversity, its influence on global climate, as well as providing living space for a large number of indigenous tribes. As tropical region, the Amazon is particularly affected by cloudy conditions, which pose a serious challenge to many remote sensing efforts. Utilization of Synthetic Aperture Radar (SAR) hence is promoted, as this warrants data availability at fixed intervals. Performing land cover mapping at the deforestation frontier in the Brazilian states of Pará and Mato Grosso, the aim of this thesis is to evaluate latest concepts from machine learning and SAR remote sensing in the light of real world applicability. As a cumulative effort, this thesis provides a scalable method based on Markov Random Fields, to increase classification performance. This method is especially useful to enhance the outcome of SAR classifications, as it directly addresses inherent SAR properties such as multi-temporality and speckle. Furthermore, ALOS-2, RADARSAT-2, and TerraSAR-X, which are current SAR sensors fulfilling different properties with regard to ground resolution and wavelength, are being investigated concerning their synergetic potentials for the mapping of vegetated LULC classes of the Brazilian Amazon. Here, the additional value of combining multiple frequencies is evaluated using reliable validation techniques based on area adjustment. Additionally, single performance of the three sensors is evaluated and their potentials concerning the task of tropical mapping are estimated. Lastly, different potentials of TanDEM-X for the purpose of tropical mapping are investigated. TanDEM-X is the first continuous spaceborne missionvi to offer a bi-static acquisition of data, enabling the generation of height models and the collection of coherence layers via a single pass

Operationalization of Remote Sensing Solutions for Sustainable Forest Management

The great potential of remote sensing technologies for operational use in sustainable forest management is addressed in this book, which is the reprint of papers published in the Remote Sensing Special Issue “Operationalization of Remote Sensing Solutions for Sustainable Forest Management”. The studies come from three continents and cover multiple remote sensing systems (including terrestrial mobile laser scanning, unmanned aerial vehicles, airborne laser scanning, and satellite data acquisition) and a diversity of data processing algorithms, with a focus on machine learning approaches. The focus of the studies ranges from identification and characterization of individual trees to deriving national- or even continental-level forest attributes and maps. There are studies carefully describing exercises on the case study level, and there are also studies introducing new methodologies for transdisciplinary remote sensing applications. Even though most of the authors look forward to continuing their research, nearly all studies introduced are ready for operational use or have already been implemented in practical forestry

Image-based time series representations for satellite images classification

Orientador: Ricardo da Silva TorresTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: A classificação de imagens de sensoriamento remoto por pixel com base no perfil temporal desempenha um papel importante em várias aplicações, tais como: reconhecimento de culturas, estudos fenológicos e monitoramento de mudanças na cobertura do solo. Avanços sensores captura de imagem aumentaram a necessidade de criação de metodologias para analisar o perfil temporal das informações coletadas. Nós investigamos dados coletados em dois tipos de sensores: (i) sensores em plataformas orbitais, esse tipo de imagem sofre interferências de nuvens e fatores atmosféricos; e (ii) sensores fixados em campo, mais especificamente, uma câmera digital no alto de uma torre, cujas imagens capturadas podem conter dezenas de espécies, dificultando a identificação de padrões de interesse. Devido às particularidades dos dados detectados remotamente, torna-se custoso enviar a imagem capturada pelo sensor diretamente para métodos de aprendizado de máquina sem realizar um pré-processamento. Para algumas aplicações de sensoriamento remoto, comumente não se utiliza as imagens brutas oriundas dos sensores, mas os índices de vegetação extraídos das regiões de interesse ao longo do tempo. Assim, o perfil temporal pode ser caracterizado como uma série de observações dos índices de vegetação dos pixels de interesse. Métodos baseados em aprendizado profundo obtiveram bons resultados em aplicações de sensoriamento remoto relacionadas à classificação de imagens. Contudo, em consequência da natureza dos dados, nem sempre é possível realizar o treinamento adequado das redes de aprendizado profundo pela limitação causada por dados faltantes. Entretanto, podemos nos beneficiar de redes previamente treinadas para detecção de objetos para extrair características e padrões de imagens. O problema alvo deste trabalho é classificar séries temporais extraídas de imagens de sensoriamento remoto representando as características temporais como imagens 2D. Este trabalho investiga abordagens que codificam séries temporais como representação de imagem para propor metodologias de classificação binária e multiclasse no contexto de sensoriamento remoto, se beneficiando de redes extratoras de características profundas. Os experimentos conduzidos para classificação binária foram realizados em dados de satélite para identificar plantações de eucalipto. Os resultados superaram métodos baseline propostos recentemente. Os experimentos realizados para classificação multiclasse focaram em imagens capturadas com câmera digital para detectar o padrão fenológico de regiões de interesse. Os resultados mostram que a acurácia aumenta se consideramos conjuntos de pixelsAbstract: Pixelwise remote sensing image classification based on temporal profile plays an important role in several applications, such as crop recognition, phenological studies, and land cover change monitoring. Advances in image capture sensors have increased the need for methodologies to analyze the temporal profile of collected information. We investigate data collected by two types of sensors: (i) sensors on orbital platforms, this type of image suffers from cloud interference and atmospheric factors; and (ii) field-mounted sensors, in particular, a digital camera on top of a tower, where captured images may contain dozens of species, making it difficult to identify patterns of interest. Due to the particularities of remotely detected data, it is prohibitive to send sensor captured images directly to machine learning methods without preprocessing. In some remote sensing applications, it is not commonly used the raw images from the sensors, but the vegetation indices extracted from regions of interest over time. Thus, the temporal profile can be characterized as a series of observations of vegetative indices of pixels of interest. Deep learning methods have been successfully in remote sensing applications related to image classification. However, due to the nature of the data, it is not always possible to properly train deep learning networks because of the lack of enough labeled data. However, we can benefit from previously trained 2D object detection networks to extract features and patterns from images. The target problem of this work is to classify remote sensing images, based on pixel time series represented as 2D representations. This work investigates approaches that encode time series into image representations to propose binary and multiclass classification methodologies in the context of remote sensing, taking advantage of data-driven feature extractor approaches. The experiments conducted for binary classification were performed on satellite data to identify eucalyptus plantations. The results surpassed the ones of recently proposed baseline methods. The experiments performed for multiclass classification focused on detecting regions of interest within images captured by a digital camera. The results show that the accuracy increases if we consider a set of pixelsDoutoradoCiência da ComputaçãoDoutora em Ciência da ComputaçãoCAPE