Combining multiple resolutions into hierarchical representations for kernel-based image classification

Geographic object-based image analysis (GEOBIA) framework has gained increasing interest recently. Following this popular paradigm, we propose a novel multiscale classification approach operating on a hierarchical image representation built from two images at different resolutions. They capture the same scene with different sensors and are naturally fused together through the hierarchical representation, where coarser levels are built from a Low Spatial Resolution (LSR) or Medium Spatial Resolution (MSR) image while finer levels are generated from a High Spatial Resolution (HSR) or Very High Spatial Resolution (VHSR) image. Such a representation allows one to benefit from the context information thanks to the coarser levels, and subregions spatial arrangement information thanks to the finer levels. Two dedicated structured kernels are then used to perform machine learning directly on the constructed hierarchical representation. This strategy overcomes the limits of conventional GEOBIA classification procedures that can handle only one or very few pre-selected scales. Experiments run on an urban classification task show that the proposed approach can highly improve the classification accuracy w.r.t. conventional approaches working on a single scale.Comment: International Conference on Geographic Object-Based Image Analysis (GEOBIA 2016), University of Twente in Enschede, The Netherland

MODELS OF FOREST INVENTORY FOR ISTANBUL FOREST USING AIRBORNE LiDAR AND SPACEBORNE IMAGERY

Active remote sensing technology (LiDAR) and passive remote sensing technology (Pleiades and Göktürk-2 satellites) were used to find a meaningful relationship between ground data and remote sensing instruments for Istanbul Forest, Turkey. Two dominant species in the field, oak (deciduous trees) and maritime pine (coniferous trees), were researched. There were 86 plots total, 41 for maritime pine and 45 for oak. Three diameter at breast height (DBH) thresholds were studied. Trees of any DBH (DBH≥0.1 cm), trees ≥8 cm DBH thresholds and, trees ≥10 cm DBH thresholds. Both satellite image metrics were derived from Grey Level Co-occurrence Measures (GLCM). All metrics derived from satellite images and LiDAR data were incorporated into a hybrid approach. All metrics were separated and compared to each other to investigate how they are functioning separately. Linear regression, randomForest, and randomForest imputation models were used. The best R2 was 0.90 using three remote sensing instruments for tree counts based on the plot level for oak species. The highest % explained variances were 67.15% for tree count based on the plot level for oak species in randomForest model and 55.85% for tree count based on the plot level for oak species in randomForest Imputation. LiDAR data had a better relationship with ground data. Band 2 and band 4 of both satellite images were stronger predictors for deciduous trees. Band 3 and band 4 of both satellite images were used more for coniferous trees. Some of the most useful GLCM options were entropy for deciduous trees and correlation, variance and second moment for coniferous trees

Application of very high-resolution satellite imagery to identify individual tree crowns

Num contexto de crescentes mudanças mundiais, as áreas florestais tem sido alvo de pressões e intervenções antrópicas, como alterações climáticas, urbanização e crescimento demográfico, que continuam a provocar desflorestação e a degradação florestal a um ritmo alarmante. Por um lado, várias partes do mundo estão a ser vítimas de tendências acentuadas de desflorestação, grande parte das quais em florestas primárias de elevado valor para a conservação da biodiversidade e a regulação de serviços naturais essenciais. Por outro lado, em outras regiões, nomeadamente na Europa, a reflorestação está a ser promovida, precisamente como medida de apoio aos serviços e recursos naturais fornecidos pelas florestas. Neste contexto, é imperativo estabelecer a capacidade de, regularmente, monitorizar recursos e ecossistemas florestais. Por esta razão, a deteção remota tem se destacado como uma das poucas formas de monitorização, capaz de realizar observações contínuas e frequentes em diferentes escalas temporais e espaciais. No entanto, a eficácia destes métodos depende em grande parte da acessibilidade de amostras para treinar e definir o modelo. Se fosse possível reutilizar amostras de imagens previamente processadas para utilização em diferentes contextos ou períodos de tempo, o trabalho associado à recolha de novas amostras seria significativamente reduzido. No contexto da classificação de imagens, a transferibilidade de conhecimento tem este mesmo objetivo, adaptando um modelo treinado numa tarefa original para fazer previsões numa nova tarefa. Neste contexto, os principais objetivos deste trabalho consistem em: (1) avaliar a capacidade de algoritmos computacionais para a deteção de copas de árvores utilizando imagens de satélite de muito alta resolução; (2) determinar quais elementos das imagens maximizam a qualidade, fiabilidade e robustez dos resultados da classificação; (3) testar os algoritmos desenvolvidos com imagens de diferentes períodos de tempo e de diferentes regiões, de modo a avaliar a sua transferibilidade em termos de diferentes características temporais e espaciais. De maneira a alcançar estes objetivos, recorremos a uma base de imagens do satélite Pleiades, com 0,5 metros de resolução, na quais realizamos correção atmosférica e pan-sharpening. No âmbito deste projeto, foram analisadas dez áreas de interesse, de nove imagens. A maioria (quatro) dos locais situava-se em Espanha e dois em Portugal. Para executar os algoritmos de classificação, foi selecionada uma área de interesse para cada imagem, exceto em Salamanca, na qual foram selecionadas duas, com solo e vegetação diferentes, para testar a resposta do classificador. Na maioria dos locais, as árvores encontram-se num padrão regular, com muitas delas a possuírem copas pequenas (cerca de 1,0 m de diâmetro), enquanto outras eram mais maduras, com diâmetro de copa de aproximadamente 6-8 m. Num par de parcelas, as árvores não se encontravam num padrão regular, mas estavam dispersas. Durante as fases de teste, utilizando um custom API no ambiente QGIS, recorremos a dois métodos de classificação distintos, um supervisionado (Random Forest) e outro não supervisionado (Mosaic Clustering). O método Random Forest recorre a múltiplas árvores de decisão individuais, treinadas num subconjunto aleatório de dados de treino, em vez do conjunto de dados original. Por área de interesse, cerca de 200 a 300 destas amostras de treino foram recolhidas para construir o modelo, que foi depois aplicado a áreas independentes não envolvidas no treino. O método Mosaic Clustering divide a área de interesse em pequenas parcelas, de extensão definida pelo utilizador. Posteriormente, o algoritmo limitava-se a atribuir células a um de dois grupos, representando árvores e não árvores, com base em valores de reflectância semelhantes de cada banda e caraterística. Depois de cobrir toda a imagem, produz a classificação final. Adicionalmente, nas primeiras fases de teste, adicionamos nova informação aos algoritmos através de elementos como o NDVI, Circular Convolution, Adaptive Thresholding e medidas texturais. Numa última fase de teste, procurámos compreender qual seria a capacidade de transferibilidade do algoritmo, treinando e aplicando modelos em imagens com características diferentes. Adotámos, como primeiro passo, um método simplificado de transferibilidade de conhecimento não supervisionado, procurando perceber quais as principais falhas que advêm da não adaptação dos conjuntos de dados aos domínios em causa, de modo a, no futuro do projeto, decidir melhor quais os métodos estabelecidos de transferibilidade e adaptação de domínio a utilizar. Para testar a transferibilidade dos algoritmos, começámos por treinar os modelos em uma imagem de cada vez, utilizando todas as diferentes combinações de informação, e depois efetuámos a classificação, utilizando cada modelo individual, em todas as restantes imagens. De seguida, treinámos os modelos em todas as imagens ao mesmo tempo, exceto na que foi utilizada para o teste. Para realizar métodos de validação, dependemos de duas abordagens diferentes, uma estatística e uma qualitativa. A abordagem estatística envolve a utilização de amostras de validação. Mais concretamente, em cada imagem, 10% das amostras realizadas foram escolhidas aleatoriamente para servirem de amostras de validação. A abordagem qualitativa procura compreender melhor o significado prático dos resultados estatísticos e o impacto da inclusão de nova informação em elementos como o número de árvores detetadas e o tipo de problemas de classificação, encontrados em cada área de interesse. No que respeita ao primeiro objetivo, contámos manualmente o número de polígonos que foram identificados como copas de árvores, para cada combinação de informação, em cada imagem, e procedemos à comparação desses resultados com os resultados estatísticos recolhidos anteriormente. Para o segundo objetivo, avaliámos os principais problemas que se destacaram nos resultados da classificação, sendo estes (1) polígonos mal classificados; (2) árvores misturadas; (3) polígonos fragmentados. Foi utilizada uma abordagem de validação adicional nos testes de transferibilidade, baseada num algoritmo de árvore de decisão individual e superficial. Este algoritmo baseou-se no algoritmo anteriormente utilizado em todas as fases de teste, com a principal diferença de que se baseava apenas numa única árvore de decisão em vez de um conjunto de árvores. Esta abordagem permitiu-nos estimar as decisões que o algoritmo Random Forest estava a tomar e, com essa informação, comparar os limiares das classes definidos por cada modelo de treino com a distribuição de valores da informação das árvores detetadas na imagem prevista. Em primeiro lugar, os resultados mostram que o algoritmo é capaz de realizar deteção de copas de árvores, utilizando imagens de satélite de muito alta resolução. Além disso, também é possível estabelecer que a inclusão de informação adicional tem um impacto na melhoria dos resultados estatísticos, no número de copas de árvores detetados, e na redução dos problemas de classificação encontrados. Em segundo lugar, foi possível identificar a Circular Convolution e as medidas texturais como os dois elementos que mais impacto positivo tiveram nos resultados de classificação, enquanto o NDVI e o Adaptive Thresholding são as que menos melhoraram os resultados. Em terceiro lugar, verificámos que os algoritmos testados ainda não são capazes de ser aplicados num contexto de transferibilidade. A fim de cumprir os objectivos do projeto Life Terra, e resolver as questões acima mencionadas, três abordagens complementares podem ser adoptadas. Em primeiro lugar, alargar o conjunto de dados de treino incluindo novas informações de diferentes contextos geográficos garantiria que o modelo esteja apto a atuar em todos os locais possíveis. Em segundo lugar, e a fim de garantir que o algoritmo seja capaz de lidar com uma maior extensão geográfica e, subsequentemente, uma maior quantidade de informação de treino, pode-se justificar uma transição para Convoluted Neural Networks. Por último, a aplicação de métodos de adaptação ao domínio pode encurtar as lacunas entre os domínios de destino e de origem, assegurando que a informação pré-existente seja o mais semelhante possível, apesar das grandes diferenças que podem ocorrer entre os locais de plantação no âmbito do projeto Life Terra. Em resumo, este estudo permitiu melhorar a nossa compreensão da capacidade do algoritmo para efetuar a deteção de copas de árvores, dos fatores que influenciam o seu desempenho, e da sua capacidade para efetuar classificações de transferibilidade. Os seus resultados servirão de orientação para uma primeira versão de implementação a ser emitida para o projeto Life Terra que pretende fornecer aos utilizadores e gestores de ecossistemas informações detalhadas sobre as árvores que plantaram ou adotaram.While various parts of the world are witnessing deforestation trends in primary forests of high value for biodiversity conservation and regulation of essential natural services, other regions, particularly in Europe, are actively promoting reforestation and ecosystem restoration activities. For this reason, dynamic monitoring of forest areas has become an important part of resource management and ecosystem maintenance, largely reliant on remote sensing methods to efficiently oversee forested areas on a timely and consistent basis. The main objectives of this work consist of: (1) evaluating the capability of computational algorithms for tree crown detection using very-high resolution satellite images; (2) determining what features of the images maximise the quality, reliability and robustness of classification outputs; (3) testing the developed algorithms with images from different time periods and different regions, in order to assess their transferability in terms of different temporal and spatial characteristics. In order to achieve this, we used a set of images from the Pleiades Satellite, in which atmospheric corrections and pan-sharpening approaches were performed. Two classification methods were tested, namely the k-means, a non-supervised clustering algorithm, and random forest, a supervised classification algorithm. Results show that the Mosaic Clustering and, specifically, the Random Forest algorithms are promising approaches for tree crown detection, with the addition of new features positively impacting tree crown detection results. Moreover, the results show that the algorithm is not yet capable of performing consistent transferability classifications, due to the domain gaps between sites. In summary, this study was able to improve our understanding of the algorithm's capability to perform tree crown detection, of which factors impact its performance, and of its ability to perform transferability classifications. Its results will serve as a guideline for a first implementation version to be issued for the Life Terra project

Crown-level mapping of tree species and health from remote sensing of rural and urban forests

Tree species composition and health are key attributes for rural and urban forest biodiversity, and ecosystem services preservation. Remote sensing has facilitated extraordinary advances in estimating and mapping tree species composition and health. Yet previous sensors and algorithms were largely unable to resolve individual tree crowns and discriminate tree species or health classes at this essential spatial scale due to the low image spectral and spatial resolution. However, current available very high spatial resolution (VHR) remote sensing data can begin to resolve individual tree crowns and measure their spectral and structural qualities with unprecedented precision. Moreover, various machine learning algorithms are now available to apply these new data sources toward the discrimination and the mapping of tree species and health classes. The dissertation includes an introductory chapter, three stand-alone manuscripts, and a concluding chapter, each of which support the overarching theme of mapping tree species composition and health using remote sensing images. The first manuscript, now published in the International Journal of Remote Sensing, confirms the utility of combining VHR multi-temporal satellite data with LiDAR datasets for tree species classification using machine learning classifiers at the crown level in a rural forest the Fernow Experimental Forest, West Virginia. This research also evaluates the contribution of each type of spectral, phenological and structural feature for discriminating four tree species: red oak (Quercus rubra), sugar maple (Acer saccharum), tulip poplar (Liriodendron tulipifera), and black cherry (Prunus serotina). The second manuscript investigates the performance of tree species classification in urban settings with three contributions: 1) 12 very high resolution WorldView-3 images (WV-3), whose image acquisition date covering the growing season from April to November; 2) a large forest inventory providing sufficient calibration/validation datasets in Washington D.C.; 3) object-based tree species classification using the RandomForest machine learning algorithm. This manuscript identifies the incremental losses in classification accuracy caused by iteratively expanding the classification to 19 species and 10 genera. It also identifies the optimum pheno-phases and spectral bands for discriminating trees species in urban settings. Building on these promising results from the second manuscript, the third manuscript detect a signal of statistical difference among individual tree health conditions using WorldView-3 images from June 11th, July 30th and August 30th , 2017 in Washington D.C.. It examines six vegetation indices calculated from WorldView-3 images to describe three health condition levels in good, fair and poor, and discusses the effects of green-down phenology for tree health analysis. Overall, this dissertation research contributes to remote sensing research by combining data from both active and passive sensors to discriminate tree species in rural forest. For the species-rich urban settings, this dissertation illustrates the importance of phenology for tree species classification at crown level using VHR remote sensing images. Finally, this dissertation provides important insights on detecting statistical differences among tree health conditions at individual crown-level in the urban environment using VHR remote sensing images

OBIA for combining LiDAR and multispectral data to characterize forested areas and land cover in a tropical region

International audiencePrioritizing and designing forest restoration strategies requires an adequate survey to inform on the status (degraded or not) of forest types and the human disturbances over a territory. Very High Spatial Resolution (VHSR) remotely sensed data offers valuable information for performing such survey. We present in this study an OBIA methodology for mapping forest types at risk and land cover in a tropical context (Mayotte Island) combining LiDAR data (1 m pixel), VHSR multispectral images (Spot 5 XS 10 m pixel and orthophotos 0.5 m pixel) and ancillary data (existing thematic information). A Digital Canopy Model (DCM) was derived from LiDAR data and additional information was built from the DCM in order to better take into account the horizontal variability of canopy height: max and high Pass filters (3m x 3m kernel size) and Haralick variance texture image (51m x 51m kernel size). OBIA emerges as a suitable framework for exploiting multisource information during segmentation as well as during the classification process. A precise map (84% total accuracy) was obtained informing on (i) surfaces of forest types (defined according to their structure, i.e. canopy height of forest patches for specific type); (ii) degradation (identified in the heterogeneity of canopy height and presence of eroded areas) and (iii) human disturbances. Improvements can be made when discriminating forest types according to their composition (deciduous, evergreen or mixed), in particular by exploiting a more radiometrically homogenous VHSR multispectral image

Introducing GEOBIA to landscape imageability assessment: a multi-temporal case study of the nature reserve “Kózki”, Poland

Geographic object-based image analysis (GEOBIA) is a primary remote sensing tool utilized in land-cover mapping and change detection. Land-cover patches are the primary data source for landscape metrics and ecological indicator calculations; however, their application to visual landscape character (VLC) indicators was little investigated to date. To bridge the knowledge gap between GEOBIA and VLC, this paper puts forward the theoretical concept of using viewpoint as a landscape imageability indicator into the practice of a multi-temporal land-cover case study and explains how to interpret the indicator. The study extends the application of GEOBIA to visual landscape indicator calculations. In doing so, eight different remote sensing imageries are the object of GEOBIA, starting from a historical aerial photograph (1957) and CORONA declassified scene (1965) to contemporary (2018) UAV-delivered imagery. The multi-temporal GEOBIA-delivered land-cover patches are utilized to find the minimal isovist set of viewpoints and to calculate three imageability indicators: the number, density, and spacing of viewpoints. The calculated indicator values, viewpoint rank, and spatial arrangements allow us to describe the scale, direction, rate, and reasons for VLC changes over the analyzed 60 years of landscape evolution. We found that the case study nature reserve (“Kózki”, Poland) landscape imageability transformed from visually impressive openness to imageability due to the impression of several landscape rooms enclosed by forest walls. Our results provide proof that the number, rank, and spatial arrangement of viewpoints constitute landscape imageability measured with the proposed indicators. Discussing the method’s technical limitations, we believe that our findings contribute to a better understanding of land-cover change impact on visual landscape structure dynamics and further VLC indicator development

Artificial Neural Networks and Evolutionary Computation in Remote Sensing

Artificial neural networks (ANNs) and evolutionary computation methods have been successfully applied in remote sensing applications since they offer unique advantages for the analysis of remotely-sensed images. ANNs are effective in finding underlying relationships and structures within multidimensional datasets. Thanks to new sensors, we have images with more spectral bands at higher spatial resolutions, which clearly recall big data problems. For this purpose, evolutionary algorithms become the best solution for analysis. This book includes eleven high-quality papers, selected after a careful reviewing process, addressing current remote sensing problems. In the chapters of the book, superstructural optimization was suggested for the optimal design of feedforward neural networks, CNN networks were deployed for a nanosatellite payload to select images eligible for transmission to ground, a new weight feature value convolutional neural network (WFCNN) was applied for fine remote sensing image segmentation and extracting improved land-use information, mask regional-convolutional neural networks (Mask R-CNN) was employed for extracting valley fill faces, state-of-the-art convolutional neural network (CNN)-based object detection models were applied to automatically detect airplanes and ships in VHR satellite images, a coarse-to-fine detection strategy was employed to detect ships at different sizes, and a deep quadruplet network (DQN) was proposed for hyperspectral image classification

Estimating Solar Energy Production in Urban Areas for Electric Vehicles

Cities have a high potential for solar energy from PVs installed on buildings\u27 rooftops. There is an increased demand for solar energy in cities to reduce the negative effect of climate change. The thesis investigates solar energy potential in urban areas. It tries to determine how to detect and identify available rooftop areas, how to calculate suitable ones after excluding the effects of the shade, and the estimated energy generated from PVs. Geographic Information Sciences (GIS) and Remote Sensing (RS) are used in solar city planning. The goal of this research is to assess available and suitable rooftops areas using different GIS and RS techniques for installing PVs and estimating solar energy production for a sample of six compounds in New Cairo, and explore how to map urban areas on the city scale. In this research, the study area is the new Cairo city which has a high potential for harvesting solar energy, buildings in each compound have the same height, which does not cast shade on other buildings affecting PV efficiency. When applying GIS and RS techniques in New Cairo city, it is found that environmental factors - such as bare soil - affect the accuracy of the result, which reached 67% on the city scale. Researching more minor scales, such as compounds, required Very High Resolution (VHR) satellite images with a spatial resolution of up to 0.5 meter. The RS techniques applied in this research included supervised classification, and feature extraction, on Pleiades-1b VHR. On the compound scale, the accuracy assessment for the samples ranged between 74.6% and 96.875%. Estimating the PV energy production requires solar data; which was collected using a weather station and a pyrometer at the American University in Cairo, which is typical of the neighboring compounds in the new Cairo region. It took three years to collect the solar incidence data. The Hay- Devis, Klucher, and Reindl (HDKR) model is then employed to extrapolate the solar radiation measured on horizontal surfaces β =0°, to that on tilted surfaces with inclination angles β =10°, 20°, 30° and 45°. The calculated (with help of GIS and Solar radiation models) net rooftop area available for capturing solar radiation was determined for sample New Cairo compounds . The available rooftop areas were subject to the restriction that all the PVs would be coplanar, none of the PVs would protrude outside the rooftop boundaries, and no shading of PVs would occur at any time of the year; moreover typical other rooftop occupied areas, and actual dimensions of typical roof top PVs were taken into consideration. From those calculations, both the realistic total annual Electrical energy produced by the PVs and their daily monthly energy produced are deduced. The former is relevant if the PVs are tied to a grid, whereas the other is more relevant if it is not; optimization is different for both. Results were extended to estimate the total number of cars that may be driven off PV converted solar radiation per home, for different scenarios