Learning Aerial Image Segmentation from Online Maps

This study deals with semantic segmentation of high-resolution (aerial) images where a semantic class label is assigned to each pixel via supervised classification as a basis for automatic map generation. Recently, deep convolutional neural networks (CNNs) have shown impressive performance and have quickly become the de-facto standard for semantic segmentation, with the added benefit that task-specific feature design is no longer necessary. However, a major downside of deep learning methods is that they are extremely data-hungry, thus aggravating the perennial bottleneck of supervised classification, to obtain enough annotated training data. On the other hand, it has been observed that they are rather robust against noise in the training labels. This opens up the intriguing possibility to avoid annotating huge amounts of training data, and instead train the classifier from existing legacy data or crowd-sourced maps which can exhibit high levels of noise. The question addressed in this paper is: can training with large-scale, publicly available labels replace a substantial part of the manual labeling effort and still achieve sufficient performance? Such data will inevitably contain a significant portion of errors, but in return virtually unlimited quantities of it are available in larger parts of the world. We adapt a state-of-the-art CNN architecture for semantic segmentation of buildings and roads in aerial images, and compare its performance when using different training data sets, ranging from manually labeled, pixel-accurate ground truth of the same city to automatic training data derived from OpenStreetMap data from distant locations. We report our results that indicate that satisfying performance can be obtained with significantly less manual annotation effort, by exploiting noisy large-scale training data.Comment: Published in IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSIN

The role of earth observation in an integrated deprived area mapping “system” for low-to-middle income countries

Urbanization in the global South has been accompanied by the proliferation of vast informal and marginalized urban areas that lack access to essential services and infrastructure. UN-Habitat estimates that close to a billion people currently live in these deprived and informal urban settlements, generally grouped under the term of urban slums. Two major knowledge gaps undermine the efforts to monitor progress towards the corresponding sustainable development goal (i.e., SDG 11—Sustainable Cities and Communities). First, the data available for cities worldwide is patchy and insufficient to differentiate between the diversity of urban areas with respect to their access to essential services and their specific infrastructure needs. Second, existing approaches used to map deprived areas (i.e., aggregated household data, Earth observation (EO), and community-driven data collection) are mostly siloed, and, individually, they often lack transferability and scalability and fail to include the opinions of different interest groups. In particular, EO-based-deprived area mapping approaches are mostly top-down, with very little attention given to ground information and interaction with urban communities and stakeholders. Existing top-down methods should be complemented with bottom-up approaches to produce routinely updated, accurate, and timely deprived area maps. In this review, we first assess the strengths and limitations of existing deprived area mapping methods. We then propose an Integrated Deprived Area Mapping System (IDeAMapS) framework that leverages the strengths of EO- and community-based approaches. The proposed framework offers a way forward to map deprived areas globally, routinely, and with maximum accuracy to support SDG 11 monitoring and the needs of different interest groups

CNN기반의 FusionNet 신경망과 농지 경계추출 알고리즘을 이용한 토지피복분류모델 개발

학위논문 (석사) -- 서울대학교 대학원 : 농업생명과학대학 생태조경.지역시스템공학부(지역시스템공학전공), 2021. 2. 송인홍.토지이용이 빠르게 변화함에 따라, 토지 피복에 대한 공간정보를 담고 있는 토지 피복 지도의 신속한 최신화는 필수적이다. 하지만, 현 토지 피복 지도는 많은 시간과 노동력을 요구하는 manual digitizing 방법으로 제작됨에 따라, 토지 피복 지도의 업데이트 및 배포에 긴 시간 간격이 발생하는 실정이다. 본 연구에서는 convolutional neural network (CNN) 기반의 인공신경망을 이용하여 high-resolution remote sensing (HRRS) 영상으로부터 토지 피복을 분류하는 모델을 개발하고, 특히 농지 경계추출 알고리즘을 적용하여 농업지역에서 분류 정확도를 개선하고자 하였다. 개발된 토지 피복 분류모델은 전처리(pre-processing) 모듈, 토지 피복 분류(land cover classification) 모듈, 그리고 후처리(post-processing) 모듈의 세 모듈로 구성된다. 전처리 모듈은 입력된 HRRS 영상을 75%씩 중첩 분할하여 관점을 다양화하는 모듈로, 한 관점에서 토지 피복을 분류할 때 발생할 수 있는 오분류를 줄이고자 하였다. 토지 피복 분류 모듈은 FusionNet model 구조를 바탕으로 개발되었고, 이는 분할된 HRRS 이미지의 픽셀별로 최적 토지 피복을 부여하도록 설계되었다. 후처리 모듈은 픽셀별 최종 토지 피복을 결정하는 모듈로, 분할된 HRRS 이미지의 분류결과를 취합하여 최빈값을 최종 토지 피복으로 결정한다. 추가로 농지에서는 농지경계를 추출하고, 필지별 분류된 토지 피복을 집계하여 한 필지에 같은 토지 피복을 부여하였다. 개발된 토지 피복 분류모델은 전라남도 지역(면적: 547 km2)의 2018년 정사영상과 토지 피복 지도를 이용하여 학습되었다. 토지 피복 분류모델 검증은 학습지역과 시간, 공간적으로 구분된, 2018년 전라남도 수북면과 2016년 충청북도 대소면의 두 검증지역에서 수행되었다. 각 검증지역에서 overall accuracy는 0.81, 0.71로 집계되었고, kappa coefficients는 0.75, 0.64로 산정되어 substantial 수준의 토지 피복 분류 정확도를 확인하였다. 특히, 개발된 모델은 필지 경계를 고려한 농업지역에서 overall accuracy 0.89, kappa coefficient 0.81로 almost perfect 수준의 우수한 분류 정확도를 보였다. 이에 개발된 토지 피복 분류모델은 특히 농업지역에서 현 토지 피복 분류 방법을 지원하여 토지 피복 지도의 빠르고 정확한 최신화에 기여할 수 있을 것으로 생각된다.The rapid update of land cover maps is necessary because spatial information of land cover is widely used in various areas. However, these maps have been released or updated in the interval of several years primarily owing to the manual digitizing method, which is time-consuming and labor-intensive. This study was aimed to develop a land cover classification model using the concept of a convolutional neural network (CNN) that classifies land cover labels from high-resolution remote sensing (HRRS) images and to increase the classification accuracy in agricultural areas using the parcel boundary extraction algorithm. The developed model comprises three modules, namely the pre-processing, land cover classification, and post-processing modules. The pre-processing module diversifies the perspective of the HRRS images by separating images with 75% overlaps to reduce the misclassification that can occur in a single image. The land cover classification module was designed based on the FusionNet model structure, and the optimal land cover type was assigned for each pixel of the separated HRRS images. The post-processing module determines the ultimate land cover types for each pixel unit by summing up the several-perspective classification results and aggregating the pixel-classification result for the parcel-boundary unit in agricultural areas. The developed model was trained with land cover maps and orthographic images (area: 547 km2) from the Jeonnam province in Korea. Model validation was conducted with two spatially and temporally different sites including Subuk-myeon of Jeonnam province in 2018 and Daseo-myeon of Chungbuk province in 2016. In the respective validation sites, the models overall accuracies were 0.81 and 0.71, and kappa coefficients were 0.75 and 0.64, implying substantial model performance. The model performance was particularly better when considering parcel boundaries in agricultural areas, exhibiting an overall accuracy of 0.89 and kappa coefficient 0.81 (almost perfect). It was concluded that the developed model may help perform rapid and accurate land cover updates especially for agricultural areas.Chapter 1. Introduction 1 1.1. Study background 1 1.2. Objective of thesis 4 Chapter 2. Literature review 6 2.1. Development of remote sensing technique 6 2.2. Land cover segmentation 9 2.3. Land boundary extraction 13 Chapter 3. Development of the land cover classification model 15 3.1. Conceptual structure of the land cover classification model 15 3.2. Pre-processing module 16 3.3. CNN based land cover classification module 17 3.4. Post processing module 22 3.4.1 Determination of land cover in a pixel unit 22 3.4.2 Aggregation of land cover to parcel boundary 24 Chapter 4. Verification of the land cover classification model 30 4.1. Study area and data acquisition 31 4.1.1. Training area 31 4.1.2. Verification area 32 4.1.3. Data acquisition 33 4.2. Training the land cover classification model 36 4.3. Verification method 37 4.3.1. The performance measurement methods of land cover classification model 37 4.3.2. Accuracy estimation methods of agricultural parcel boundary 39 4.3.3. Comparison of boundary based classification result with ERDAS Imagine 41 4.4. Verification of land cover classification model 42 4.4.1. Performance of land cover classification at the child subcategory 42 4.4.2. Classification accuracy of the aggregated land cover to main category 46 4.4.3. Classification accuracy of boundary based aggregation in agricultural area 57 Chapter 5. Conclusions 71 Reference 73 국 문 초 록 83Maste

Large-Scale Mapping of Human Activity using Geo-Tagged Videos

This paper is the first work to perform spatio-temporal mapping of human activity using the visual content of geo-tagged videos. We utilize a recent deep-learning based video analysis framework, termed hidden two-stream networks, to recognize a range of activities in YouTube videos. This framework is efficient and can run in real time or faster which is important for recognizing events as they occur in streaming video or for reducing latency in analyzing already captured video. This is, in turn, important for using video in smart-city applications. We perform a series of experiments to show our approach is able to accurately map activities both spatially and temporally. We also demonstrate the advantages of using the visual content over the tags/titles.Comment: Accepted at ACM SIGSPATIAL 201

Automated extraction of water bodies from NIR and RGB aerial imagery in northern Alaska using supervised and unsupervised machine learning techniques

Thawing and freezing of permafrost ground are affected by various reasons: air temperature, vegetation, snow accumulation, subsurface physical properties, and moisture. Due to the rising of air temperature, the permafrost temperature and the thermokarst activity increase. Thermokarst instability causes an imbalance for the hydrology system, topography, soils, sediment and nutrient cycle to lakes and streams. Hence the lakes and ponds are ubiquitous in permafrost region. The plants and animals fulfil their nutrient needs from water in the environment. Other animals acquire their needs from the plants and animals that they consume. Therefore the influence of degradation of lakes and ponds strongly affect biogeochemical cycles. This research aims to implement an automated workflow to map the water bodies caused by permafrost thawing. The scientific challenge is to test the machine learning techniques adaptability to assist the observation and mapping of the water bodies using aerial imagery. The study area is mainly located in northern Alaska and consists of five different locations: Ikpikpuk, Teschekpuk Central, Teshekpuk East, Tesheckpuk West, Meade East, and Meade West. To estimate the degradation of the high centred polygons distribution and potential degradation of ice wedges, I mapped the polygonal terrain and ice-wedge melt ponds using areal photogrammetry data of NIR and RGB bands captured by Thaw Trend Air 2019 flight campaign. The techniques used are unsupervised K-mean classification, supervised segment mean shift, and supervised random forest classification to model the water polygons from airborne photogrammetry. There are two phases to perform the machine learning classification; the first step is to test the accuracy of each technique and get to a conclusion about the most adapted method. The second is to prepare the Orthomosaic data, run the chosen workflow, and visualize the final results. The morphology filter with opening option application and clean boundary filters are practical before classification as they sharpen the image features. The conclusion is to use the Random forest classification as it was helpful in all NIR Orthomosaics; however, the RGB images required downsampling to provide adequate accuracy

Spatially adaptive semi‐supervised learning with Gaussian processes for hyperspectral data analysis

This paper presents a semi‐supervised learning algorithm called Gaussian process expectation‐maximization (GP‐EM), for classification of landcover based on hyperspectral data analysis. Model parameters for each land cover class are first estimated by a supervised algorithm using Gaussian process regressions to find spatially adaptive parameters, and the estimated parameters are then used to initialize a spatially adaptive mixture‐of‐Gaussians model. The mixture model is updated by expectation‐maximization iterations using the unlabeled data, and the spatially adaptive parameters for unlabeled instances are obtained by Gaussian process regressions with soft assignments. Spatially and temporally distant hyperspectral images taken from the Botswana area by the NASA EO‐1 satellite are used for experiments. Detailed empirical evaluations show that the proposed framework performs significantly better than all previously reported results by a wide variety of alternative approaches and algorithms on the same datasets. © 2011 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 4: 358–371, 2011Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/87150/1/10119_ftp.pd

A Comparison of the Classification of Vegetation Characteristics by Spectral Mixture Analysis and Standard Classifiers on Remotely Sensed Imagery within the Siberia Region

As an alternative to the traditional method of inferring vegetation cover characteristics from satellite data by classifying each pixel into a specific land cover type based on predefined classification schemes, the Spectral Mixture Analysis (SMA) method is applied to images of the Siberia region. A linear mixture model was applied to determine proportional estimates of land cover for, (a) agriculture and floodplain soils, (b) broadleaf, and (c) conifer classes, in pixels of 30 m resolution Landsat data. In order to evaluate the areal estimates, results were compared with ground truth data, as well as those estimates derived from more sophisticated method of image classification, providing improved estimates of endmember values and subpixel areal estimates of vegetation cover classes than the traditional approach of using predefined classification schemes with discrete numbers of cover types. This technique enables the estimation of proportional land cover type in a single pixel and could potentially serve as a tool for deriving improved estimates of vegetation parameters that are necessary for modeling carbon processes

Assessing the role of EO in biodiversity monitoring: options for integrating in-situ observations with EO within the context of the EBONE concept

The European Biodiversity Observation Network (EBONE) is a European contribution on terrestrial monitoring to GEO BON, the Group on Earth Observations Biodiversity Observation Network. EBONE’s aims are to develop a system of biodiversity observation at regional, national and European levels by assessing existing approaches in terms of their validity and applicability starting in Europe, then expanding to regions in Africa. The objective of EBONE is to deliver: 1. A sound scientific basis for the production of statistical estimates of stock and change of key indicators; 2. The development of a system for estimating past changes and forecasting and testing policy options and management strategies for threatened ecosystems and species; 3. A proposal for a cost-effective biodiversity monitoring system. There is a consensus that Earth Observation (EO) has a role to play in monitoring biodiversity. With its capacity to observe detailed spatial patterns and variability across large areas at regular intervals, our instinct suggests that EO could deliver the type of spatial and temporal coverage that is beyond reach with in-situ efforts. Furthermore, when considering the emerging networks of in-situ observations, the prospect of enhancing the quality of the information whilst reducing cost through integration is compelling. This report gives a realistic assessment of the role of EO in biodiversity monitoring and the options for integrating in-situ observations with EO within the context of the EBONE concept (cfr. EBONE-ID1.4). The assessment is mainly based on a set of targeted pilot studies. Building on this assessment, the report then presents a series of recommendations on the best options for using EO in an effective, consistent and sustainable biodiversity monitoring scheme. The issues that we faced were many: 1. Integration can be interpreted in different ways. One possible interpretation is: the combined use of independent data sets to deliver a different but improved data set; another is: the use of one data set to complement another dataset. 2. The targeted improvement will vary with stakeholder group: some will seek for more efficiency, others for more reliable estimates (accuracy and/or precision); others for more detail in space and/or time or more of everything. 3. Integration requires a link between the datasets (EO and in-situ). The strength of the link between reflected electromagnetic radiation and the habitats and their biodiversity observed in-situ is function of many variables, for example: the spatial scale of the observations; timing of the observations; the adopted nomenclature for classification; the complexity of the landscape in terms of composition, spatial structure and the physical environment; the habitat and land cover types under consideration. 4. The type of the EO data available varies (function of e.g. budget, size and location of region, cloudiness, national and/or international investment in airborne campaigns or space technology) which determines its capability to deliver the required output. EO and in-situ could be combined in different ways, depending on the type of integration we wanted to achieve and the targeted improvement. We aimed for an improvement in accuracy (i.e. the reduction in error of our indicator estimate calculated for an environmental zone). Furthermore, EO would also provide the spatial patterns for correlated in-situ data. EBONE in its initial development, focused on three main indicators covering: (i) the extent and change of habitats of European interest in the context of a general habitat assessment; (ii) abundance and distribution of selected species (birds, butterflies and plants); and (iii) fragmentation of natural and semi-natural areas. For habitat extent, we decided that it did not matter how in-situ was integrated with EO as long as we could demonstrate that acceptable accuracies could be achieved and the precision could consistently be improved. The nomenclature used to map habitats in-situ was the General Habitat Classification. We considered the following options where the EO and in-situ play different roles: using in-situ samples to re-calibrate a habitat map independently derived from EO; improving the accuracy of in-situ sampled habitat statistics, by post-stratification with correlated EO data; and using in-situ samples to train the classification of EO data into habitat types where the EO data delivers full coverage or a larger number of samples. For some of the above cases we also considered the impact that the sampling strategy employed to deliver the samples would have on the accuracy and precision achieved. Restricted access to European wide species data prevented work on the indicator ‘abundance and distribution of species’. With respect to the indicator ‘fragmentation’, we investigated ways of delivering EO derived measures of habitat patterns that are meaningful to sampled in-situ observations