Review on Active and Passive Remote Sensing Techniques for Road Extraction

Digital maps of road networks are a vital part of digital cities and intelligent transportation. In this paper, we provide a comprehensive review on road extraction based on various remote sensing data sources, including high-resolution images, hyperspectral images, synthetic aperture radar images, and light detection and ranging. This review is divided into three parts. Part 1 provides an overview of the existing data acquisition techniques for road extraction, including data acquisition methods, typical sensors, application status, and prospects. Part 2 underlines the main road extraction methods based on four data sources. In this section, road extraction methods based on different data sources are described and analysed in detail. Part 3 presents the combined application of multisource data for road extraction. Evidently, different data acquisition techniques have unique advantages, and the combination of multiple sources can improve the accuracy of road extraction. The main aim of this review is to provide a comprehensive reference for research on existing road extraction technologies.Peer reviewe

Optimization of Rooftop Delineation from Aerial Imagery with Deep Learning

High-definition (HD) maps of building rooftops or footprints are important for urban application and disaster management. Rapid creation of such HD maps through rooftop delineation at the city scale using high-resolution satellite and aerial images with deep leaning methods has become feasible and draw much attention. In the context of rooftop delineation, the end-to-end Deep Convolutional Neural Networks (DCNNs) have demonstrated remarkable performance in accurately delineating rooftops from aerial imagery. However, several challenges still exist in this task, which are addressed in this thesis. These challenges include: (1) the generalization issues of models when test data differ from training data, (2) the scale-variance issues in rooftop delineation, and (3) the high cost of annotating accurate rooftop boundaries. To address the challenges mentioned above, this thesis proposes three novel deep learning-based methods. Firstly, a super-resolution network named Momentum and Spatial-Channel Attention Residual Feature Aggregation Network (MSCA-RFANet) is proposed to tackle the generalization issue. The proposed super-resolution network shows better performance compared to its baseline and other state-of-the-art methods. In addition, data composition with MSCA-RFANet shows high performance on dealing with the generalization issues. Secondly, an end-to-end rooftop delineation network named Higher Resolution Network with Dynamic Scale Training (HigherNet-DST) is developed to mitigate the scale-variance issue. The experimental results on publicly available building datasets demonstrate that HigherNet-DST achieves competitive performance in rooftop delineation, particularly excelling in accurately delineating small buildings. Lastly, a weakly supervised deep learning network named Box2Boundary is developed to reduce the annotation cost. The experimental results show that Box2Boundary with post processing is effective in dealing with the cost annotation issues with decent performance. Consequently, the research with these three sub-topics and the three resulting papers are thought to hold potential implications for various practical applications

Very High Resolution (VHR) Satellite Imagery: Processing and Applications

Recently, growing interest in the use of remote sensing imagery has appeared to provide synoptic maps of water quality parameters in coastal and inner water ecosystems;, monitoring of complex land ecosystems for biodiversity conservation; precision agriculture for the management of soils, crops, and pests; urban planning; disaster monitoring, etc. However, for these maps to achieve their full potential, it is important to engage in periodic monitoring and analysis of multi-temporal changes. In this context, very high resolution (VHR) satellite-based optical, infrared, and radar imaging instruments provide reliable information to implement spatially-based conservation actions. Moreover, they enable observations of parameters of our environment at greater broader spatial and finer temporal scales than those allowed through field observation alone. In this sense, recent very high resolution satellite technologies and image processing algorithms present the opportunity to develop quantitative techniques that have the potential to improve upon traditional techniques in terms of cost, mapping fidelity, and objectivity. Typical applications include multi-temporal classification, recognition and tracking of specific patterns, multisensor data fusion, analysis of land/marine ecosystem processes and environment monitoring, etc. This book aims to collect new developments, methodologies, and applications of very high resolution satellite data for remote sensing. The works selected provide to the research community the most recent advances on all aspects of VHR satellite remote sensing

Recent Advances in Image Restoration with Applications to Real World Problems

In the past few decades, imaging hardware has improved tremendously in terms of resolution, making widespread usage of images in many diverse applications on Earth and planetary missions. However, practical issues associated with image acquisition are still affecting image quality. Some of these issues such as blurring, measurement noise, mosaicing artifacts, low spatial or spectral resolution, etc. can seriously affect the accuracy of the aforementioned applications. This book intends to provide the reader with a glimpse of the latest developments and recent advances in image restoration, which includes image super-resolution, image fusion to enhance spatial, spectral resolution, and temporal resolutions, and the generation of synthetic images using deep learning techniques. Some practical applications are also included

A methodology to produce geographical information for land planning using very-high resolution images

Actualmente, os municípios são obrigados a produzir, no âmbito da elaboração dos instrumentos de gestão territorial, cartografia homologada pela autoridade nacional. O Plano Director Municipal (PDM) tem um período de vigência de 10 anos. Porém, no que diz respeito à cartografia para estes planos, principalmente em municípios onde a pressão urbanística é elevada, esta periodicidade não é compatível com a dinâmica de alteração de uso do solo. Emerge assim, a necessidade de um processo de produção mais eficaz, que permita a obtenção de uma nova cartografia de base e temática mais frequentemente. Em Portugal recorre-se à fotografia aérea como informação de base para a produção de cartografia de grande escala. Por um lado, embora este suporte de informação resulte em mapas bastante rigorosos e detalhados, a sua produção têm custos muito elevados e consomem muito tempo. As imagens de satélite de muito alta-resolução espacial podem constituir uma alternativa, mas sem substituir as fotografias aéreas na produção de cartografia temática, a grande escala. O tema da tese trata assim da satisfação das necessidades municipais em informação geográfica actualizada. Para melhor conhecer o valor e utilidade desta informação, realizou-se um inquérito aos municípios Portugueses. Este passo foi essencial para avaliar a pertinência e a utilidade da introdução de imagens de satélite de muito alta-resolução espacial na cadeia de procedimentos de actualização de alguns temas, quer na cartografia de base quer na cartografia temática. A abordagem proposta para solução do problema identificado baseia-se no uso de imagens de satélite e outros dados digitais em ambiente de Sistemas de Informação Geográfica. A experimentação teve como objectivo a extracção automática de elementos de interesse municipal a partir de imagens de muito alta-resolução espacial (fotografias aéreas ortorectificadas, imagem QuickBird, e imagem IKONOS), bem como de dados altimétricos (dados LiDAR). Avaliaram-se as potencialidades da informação geográfica extraídas das imagens para fins cartográficos e analíticos. Desenvolveram-se quatro casos de estudo que reflectem diferentes usos para os dados geográficos a nível municipal, e que traduzem aplicações com exigências diferentes. No primeiro caso de estudo, propõe-se uma metodologia para actualização periódica de cartografia a grande escala, que faz uso de fotografias aéreas vi ortorectificadas na área da Alta de Lisboa. Esta é uma aplicação quantitativa onde as qualidades posicionais e geométricas dos elementos extraídos são mais exigentes. No segundo caso de estudo, criou-se um sistema de alarme para áreas potencialmente alteradas, com recurso a uma imagem QuickBird e dados LiDAR, no Bairro da Madre de Deus, com objectivo de auxiliar a actualização de cartografia de grande escala. No terceiro caso de estudo avaliou-se o potencial solar de topos de edifícios nas Avenidas Novas, com recurso a dados LiDAR. No quarto caso de estudo, propõe-se uma série de indicadores municipais de monitorização territorial, obtidos pelo processamento de uma imagem IKONOS que cobre toda a área do concelho de Lisboa. Esta é uma aplicação com fins analíticos onde a qualidade temática da extracção é mais relevante.Currently, the Portuguese municipalities are required to produce homologated cartography, under the Territorial Management Instruments framework. The Municipal Master Plan (PDM) has to be revised every 10 years, as well as the topographic and thematic maps that describe the municipal territory. However, this period is inadequate for representing counties where urban pressure is high, and where the changes in the land use are very dynamic. Consequently, emerges the need for a more efficient mapping process, allowing obtaining recent geographic information more often. Several countries, including Portugal, continue to use aerial photography for large-scale mapping. Although this data enables highly accurate maps, its acquisition and visual interpretation are very costly and time consuming. Very-High Resolution (VHR) satellite imagery can be an alternative data source, without replacing the aerial images, for producing large-scale thematic cartography. The focus of the thesis is the demand for updated geographic information in the land planning process. To better understand the value and usefulness of this information, a survey of all Portuguese municipalities was carried out. This step was essential for assessing the relevance and usefulness of the introduction of VHR satellite imagery in the chain of procedures for updating land information. The proposed methodology is based on the use of VHR satellite imagery, and other digital data, in a Geographic Information Systems (GIS) environment. Different algorithms for feature extraction that take into account the variation in texture, color and shape of objects in the image, were tested. The trials aimed for automatic extraction of features of municipal interest, based on aerial and satellite high-resolution (orthophotos, QuickBird and IKONOS imagery) as well as elevation data (altimetric information and LiDAR data). To evaluate the potential of geographic information extracted from VHR images, two areas of application were identified: mapping and analytical purposes. Four case studies that reflect different uses of geographic data at the municipal level, with different accuracy requirements, were considered. The first case study presents a methodology for periodic updating of large-scale maps based on orthophotos, in the area of Alta de Lisboa. This is a situation where the positional and geometric accuracy of the extracted information are more demanding, since technical mapping standards must be complied. In the second case study, an alarm system that indicates the location of potential changes in building areas, using a QuickBird image and LiDAR data, was developed for the area of Bairro da Madre de Deus. The goal of the system is to assist the updating of large scale mapping, providing a layer that can be used by the municipal technicians as the basis for manual editing. In the third case study, the analysis of the most suitable roof-tops for installing solar systems, using LiDAR data, was performed in the area of Avenidas Novas. A set of urban environment indicators obtained from VHR imagery is presented. The concept is demonstrated for the entire city of Lisbon, through IKONOS imagery processing. In this analytical application, the positional quality issue of extraction is less relevant.GEOSAT – Methodologies to extract large scale GEOgraphical information from very high resolution SATellite images (PTDC/GEO/64826/2006), e-GEO – Centro de Estudos de Geografia e Planeamento Regional, da Faculdade de Ciências Sociais e Humanas, no quadro do Grupo de Investigação Modelação Geográfica, Cidades e Ordenamento do Territóri

An integrative approach using remote sensing and social analysis to identify different settlement types and the specific living conditions of its inhabitants

Someday in 2007, the world population reached a historical landmark: for the first time in human history, more than half of the world´s population was urban. A stagnation of this urbanization process is not in sight, so that by 2050, already 70 percent of humankind is projected to live in urban settlements. Over the last few decades, enormous migrations from rural hinterlands to steadily growing cities could be witnessed coming along with a dramatic growth of the world’s urban population. The speed and the scale of this growth, particularly in the so called less developed regions, are posing tremendous challenges to the countries concerned as well as to the world community. Within mega cities the strongest trends and the most extreme dimensions of the urbanization process can be observed. Their rapid growth results in uncontrolled processes of fragmentation which is often associated with pronounced poverty, social inequality, socio-spatial and political fragmentation, environmental degradation as well as population demands that outstrip environmental service capacity. For the majority of the mega cities a tremendous increase of informal structures and processes has to be observed. Consequentially informal settlements are growing, which represent those characteristic municipal areas being subject to particularly high population density, dynamics as well as marginalization. They have quickly become the most visible expression of urban poverty in developing world cities. Due to the extreme dynamics, the high complexity and huge spatial dimension of mega cities, urban administrations often only have an obsolete or not even existing data basis available to be at all informed about developments, trends and dimensions of urban growth and change. The knowledge about the living conditions of the residents is correspondingly very limited, incomplete and not up to date. Traditional methods such as statistical and regional analyses or fieldwork are no longer capable to capture such urban process. New data sources and monitoring methodologies are required in order to provide an up to date information basis as well as planning strate¬gies to enable sustainable developments and to simplify planning processes in complex urban structures. This research shall seize the described problem and aims to make a contribution to the requirements of monitoring fast developing mega cities. Against this background a methodology is developed to compensate the lack of socio-economic data and to deduce meaningful information on the living conditions of the inhabitants of mega cities. Neither social science methods alone nor the exclusive analysis of remote sensing data can solve the problem of the poor quality and outdated data base. Conventional social science methods cannot cope with the enormous developments and the tremendous growth as they are too labor-, as well as too time- and too cost-intensive. On the other hand, the physical discipline of remote sensing does not allow for direct conclusions on social parameters out of remote sensing images. The prime objective of this research is therefore the development of an integrative approach − bridging remote sensing and social analysis – in order to derive useful information about the living conditions in this specific case of the mega city Delhi and its inhabitants. Hence, this work is established in the overlapping range of the research topics remote sensing, urban areas and social science. Delhi, as India’s fast growing capital, meanwhile with almost 25 million residents the second largest city of the world, represents a prime example of a mega city. Since the second half of the 20th century, Delhi has been transformed from a modest town with mainly administrative and trade-related functions to a complex metropolis with a steep socio-economic gradient. The quality and amount of administrative and socio-economic data are poor and the knowledge about the circumstances of Delhi’s residents is correspondingly insufficient and outdated. Delhi represents therefore a perfectly suited study area for this research. In order to gather information about the living conditions within the different settlement types a methodology was developed and conducted to analyze the urban environment of the mega city Delhi. To identify different settlement types within the urban area, regarding the complex and heterogeneous appearance of the Delhi area, a semi-automated, object-oriented classification approach, based on segmentation derived image objects, was implemented. As the complete conceptual framework of this research, the classification methodology was developed based on a smaller representative training area at first and applied to larger test sites within Delhi afterwards. The object-oriented classification of VHR satellite imagery of the QuickBird sensor allowed for the identification of five different urban land cover classes within the municipal area of Delhi. In the focus of the image analysis is yet the identification of different settlement types and amongst these of informal settlements in particular. The results presented within this study demonstrate, that, based on density classes, the developed methodology is suitable to identify different settlement types and to detect informal settlements which are mega urban risk areas and thus potential residential zones of vulnerable population groups. The remote sensing derived land cover maps form the foundation for the integrative analysis concept and deliver there¬fore the general basis for the derivation of social attributes out of remote sensing data. For this purpose settlement characteristics (e.g., area of the settlement, average building size, and number of houses) are estimated from the classified QuickBird data and used to derive spatial information about the population distribution. In a next step, the derived information is combined with in-situ information on socio-economic conditions (e.g., family size, mean water consumption per capita/family) extracted from georeferenced questionnaires conducted during two field trips in Delhi. This combined data is used to characterize a given settlement type in terms of specific population and water related variables (e.g., population density, total water consumption). With this integrative methodology a catalogue can be compiled, comprising the living conditions of Delhi’s inhabitants living in specific settlement structures – and this in a quick, large-scaled, cost effective, by random or regularly repeatable way with a relatively small required data basis.The combined application of remotely sensed imagery and socio-economic data allows for the mapping, capturing and characterizing the socio-economic structures and dynamics within the mega city of Delhi, as well as it establishes a basis for the monitoring of the mega city of Delhi or certain areas within the city respectively by remote sensing. The opportunity to capture the condition of a mega city and to monitor its development in general enables the persons in charge to identify unbeneficial trends and to intervene accordingly from an urban planning perspective and to countersteer against a non-adequate supply of the inhabitants of different urban districts, primarily of those of informal settlements. This study is understood to be a first step to the development of methods which will help to identify and understand the different forms, actors and processes of urbanization in mega cities. It could support a more proactive and sustainable urban planning and land management – which in turn will increase the importance of urban remote sensing techniques. In this regard, the most obvious and direct beneficiaries are on the one hand the governmental agencies and urban planners and on the other hand, and which is possibly the most important goal, the inhabitants of the affected areas, whose living conditions can be monitored and improved as required. Only if the urban monitoring is quickly, inexpensively and easily available, it will be accepted and applied by the authorities, which in turn enables for the poorest to get the support they need. All in all, the listed benefits are very convincing and corroborate the combined use of remotely sensed and socio-economic data in mega city research

Deep learning for land cover and land use classification

Recent advances in sensor technologies have witnessed a vast amount of very fine spatial resolution (VFSR) remotely sensed imagery being collected on a daily basis. These VFSR images present fine spatial details that are spectrally and spatially complicated, thus posing huge challenges in automatic land cover (LC) and land use (LU) classification. Deep learning reignited the pursuit of artificial intelligence towards a general purpose machine to be able to perform any human-related tasks in an automated fashion. This is largely driven by the wave of excitement in deep machine learning to model the high-level abstractions through hierarchical feature representations without human-designed features or rules, which demonstrates great potential in identifying and characterising LC and LU patterns from VFSR imagery. In this thesis, a set of novel deep learning methods are developed for LC and LU image classification based on the deep convolutional neural networks (CNN) as an example. Several difficulties, however, are encountered when trying to apply the standard pixel-wise CNN for LC and LU classification using VFSR images, including geometric distortions, boundary uncertainties and huge computational redundancy. These technical challenges for LC classification were solved either using rule-based decision fusion or through uncertainty modelling using rough set theory. For land use, an object-based CNN method was proposed, in which each segmented object (a group of homogeneous pixels) was sampled and predicted by CNN with both within-object and between-object information. LU was, thus, classified with high accuracy and efficiency. Both LC and LU formulate a hierarchical ontology at the same geographical space, and such representations are modelled by their joint distribution, in which LC and LU are classified simultaneously through iteration. These developed deep learning techniques achieved by far the highest classification accuracy for both LC and LU, up to around 90% accuracy, about 5% higher than the existing deep learning methods, and 10% greater than traditional pixel-based and object-based approaches. This research made a significant contribution in LC and LU classification through deep learning based innovations, and has great potential utility in a wide range of geospatial applications

An integrative approach using remote sensing and social analysis to identify different settlement types and the specific living conditions of its inhabitants

Someday in 2007, the world population reached a historical landmark: for the first time in human history, more than half of the world´s population was urban. A stagnation of this urbanization process is not in sight, so that by 2050, already 70 percent of humankind is projected to live in urban settlements. Over the last few decades, enormous migrations from rural hinterlands to steadily growing cities could be witnessed coming along with a dramatic growth of the world’s urban population. The speed and the scale of this growth, particularly in the so called less developed regions, are posing tremendous challenges to the countries concerned as well as to the world community. Within mega cities the strongest trends and the most extreme dimensions of the urbanization process can be observed. Their rapid growth results in uncontrolled processes of fragmentation which is often associated with pronounced poverty, social inequality, socio-spatial and political fragmentation, environmental degradation as well as population demands that outstrip environmental service capacity. For the majority of the mega cities a tremendous increase of informal structures and processes has to be observed. Consequentially informal settlements are growing, which represent those characteristic municipal areas being subject to particularly high population density, dynamics as well as marginalization. They have quickly become the most visible expression of urban poverty in developing world cities. Due to the extreme dynamics, the high complexity and huge spatial dimension of mega cities, urban administrations often only have an obsolete or not even existing data basis available to be at all informed about developments, trends and dimensions of urban growth and change. The knowledge about the living conditions of the residents is correspondingly very limited, incomplete and not up to date. Traditional methods such as statistical and regional analyses or fieldwork are no longer capable to capture such urban process. New data sources and monitoring methodologies are required in order to provide an up to date information basis as well as planning strate¬gies to enable sustainable developments and to simplify planning processes in complex urban structures. This research shall seize the described problem and aims to make a contribution to the requirements of monitoring fast developing mega cities. Against this background a methodology is developed to compensate the lack of socio-economic data and to deduce meaningful information on the living conditions of the inhabitants of mega cities. Neither social science methods alone nor the exclusive analysis of remote sensing data can solve the problem of the poor quality and outdated data base. Conventional social science methods cannot cope with the enormous developments and the tremendous growth as they are too labor-, as well as too time- and too cost-intensive. On the other hand, the physical discipline of remote sensing does not allow for direct conclusions on social parameters out of remote sensing images. The prime objective of this research is therefore the development of an integrative approach − bridging remote sensing and social analysis – in order to derive useful information about the living conditions in this specific case of the mega city Delhi and its inhabitants. Hence, this work is established in the overlapping range of the research topics remote sensing, urban areas and social science. Delhi, as India’s fast growing capital, meanwhile with almost 25 million residents the second largest city of the world, represents a prime example of a mega city. Since the second half of the 20th century, Delhi has been transformed from a modest town with mainly administrative and trade-related functions to a complex metropolis with a steep socio-economic gradient. The quality and amount of administrative and socio-economic data are poor and the knowledge about the circumstances of Delhi’s residents is correspondingly insufficient and outdated. Delhi represents therefore a perfectly suited study area for this research. In order to gather information about the living conditions within the different settlement types a methodology was developed and conducted to analyze the urban environment of the mega city Delhi. To identify different settlement types within the urban area, regarding the complex and heterogeneous appearance of the Delhi area, a semi-automated, object-oriented classification approach, based on segmentation derived image objects, was implemented. As the complete conceptual framework of this research, the classification methodology was developed based on a smaller representative training area at first and applied to larger test sites within Delhi afterwards. The object-oriented classification of VHR satellite imagery of the QuickBird sensor allowed for the identification of five different urban land cover classes within the municipal area of Delhi. In the focus of the image analysis is yet the identification of different settlement types and amongst these of informal settlements in particular. The results presented within this study demonstrate, that, based on density classes, the developed methodology is suitable to identify different settlement types and to detect informal settlements which are mega urban risk areas and thus potential residential zones of vulnerable population groups. The remote sensing derived land cover maps form the foundation for the integrative analysis concept and deliver there¬fore the general basis for the derivation of social attributes out of remote sensing data. For this purpose settlement characteristics (e.g., area of the settlement, average building size, and number of houses) are estimated from the classified QuickBird data and used to derive spatial information about the population distribution. In a next step, the derived information is combined with in-situ information on socio-economic conditions (e.g., family size, mean water consumption per capita/family) extracted from georeferenced questionnaires conducted during two field trips in Delhi. This combined data is used to characterize a given settlement type in terms of specific population and water related variables (e.g., population density, total water consumption). With this integrative methodology a catalogue can be compiled, comprising the living conditions of Delhi’s inhabitants living in specific settlement structures – and this in a quick, large-scaled, cost effective, by random or regularly repeatable way with a relatively small required data basis.The combined application of remotely sensed imagery and socio-economic data allows for the mapping, capturing and characterizing the socio-economic structures and dynamics within the mega city of Delhi, as well as it establishes a basis for the monitoring of the mega city of Delhi or certain areas within the city respectively by remote sensing. The opportunity to capture the condition of a mega city and to monitor its development in general enables the persons in charge to identify unbeneficial trends and to intervene accordingly from an urban planning perspective and to countersteer against a non-adequate supply of the inhabitants of different urban districts, primarily of those of informal settlements. This study is understood to be a first step to the development of methods which will help to identify and understand the different forms, actors and processes of urbanization in mega cities. It could support a more proactive and sustainable urban planning and land management – which in turn will increase the importance of urban remote sensing techniques. In this regard, the most obvious and direct beneficiaries are on the one hand the governmental agencies and urban planners and on the other hand, and which is possibly the most important goal, the inhabitants of the affected areas, whose living conditions can be monitored and improved as required. Only if the urban monitoring is quickly, inexpensively and easily available, it will be accepted and applied by the authorities, which in turn enables for the poorest to get the support they need. All in all, the listed benefits are very convincing and corroborate the combined use of remotely sensed and socio-economic data in mega city research

How well do deep learning-based methods for land cover classification and object detection perform on high resolution remote sensing imagery?

© 2020 by the authors. Land cover information plays an important role in mapping ecological and environmental changes in Earth's diverse landscapes for ecosystem monitoring. Remote sensing data have been widely used for the study of land cover, enabling efficient mapping of changes of the Earth surface from Space. Although the availability of high-resolution remote sensing imagery increases significantly every year, traditional land cover analysis approaches based on pixel and object levels are not optimal. Recent advancement in deep learning has achieved remarkable success on image recognition field and has shown potential in high spatial resolution remote sensing applications, including classification and object detection. In this paper, a comprehensive review on land cover classification and object detection approaches using high resolution imagery is provided. Through two case studies, we demonstrated the applications of the state-of-the-art deep learning models to high spatial resolution remote sensing data for land cover classification and object detection and evaluated their performances against traditional approaches. For a land cover classification task, the deep-learning-based methods provide an end-to-end solution by using both spatial and spectral information. They have shown better performance than the traditional pixel-based method, especially for the categories of different vegetation. For an objective detection task, the deep-learning-based object detection method achieved more than 98% accuracy in a large area; its high accuracy and efficiency could relieve the burden of the traditional, labour-intensive method. However, considering the diversity of remote sensing data, more training datasets are required in order to improve the generalisation and the robustness of deep learning-based models