Individual Building Rooftop and Tree Crown Segmentation from High-Resolution Urban Aerial Optical Images

We segment buildings and trees from aerial photographs by using superpixels, and we estimate the tree’s parameters by using a cost function proposed in this paper. A method based on image complexity is proposed to refine superpixels boundaries. In order to classify buildings from ground and classify trees from grass, the salient feature vectors that include colors, Features from Accelerated Segment Test (FAST) corners, and Gabor edges are extracted from refined superpixels. The vectors are used to train the classifier based on Naive Bayes classifier. The trained classifier is used to classify refined superpixels as object or nonobject. The properties of a tree, including its locations and radius, are estimated by minimizing the cost function. The shadow is used to calculate the tree height using sun angle and the time when the image was taken. Our segmentation algorithm is compared with other two state-of-the-art segmentation algorithms, and the tree parameters obtained in this paper are compared to the ground truth data. Experiments show that the proposed method can segment trees and buildings appropriately, yielding higher precision and better recall rates, and the tree parameters are in good agreement with the ground truth data

Geocoding of trees from street addresses and street-level images

We introduce an approach for updating older tree inventories with geographic coordinates using street-level panorama images and a global optimization framework for tree instance matching. Geolocations of trees in inventories until the early 2000s where recorded using street addresses whereas newer inventories use GPS. Our method retrofits older inventories with geographic coordinates to allow connecting them with newer inventories to facilitate long-term studies on tree mortality etc. What makes this problem challenging is the different number of trees per street address, the heterogeneous appearance of different tree instances in the images, ambiguous tree positions if viewed from multiple images and occlusions. To solve this assignment problem, we (i) detect trees in Google street-view panoramas using deep learning, (ii) combine multi-view detections per tree into a single representation, (iii) and match detected trees with given trees per street address with a global optimization approach. Experiments for trees in 5 cities in California, USA, show that we are able to assign geographic coordinates to 38% of the street trees, which is a good starting point for long-term studies on the ecosystem services value of street trees at large scale

Quantifying the urban forest environment using dense discrete return LiDAR and aerial color imagery for segmentation and object-level biomass assessment

The urban forest is becoming increasingly important in the contexts of urban green space and recreation, carbon sequestration and emission offsets, and socio-economic impacts. In addition to aesthetic value, these green spaces remove airborne pollutants, preserve natural resources, and mitigate adverse climate changes, among other benefits. A great deal of attention recently has been paid to urban forest management. However, the comprehensive monitoring of urban vegetation for carbon sequestration and storage is an under-explored research area. Such an assessment of carbon stores often requires information at the individual tree level, necessitating the proper masking of vegetation from the built environment, as well as delineation of individual tree crowns. As an alternative to expensive and time-consuming manual surveys, remote sensing can be used effectively in characterizing the urban vegetation and man-made objects. Many studies in this field have made use of aerial and multispectral/hyperspectral imagery over cities. The emergence of light detection and ranging (LiDAR) technology, however, has provided new impetus to the effort of extracting objects and characterizing their 3D attributes - LiDAR has been used successfully to model buildings and urban trees. However, challenges remain when using such structural information only, and researchers have investigated the use of fusion-based approaches that combine LiDAR and aerial imagery to extract objects, thereby allowing the complementary characteristics of the two modalities to be utilized. In this study, a fusion-based classification method was implemented between high spatial resolution aerial color (RGB) imagery and co-registered LiDAR point clouds to classify urban vegetation and buildings from other urban classes/cover types. Structural, as well as spectral features, were used in the classification method. These features included height, flatness, and the distribution of normal surface vectors from LiDAR data, along with a non-calibrated LiDAR-based vegetation index, derived from combining LiDAR intensity at 1064 nm with the red channel of the RGB imagery. This novel index was dubbed the LiDAR-infused difference vegetation index (LDVI). Classification results indicated good separation between buildings and vegetation, with an overall accuracy of 92% and a kappa statistic of 0.85. A multi-tiered delineation algorithm subsequently was developed to extract individual tree crowns from the identified tree clusters, followed by the application of species-independent biomass models based on LiDAR-derived tree attributes in regression analysis. These LiDAR-based biomass assessments were conducted for individual trees, as well as for clusters of trees, in cases where proper delineation of individual trees was impossible. The detection accuracy of the tree delineation algorithm was 70%. The LiDAR-derived biomass estimates were validated against allometry-based biomass estimates that were computed from field-measured tree data. It was found out that LiDAR-derived tree volume, area, and different distribution parameters of height (e.g., maximum height, mean of height) are important to model biomass. The best biomass model for the tree clusters and the individual trees showed an adjusted R-Squared value of 0.93 and 0.58, respectively. The results of this study showed that the developed fusion-based classification approach using LiDAR and aerial color (RGB) imagery is capable of producing good object detection accuracy. It was concluded that the LDVI can be used in vegetation detection and can act as a substitute for the normalized difference vegetation index (NDVI), when near-infrared multiband imagery is not available. Furthermore, the utility of LiDAR for characterizing the urban forest and associated biomass was proven. This work could have significant impact on the rapid and accurate assessment of urban green spaces and associated carbon monitoring and management

Understanding High Resolution Aerial Imagery Using Computer Vision Techniques

Computer vision can make important contributions to the analysis of remote sensing satellite or aerial imagery. However, the resolution of early satellite imagery was not sufficient to provide useful spatial features. The situation is changing with the advent of very-high-spatial-resolution (VHR) imaging sensors. This change makes it possible to use computer vision techniques to perform analysis of man-made structures. Meanwhile, the development of multi-view imaging techniques allows the generation of accurate point clouds as ancillary knowledge. This dissertation aims at developing computer vision and machine learning algorithms for high resolution aerial imagery analysis in the context of application problems including debris detection, building detection and roof condition assessment. High resolution aerial imagery and point clouds were provided by Pictometry International for this study. Debris detection after natural disasters such as tornadoes, hurricanes or tsunamis, is needed for effective debris removal and allocation of limited resources. Significant advances in aerial image acquisition have greatly enabled the possibilities for rapid and automated detection of debris. In this dissertation, a robust debris detection algorithm is proposed. Large scale aerial images are partitioned into homogeneous regions by interactive segmentation. Debris areas are identified based on extracted texture features. Robust building detection is another important part of high resolution aerial imagery understanding. This dissertation develops a 3D scene classification algorithm for building detection using point clouds derived from multi-view imagery. Point clouds are divided into point clusters using Euclidean clustering. Individual point clusters are identified based on extracted spectral and 3D structural features. The inspection of roof condition is an important step in damage claim processing in the insurance industry. Automated roof condition assessment from remotely sensed images is proposed in this dissertation. Initially, texture classification and a bag-of-words model were applied to assess the roof condition using features derived from the whole rooftop. However, considering the complexity of residential rooftop, a more sophisticated method is proposed to divide the task into two stages: 1) roof segmentation, followed by 2) classification of segmented roof regions. Deep learning techniques are investigated for both segmentation and classification. A deep learned feature is proposed and applied in a region merging segmentation algorithm. A fine-tuned deep network is adopted for roof segment classification and found to achieve higher accuracy than traditional methods using hand-crafted features. Contributions of this study include the development of algorithms for debris detection using 2D images and building detection using 3D point clouds. For roof condition assessment, the solutions to this problem are explored in two directions: features derived from the whole rooftop and features extracted from each roof segments. Through our research, roof segmentation followed by segments classification was found to be a more promising method and the workflow processing developed and tested. Deep learning techniques are also investigated for both roof segmentation and segments classification. More unsupervised feature extraction techniques using deep learning can be explored in future work

3D Classification of Power Line Scene Using Airborne Lidar Data

Failure to adequately maintain vegetation within a power line corridor has been identified as a main cause of the August 14, 2003 electric power blackout. Such that, timely and accurate corridor mapping and monitoring are indispensible to mitigate such disaster. Moreover, airborne LiDAR (Light Detection And Ranging) has been recently introduced and widely utilized in industries and academies thanks to its potential to automate the data processing for scene analysis including power line corridor mapping. However, today’s corridor mapping practice using LiDAR in industries still remains an expensive manual process that is not suitable for the large-scale, rapid commercial compilation of corridor maps. Additionally, in academies only few studies have developed algorithms capable of recognizing corridor objects in the power line scene, which are mostly based on 2-dimensional classification. Thus, the objective of this dissertation is to develop a 3-dimensional classification system which is able to automatically identify key objects in the power line corridor from large-scale LiDAR data. This dissertation introduces new features for power structures, especially for the electric pylon, and existing features which are derived through diverse piecewise (i.e., point, line and plane) feature extraction, and then constructs a classification model pool by building individual models according to the piecewise feature sets and diverse voltage training samples using Random Forests. Finally, this dissertation proposes a Multiple Classifier System (MCS) which provides an optimal committee of models from the model pool for classification of new incoming power line scene. The proposed MCS has been tested on a power line corridor where medium voltage transmission lines (115 kV and 230 kV) pass. The classification results based on the MCS applied by optimally selecting the pre-built classification models according to the voltage type of the test corridor demonstrate a good accuracy (89.07%) and computationally effective time cost (approximately 4 hours/km) without additional training fees

Towards Automated Analysis of Urban Infrastructure after Natural Disasters using Remote Sensing

Natural disasters, such as earthquakes and hurricanes, are an unpreventable component of the complex and changing environment we live in. Continued research and advancement in disaster mitigation through prediction of and preparation for impacts have undoubtedly saved many lives and prevented significant amounts of damage, but it is inevitable that some events will cause destruction and loss of life due to their sheer magnitude and proximity to built-up areas. Consequently, development of effective and efficient disaster response methodologies is a research topic of great interest. A successful emergency response is dependent on a comprehensive understanding of the scenario at hand. It is crucial to assess the state of the infrastructure and transportation network, so that resources can be allocated efficiently. Obstructions to the roadways are one of the biggest inhibitors to effective emergency response. To this end, airborne and satellite remote sensing platforms have been used extensively to collect overhead imagery and other types of data in the event of a natural disaster. The ability of these platforms to rapidly probe large areas is ideal in a situation where a timely response could result in saving lives. Typically, imagery is delivered to emergency management officials who then visually inspect it to determine where roads are obstructed and buildings have collapsed. Manual interpretation of imagery is a slow process and is limited by the quality of the imagery and what the human eye can perceive. In order to overcome the time and resource limitations of manual interpretation, this dissertation inves- tigated the feasibility of performing fully automated post-disaster analysis of roadways and buildings using airborne remote sensing data. First, a novel algorithm for detecting roadway debris piles from airborne light detection and ranging (lidar) point clouds and estimating their volumes is presented. Next, a method for detecting roadway flooding in aerial imagery and estimating the depth of the water using digital elevation models (DEMs) is introduced. Finally, a technique for assessing building damage from airborne lidar point clouds is presented. All three methods are demonstrated using remotely sensed data that were collected in the wake of recent natural disasters. The research presented in this dissertation builds a case for the use of automatic, algorithmic analysis of road networks and buildings after a disaster. By reducing the latency between the disaster and the delivery of damage maps needed to make executive decisions about resource allocation and performing search and rescue missions, significant loss reductions could be achieved

Estimation of Carbon Storage in Urban Trees Using Multispectral Airborne Laser Scanning Data

With the continued growth of global population, urbanization becomes an inevitable trend. As substantial urban expansion undergoes, ecosystem and global land cover have been altered consequently. Urban development becomes the biggest contributor to global carbon emissions while the process of urbanization results in urban heat islands, climate change, and losses of carbon sinks. Urban vegetation has drawn direct attention of city planners and policy makers by considering the importance of vegetation in urban climate modification and energy conservation in different ways. For instance, tree shading and wind shielding effects can attenuate the direct solar heat and air infiltration into individual houses. In city wide, vegetation contributes the largest proportion of carbon storage which reduces climate warming and urban heat island effects by sequestering CO2 and storing carbon in biomass. The carbon content stored in individual trees can be estimated by dendrometric parameters such as the diameter at breast height (DBH) using allometry-based models. With the development of airborne laser scanning (ALS) technology, ALS data and very high resolution multispectral imagery have proven to be promising tools for deriving dendrometric parameters in forest. With the emerging multispectral ALS technology, it became possible to obtain both the range and spectral information from a single source meanwhile the intensity of multispectral ALS showed its power in vegetation mapping. This study aims to develop a workflow that can quantify the carbon storage in urban trees using multispectral ALS data. The workflow consists of four steps: multispectral ALS data processing, vegetation isolation, dendrometric parameters estimation, and carbon storage modeling. First, the raw multispectral ALS data is intensity-rectified and filtered to generate a normalized Digital Surface Model (nDSM) and multispectral ALS intensity information at wavelengths: 532 nm (Green), 1064 nm (Near-infrared, NIR), and 1550 nm (Shortwave Infrared, SWIR), respectively. Vegetation covers are isolated by the support vector machine (SVM) classifier using multispectral ALS intensity information and nDSM in which total six classes including two vegetation classes (grass and tree) are classified. Individual tree crown is delineated by local maxima filtering and marker-controlled watershed segmentation. Tree height and crown width are derived from the crown segments and compared with field measurements. An ALS-DBH (diameter at breast height) multiple linear regression model is developed to predict field-measured DBH using ALS-derived tree height and crown width and assessed by cross validation. Then the carbon storage in individual trees is calculated by allometric equations using ALS-estimated DBH and height. A total of 40 trees are sampled in the field that four attributes: height, crown width, DBH, and biomass are recorded for each single tree. The results show that the land cover classification with multispectral ALS intensity images and nDSM achieves above 90% overall accuracy. The result of local maxima filtering is improved by using both multispectral ALS intensity and nDSM as input data. The ALS-derived tree height has a root mean square error (RMSE) of 1.21 m (relative RMSE = 6.8%) and the ALS-derived crown width has a RMSE of 1.47 m (relative RMSE = 16.4%). The prediction performance of the ALS-DBH model achieves R2 over 0.80 with a RMSE of 4.6 cm. The predicted carbon storage using ALS-modeled DBH corresponded to a RMSE of 142 kg (28.6%) and a bias of 14.4 kg. Results suggest that ALS-based dendrometric parameter estimation and allometric models can yield consistent performance and accurate estimation. Citywide carbon storage estimation is derived in this study by extrapolating the values within the study area to the entire city based on the specific proportion of each land cover type in the entire city. The proposed workflow also reveals the potential of multispectral ALS data in estimating carbon storage at individual-tree level and mapping vegetation in the urban environment

Remote Sensing for Land Administration 2.0

The reprint “Land Administration 2.0” is an extension of the previous reprint “Remote Sensing for Land Administration”, another Special Issue in Remote Sensing. This reprint unpacks the responsible use and integration of emerging remote sensing techniques into the domain of land administration, including land registration, cadastre, land use planning, land valuation, land taxation, and land development. The title was chosen as “Land Administration 2.0” in reference to both this Special Issue being the second volume on the topic “Land Administration” and the next-generation requirements of land administration including demands for 3D, indoor, underground, real-time, high-accuracy, lower-cost, and interoperable land data and information