A deep 2D/3D Feature-Level fusion for classification of UAV multispectral imagery in urban areas

In this paper, a deep convolutional neural network (CNN) is developed to classify the Unmanned Aerial Vehicle (UAV) derived multispectral imagery and normalized digital surface model (DSM) data in urban areas. For this purpose, a multi-input deep CNN (MIDCNN) architecture is designed using 11 parallel CNNs; 10 deep CNNs to extract the features from all possible triple combinations of spectral bands as well as one deep CNN dedicated to the normalized DSM data. The proposed method is compared with the traditional single-input (SI) and double-input (DI) deep CNN designations and random forest (RF) classifier, and evaluated using two independent test datasets. The results indicate that increasing the CNN layers parallelly augmented the classifier’s generalization and reduced overfitting risk. The overall accuracy and kappa value of the proposed method are 95% and 0.93, respectively, for the first test dataset, and 96% and 0.94, respectively, for the second test data set

Building change detection using the parallel spatial-channel attention block and edge-guided deep network

Building change detection in high-resolution satellite images plays a special role in urban management and development. Recently, methods for building change detection have been greatly improved by developing deep learning. Although deep learning technologies, especially Siamese convolutional neural networks, have been successful and popular, they usually have problems in extracting features that are not discriminative enough and also cause the loss of shape and details at the edges. To address these problems, a dual-branch deep network and a parallel spatial-channel attention mechanism were suggested to extract spatial and spectral dependencies and more discriminative features. The spatial attention unit measured the rich context of local features, and the distinction between changed objects and backgrounds was increased using spatial attention in deep features. The channel attention module adjusted the weight of channels and acted as a channel selection process. Mixing two attentions to parallel mode made the features more practical, and useful information was learned more robustly. Moreover, a dual loss function was proposed in which the edge-based consistency constraints were used in the first part to converge the edges of the training and the predicted data. The weighted binary cross-entropy was added to the second part of the loss function. The proposed method was implemented on two remote sensing datasets, and the results were evaluated with state-of-the-art methods. With the proposed model, the F1-score was improved by 2.43% and 1.83% in the first and second datasets, respectively

A Novel Boundary Loss Function in Deep Convolutional Networks to Improve the Buildings Extraction From High-Resolution Remote Sensing Images

In recent years, there has been a significant increase in the production of high-resolution aerial and satellite imagery. Analyzing and extracting urban features from these images, especially building components, is a significant challenge in photogrammetry and remote sensing. Meanwhile, deep convolution neural networks have been used as a powerful model in the semantic segmentation of building features. However, due to the structure of these types of deep convolution networks, accurate retrieval of building boundary information during training will be difficult, and this will lead to ambiguous boundary areas. On the other hand, the use of distributed-based loss functions, such as the binary cross-entropy loss alone cannot improve the segmentation accuracy in the boundary areas of building features. Accordingly, in this article, a derivative boundary loss function is introduced to optimize and enhance the extraction of buildings in border areas. To calculate the proposed boundary loss function, distance transform image (DTI) is obtained from ground-truth images and predicted images, that derived from the segmentation network. In this article, a derivative method for calculating DTI is presented. Another advantage of the proposed loss function is its ability to be used in a wide range of deep convolution segmentation networks. The proposed method was investigated and tested using the ISPRS Potsdam and Vaihingen datasets. The results of this implementation show the superiority in the criteria of evaluation and improvement of buildings extraction if the proposed boundary loss function is used along with the distributed-based loss function

UAV Remote Sensing for Smart Agriculture

As the world population continues to grow, the demand for food is also increasing. At the same time, a series of global and local challenges are threatening ‘food security’. In this context, various technologies and techniques have been proposed and considered in recent decades to secure the efficient usage of the planet’s agricultural resources, i.e. ‘smart agriculture’. This requires the accurate and advanced acquisition, modelling and management of relevant data. This article presents a brief discussion of how unmanned aerial vehicles (UAVs or ‘drones’) can play a critical role in smart agriculture, including a focus on their capabilities and applications

Intensifying the spatial resolution of 3D thermal models from aerial imagery using deep learning-based image super-resolution

Nowadays, 3D thermal models can play an important role in buildings' energy management while acquiring multisource data to generate a high-resolution 3D thermal model. Consequently, in this article, a method for intensifying 3D thermal model using deep learning-based image super-resolution is presented. In the proposed method, first, the enhanced deep residual super-resolution (EDSR) deep network is re-trained based on thermal aerial images. Second, the resolution of low-resolution thermal images is enhanced using the newly trained network. Finally, the state-of-the-art structures from motion (SfM), semi global matching (SGM) and space intersection are utilized to generate intensified 3D thermal model from the resolution enhanced thermal images. Spatial evaluations indicate a 5% increase in edge-based image fusion metric (EFM) for the intensified 3D model. Besides, the evaluations show that the modulation transfer function (MTF) curves of the intensified 3D thermal model are closer to a reference model against the original 3D thermal model. Highlights A 3D thermal model intensification solution using EDSR is proposed which is independent of hardware techniques and multisource data. Considering the importance of edge sharpness in the intensified 3D thermal model, the quality of edges is assessed using MTF curves and the EFM metric. In comparison to the original 3D thermal model, the MTF curves of the intensified 3D thermal model are closer to the MTF curve of the high-resolution 3D model. The EFM metric shows higher values for MTF curves of the intensified 3D thermal model against MTF curves of the original 3D thermal model

Fusion of UAV-based infrared and visible images for thermal leakage map generation of building facades

To make the best use of available energy resources and reduce costs, improving the energy efficiency of buildings has become a critical issue for the construction industry. Today, developing a three-dimensional model of the energy consumption rates in buildings based on thermal infrared images is essential to visualize, identify and increase energy efficiency. The purpose of this study is to suggest a methodology for generating a thermal leakage map of building facades utilizing the fusion of thermal infrared and visible images captured by Unmanned Aerial Vehicles (UAVs). In general, the proposed method involves three basic steps: the generation of thermal infrared and visible dense point clouds from the building’s facade using Structure from Motion (SfM) and Multi-View Stereo (MVS) algorithms; the fusion of visible and thermal infrared dense point clouds using the Iterative Closest Point (ICP) algorithm to overcome thermal infrared point cloud constraints; the use of edge extraction and region-based segmentation methods to determine the location of the thermal leakage of building facade’s. To that end, two datasets obtained for separate building facades are used to assess the proposed strategy. The results of the data analyses for the extraction of the desired components and determination of thermal leakage locations on the building facets provided a Precision and Recall score of 87 and 90% for the first dataset and 87 and 88 for the second dataset. Examining the outcomes of calculating thermal leakage zones indicates improving Precision and Recall

Detection and Recognition of Drones Based on a Deep Convolutional Neural Network Using Visible Imagery

Drones are becoming increasingly popular not only for recreational purposes but also in a variety of applications in engineering, disaster management, logistics, securing airports, and others. In addition to their useful applications, an alarming concern regarding physical infrastructure security, safety, and surveillance at airports has arisen due to the potential of their use in malicious activities. In recent years, there have been many reports of the unauthorized use of various types of drones at airports and the disruption of airline operations. To address this problem, this study proposes a novel deep learning-based method for the efficient detection and recognition of two types of drones and birds. Evaluation of the proposed approach with the prepared image dataset demonstrates better efficiency compared to existing detection systems in the literature. Furthermore, drones are often confused with birds because of their physical and behavioral similarity. The proposed method is not only able to detect the presence or absence of drones in an area but also to recognize and distinguish between two types of drones, as well as distinguish them from birds. The dataset used in this work to train the network consists of 10,000 visible images containing two types of drones as multirotors, helicopters, and also birds. The proposed deep learning method can directly detect and recognize two types of drones and distinguish them from birds with an accuracy of 83%, mAP of 84%, and IoU of 81%. The values of average recall, average accuracy, and average F1-score were also reported as 84%, 83%, and 83%, respectively, in three classes

Optimum path determination to facilitate fire station rescue missions using ant colony optimization algorithms (case study: city of Karaj)

The successful conduct of a rescue mission in urban areas is directly related to the timely deployment of equipment and personnel to the incident location which justifies the quest for optimum path selection for emergency purposes. In this study, it is attempted to use Ant Colony Optimization (ACO) to find the optimum paths between fire stations and incident locations. It is also attempted to build up an evaluation tool using ACO to detect critical road segments that the overall accessibility to fire station services throughout the urban area is constituted upon their excellent functionality. Therefore, an ACO solution is designed to find optimum paths between the fire station and some randomly distributed incident locations. Regarding different variants of ACO, the algorithm enjoys the Simple Ant Colony Optimization deployment strategy combined with Ant Algorithm Transition rules. Iteration best pheromone updating is also used as the pheromone reinforcement strategy. The cost function used to optimize the path considers the shortest Euclidean distance on the network. The results explicitly state that the proposed method is successful to create the optimum path in 95.45 percent of all times, compared to Dijkstra deterministic approaches. Moreover, the pheromone map as an indicator of the criticality of road elements is generated and discussed. Visual inspection shows that the pheromone map is verified as the road criticality map concerning fire station access to the region and therefore pre-emptive measures can be defined by analyzing the generated pheromone map