Concept for classifying facade elements based on material, geometry and thermal radiation using multimodal UAV remote sensing

This paper presents a concept for classification of facade elements, based on the material and the geometry of the elements in addition to the thermal radiation of the facade with the usage of a multimodal Unmanned Aerial Vehicle (UAV) system. Once the concept is finalized and functional, the workflow can be used for energy demand estimations for buildings by exploiting existing methods for estimation of heat transfer coefficient and the transmitted heat loss. The multimodal system consists of a thermal, a hyperspectral and an optical sensor, which can be operational with a UAV. While dealing with sensors that operate in different spectra and have different technical specifications, such as the radiometric and the geometric resolution, the challenges that are faced are presented. Addressed are the different approaches of data fusion, such as image registration, generation of 3D models by performing image matching and the means for classification based on either the geometry of the object or the pixel values. As a first step towards realizing the concept, the result from a geometric calibration with a designed multimodal calibration pattern is presented

Urban Material Classification Using Spectral and Textural Features Retrieved from Autoencoders

Classification of urban materials using remote sensing data, in particular hyperspectral data, is common practice. Spectral libraries can be utilized to train a classifier since they provide spectral features about selected urban materials. However, urban materials can have similar spectral characteristic features due to high inter-class correlation which can lead to misclassification. Spectral libraries rarely provide imagery of their samples, which disables the possibility of classifying urban materials with additional textural information. Thus, this paper conducts material classification comparing the benefits of using close-range acquired spectral and textural features. The spectral features consist of either the original spectra, a PCA-based encoding or the compressed spectral representation of the original spectra retrieved using a deep autoencoder. The textural features are generated using a deep denoising convolutional autoencoder. The spectral and textural features are gathered from the recently published spectral library KLUM. Three classifiers are used, the two well-established Random Forest and Support Vector Machine classifiers in addition to a Histogram-based Gradient Boosting Classification Tree. The achieved overall accuracy was within the range of 70–80% with a standard deviation between 2–10% across all classification approaches. This indicates that the amount of samples still is insufficient for some of the material classes for this classification task. Nonetheless, the classification results indicate that the spectral features are more important for assigning material labels than the textural features

KLUM: An urban VNIR and SWIR spectral library consisting of building materials

Knowledge about the existing materials in urban areas has, in recent times, increased in importance. With the use of imaging spectroscopy and hyperspectral remote sensing techniques, it is possible to measure and collect the spectra of urban materials. Most spectral libraries consist of either spectra acquired indoors in a controlled lab environment or of spectra from afar using airborne systems accompanied with in situ measurements. Furthermore, most publicly available spectral libraries have, so far, not focused on facade materials but on roofing materials, roads, and pavements. In this study, we present an urban spectral library consisting of collected in situ material spectra with imaging spectroscopy techniques in the visible and near-infrared (VNIR) and short-wave infrared (SWIR) spectral range, with particular focus on facade materials and material variation. The spectral library consists of building materials, such as facade and roofing materials, in addition to surrounding ground material, but with a focus on facades. This novelty is beneficial to the community as there is a shift to oblique-viewed Unmanned Aerial Vehicle (UAV)-based remote sensing and thus, there is a need for new types of spectral libraries. The post-processing consists partly of an intra-set solar irradiance correction and recalculation of reference spectra caused by signal clipping. Furthermore, the clustering of the acquired spectra was performed and evaluated using spectral measures, including Spectral Angle and a modified Spectral Gradient Angle. To confirm and compare the material classes, we used samples from publicly available spectral libraries. The final material classification scheme is based on a hierarchy with subclasses, which enables a spectral library with a larger material variation and offers the possibility to perform a more refined material analysis. The analysis reveals that the color and the surface structure, texture or coating of a material plays a significantly larger role than what has been presented so far. The samples and their corresponding detailed metadata can be found in the Karlsruhe Library of Urban Materials (KLUM) archiv

Classification and representation of commonly used roofing material using multisensorial aerial data

© Authors 2018. CC BY 4.0 License. As more cities are starting to experience the urban heat islands effect, knowledge about the energy emitted from building roofs is of primary importance. Since this energy depends both on roof orientations and materials, we tackled both issues by analysing sensor data from multispectral, thermal infrared, high-resolution RGB, and airborne laser datasets (each with different spatial resolutions) of a council in Perth, Australia. To localise the roofs, we acquired building outlines that had to be updated using the normalised digital surface model, the NDVI and the planarity. Then, we computed a semantic 3D model of the study area, with roof detail analysis being a particular focus. The main objective of this study, however, was to classify three commonly used roofing materials: Cement tiles, Colorbond and Zincalume by combining the multispectral and thermal infrared image bands while the high-resolution RGB dataset was used to provide additional information about the roof texture. Three types of image segmentation approaches were evaluated to assess any differences while performing the material classification; pixel-wise, superpixel-wise and building-wise image segmentation. Due to the limited amount of labelled data, we extended the dataset by labelling data ourselves and merged Colorbond and Zincalume into one separate class. The supervised classifier Random Forest was applied to all reasonable configurations of segmentation kinds, numbers of classes, and finally, keeping track of the added value of principal component analysis

CONCEPT FOR CLASSIFYING FACADE ELEMENTS BASED ON MATERIAL, GEOMETRY AND THERMAL RADIATION USING MULTIMODAL UAV REMOTE SENSING

This paper presents a concept for classification of facade elements, based on the material and the geometry of the elements in addition to the thermal radiation of the facade with the usage of a multimodal Unmanned Aerial Vehicle (UAV) system. Once the concept is finalized and functional, the workflow can be used for energy demand estimations for buildings by exploiting existing methods for estimation of heat transfer coefficient and the transmitted heat loss. The multimodal system consists of a thermal, a hyperspectral and an optical sensor, which can be operational with a UAV. While dealing with sensors that operate in different spectra and have different technical specifications, such as the radiometric and the geometric resolution, the challenges that are faced are presented. Addressed are the different approaches of data fusion, such as image registration, generation of 3D models by performing image matching and the means for classification based on either the geometry of the object or the pixel values. As a first step towards realizing the concept, the result from a geometric calibration with a designed multimodal calibration pattern is presented

From multi-sensor aerial data to thermal and infrared simulation of semantic 3D models: Towards identification of urban heat islands

Urban heat islands degrade the quality of life in many urban centers. To achieve their detection in urban canopy and to predict their development in the future, infrared simulation turns out to be a suitable tool. For simulation of the temperature, various scene properties must be taken into account. Starting at raw sensor data acquired from the air, we developed an end-to-end pipeline to the semantic mesh, in which temperatures and radiance can be simulated depending on actual weather data and initial conditions and which has a potential to track the urban heat islands. To acquire the mesh, we focus on retrieving land cover classes and 3D geometry. The land cover map helps to identify buildings, to update the existing geographic maps, and to analyze building roofs with respect to their materials and thus, sustainability. The 3D geometry basically presupposes storing the scene efficiently into triangles. For each triangle, we are not only interested in material properties, but also in neighborhood relations allowing to model heat conduction. Together with terms for convection and radiation, we formulate the heat balance equation and compute the surface temperature as a function of time. The pipeline was tested on a dataset from a large Australian city exhibiting most properties which bear risks to contribute to heat islands: Its location in a subtropical (Mediterranean) climate zone, rapidly growing population, and, at least initially, a certain lack of sensibility towards sustainable management of resources and materials. To analyze both latter factors, two intermediate results from our method, namely tracking urbanization degree and identification of common roofing materials, are addressed and thoroughly evaluated in the dataset. It could be deduced that the area occupied by buildings increased by roughly 5% and that roughly every 6th building has a steel roof. Finally, high similarities with the ground truth were achieved both for temperature curves in some 20 test points and for large-scale evaluation. Deviations from the ground truth emerge in case of building roofs leading to the conclusion that the inner model assumption could be less accurate and, therefore, runs the danger to increase the urban heat island effect