Co-registration of three-dimensional building models with image\ud features from infrared video sequences

In the European Union (EU) countries buildings consume 40% of the energy and cause 36% of CO2 emissions.\ud The thermal information of facades and roofs are important for building inspection and energy saving. Texturing\ud the existing three-dimensional (3D) building models with infrared (IR) images enriches the model database and\ud enables analysis of energy loss of buildings.\ud The main purpose of the presented thesis is to investigate methods for automatic extraction of the IR textures for\ud roofs and facades of the existing building model. The correction of the exterior orientation parameters of the IR\ud camera mounted on mobile platform is studied. The developed method bases on a point-to-point matching of the\ud features extracted from IR images with a wire frame building model.\ud Firstly, extraction of different feature types is studied on a sample IR image; Förstner and intersection points are\ud chosen for representation of the image features. Secondly, the 3D building model is projected into each frame of\ud the IR video sequence using orientation parameters; only coarse exterior orientation parameters are known. Then\ud the automatic co-registration of a 3D building model projection into the image sequence with image features is\ud carried out. The matching of a model and extracted features is applied iteratively and exterior orientation\ud parameters are adjusted with least square adjustment. The method is tested on a dataset of dense urban area.\ud Finally, an evaluation of developed method is presented with fiv

Automatic coregistration of three-dimensional building models with image features

The aim of this article is to investigate methods for\ud the automatic extraction of the infrared (IR) textures\ud for the roofs and facades of existing building models.\ud We focus on the correction of the measured exterior\ud orientation parameters of the IR camera mounted on\ud a mobile platform. The developed method is based on\ud point-to-point matching of the features extracted from\ud IR images with a wire-frame building model. Firstly,\ud the extraction of different feature types is studied on\ud a sample IR image; Förstner and intersection points\ud are chosen for a representation of the image features.\ud Secondly, the three-dimensional (3D) building model\ud is projected into each frame of the IR video sequence\ud using orientation parameters; only coarse exterior\ud orientation parameters are known. Then the automatic\ud co-registration of a 3D building model projection into\ud the image sequence with image features is carried out.\ud The matching of a model and extracted features is\ud applied iteratively, and exterior orientation parameters\ud are adjusted with least square adjustment. The method\ud is tested on a dataset of a dense urban area. Finally,\ud an evaluation of the developed method is presented\ud with five quality parameters, i.e. efficiency of the\ud method, completeness and correctness of matching\ud and extraction

Automatic coregistration of three-dimensional building models with image features

Namen naše raziskave je samodejna določitev tekstur streh in fasad stavb z infrardečih (IR) posnetkov za teksturiranje obstoječega trirazsežnega (3D) modela stavb. Za to je treba izboljšati natančnosti neposredno izmerjenih parametrov zunanje orientacije IR-kamere, pritrjene na mobilno platformo. Ta prispevek opisuje metodo, razvito za izboljšanje parametrov zunanje orientacije, ki temelji na ujemanju točk samodejno zaznanih grafičnih gradnikov z IR-videoposnetka in žičnega modela stavb. Najprej proučimo zaznavo različnih tipov grafičnih gradnikov na testnem IR-posnetku. Förstnerjeve in presečiščne točke izberemo kot primerne grafične gradnike z a predstavitev obravnavanih značilnosti stavb na IR-posnetku. 3D-model stavb projiciramo na vsak posamezen posnetek videosekvence ob upoštevanju orientacijskih parametrov, od katerih so parametri zunanje orientacije podani s približnimi vrednostmi. Nato izvedemo samodejno koregistracijo 3D-modela stavb, projiciranega na videoposnetek, in grafičnih gradnikov, zaznanih z istega IR-videoposnetka. Samodejno ujemanje 3D-modela stavb in zaznanih grafičnih gradnikov poteka iterativno in skupaj z izravnavo parametrov zunanje orientacije z metodo najmanjših kvadratov. Razvito metodologijo za koregistracijo in izravnavo zunanjih orientacijskih parametrov smo preizkusili na strnjenem poseljenem območju. Kakovost metodologije ocenimo s petimi parametri: učinkovitostjo metodologije, popolnostjo in pravilnostjo algoritmov za ujemanje in zaznavo grafičnih gradnikov.The aim of this article is to investigate methods for the automatic extraction of the infrared (IR) textures for the roofs and facades of existing building models. We focus on the correction of the measured exterior orientation parameters of the IR camera mounted on a mobile platform. The developed method is based on point-to-point matching of the features extracted from IR images with a wire-frame building model. Firstly, the extraction of different feature types is studied on a sample IR imageFörstner and intersection points are chosen for a representation of the image features. Secondly, the three-dimensional (3D) building model is projected into each frame of the IR video sequence using orientation parametersonly coarse exterior orientation parameters are known. Then the automatic co-registration of a 3D building model projection into the image sequence with image features is carried out. The matching of a model and extracted features is applied iteratively, and exterior orientation parameters are adjusted with least square adjustment. The method is tested on a dataset of a dense urban area. Finally, an evaluation of the developed method is presented with five quality parameters, i.e. efficiency of the method, completeness and correctness of matching and extraction

Spectral Information Retrieval for Sub-Pixel Building Edge Detection

Building extraction from imagery has been an active research area for decades. However, the precise building detection from hyperspectral (HSI) images solely is a less often addressed research question due to the low spatial resolution of data. The building boundaries are usually represented by spectrally mixed pixels, and classical edge detector algorithms fail to detect borders with sufficient completeness. The idea of the proposed method is to use fraction of materials in mixed pixels to derive weights for adjusting building boundaries. The building regions are detected using seeded region growing and merging in a HSI image; for the initial seed point selection the digital surface model (DSM) is used. Prior to region growing, the seeds are statistically tested for outliers on the basis of their spectral characteristics. Then, the border pixels of building regions are compared in spectrum to the seed points by calculating spectral dissimilarity. From this spectral dissimilarity the weights for weighted and constrained least squares (LS) adjustment are derived. We used the Spectral Angle Mapper (SAM) for spectral similarity measure, but the proposed boundary estimation method could benefit from soft classification or spectral unmixing results. The method was tested on a HSI image with spatial resolution of 4 m, and buildings of rectangular shape. The importance of constraints to the relations between building parts, e.g. perpendicularity is shown on example with a building with inner yards. The adjusted building boundaries are compared to the laser DSM, and have a relative accuracy of boundaries 1/4 of a pixel

Airborne real-time mapping and monitoring

In cases of large scale events and disasters public offices and organizations with security responsibilities as well as traffic authorities need accurate and reliable information for situation awareness. These pieces of information are used by the authorities to quickly meet decisions in order to safely and efficiently manage such events with the given resources. Particularly, information on the transportation system and traffic are of great importance, because the mobility of the public and relief teams is crucial for fast disaster relief. The VABENE++ Traffic Management for Large Scale Events and Disasters is a project of the German Aerospace Center (DLR) to develop tools for data acquisition, information extraction from several sources, decision support, and data distribution. Aerial images enable fast assessment of the situation of the target area and are therefore well suited for disaster relief applications. Each of the 4k camera systems, developed by the DLR, consists of three non-metric off-the-shelf-cameras, of which one is nadir looking, and two are side-wards looking. The 4k system can be mounted in along-track or across-track mode, in the two research aircrafts, Cessna 208B Grand Caravan and Dornier DO228-212. The camera system was further developed and implemented as a 4k camera system, which can be mounted on a helicopter BO105. The 4k system can acquire real time HD video with the nadir camera. As soon as the images are acquired, they are georeferenced with a real time GPS/IMU data and orthorectified using digital elevation model. According to the required information, additional thematic processing aboard is carried out, such as automatic traffic information extraction. Then, the images and extracted information are transmitted over a microwave data link to the ground station and distributed to the end users. The main backbone of the processing chain is the traffic data platform (TDP), where all acquired data and extracted traffic information and derived products are centrally stored. All the available data are combined to gain an overview of the situational information. The TDP is realized as a mobile mTDP or stationary sTDP. The former is used in mobile ground stations and the latter is centrally hosted at the DLR. The mTDP synchronizes with the sTDP whenever the internet connection is available. One example for a support system for decision makers, such as rescues forces and traffic management, is a web-based portal. It combines real-time aerial and ground-based traffic monitoring data and visualizes them. The traffic state is estimated and additionally, the traffic prognosis and simulations are computed to identify possible bottleneck places, e.g. in evacuation scenarios or for the logistics of the rescue forces. The VABENE++ system ensures real-time data availability through high degree of automation of the entire workflow. The mobile ground station and several platforms, on which the camera system can be mounted, enable usage of the complete system in various scenarios, from mapping applications to the vehicle tracking and traffic situation assessment. It was shown through several field-tests that the system works reliably and can serve as additional source of information for decision makers

3D Building Reconstruction from LIDAR Point Clouds by Adaptive Dual Contouring

This paper presents a novel workflow for data-driven building reconstruction from Light Detection and Ranging (LiDAR) point clouds. The method comprises building extraction, a detailed roof segmentation using region growing with adaptive thresholds, segment bound- ary creation, and a structural 3D building reconstruction approach using adaptive 2.5D Dual Contouring. First, a 2D-grid is overlain on the segmented point cloud. Second, in each grid cell 3D vertices of the building model are estimated from the corresponding LiDAR points. Then, the number of 3D vertices is reduced in a quad-tree collapsing procedure, and the remaining vertices are connected according to their adjacency in the grid. Roof segments are represented by a Triangular Irregular Network (TIN) and are connected to each other by common vertices or - at height discrepancies - by vertical walls. Resulting 3D building models show a very high accuracy and level of detail, including roof superstructures such as dormers. The workflow is tested and evaluated for two data sets, using the evaluation method and test data of the “ISPRS Test Project on Urban Classification and 3D Building Reconstruction” (Rottensteiner et al., 2012). Results show that the proposed method is comparable with the state of the art approaches, and outperforms them regarding undersegmentation and completeness of the scene reconstruction

Spectral Information Retrieval for Sub-Pixel Building Detection

Building extraction from imagery has been an active research area for decades. However, the precise building detection from hyperspectral (HSI) images solely is a less often addressed research question due to the low spatial resolution of data. The building boundaries are usually represented by spectrally mixed pixels, and classical edge detector algorithms fail to detect borders with sufficient completeness. The idea of the proposed method is to use fraction of materials in mixed pixels to derive weights for adjusting building boundaries. The building regions are detected using seeded region growing and merging in a HSI image; for the initial seed point selection the digital surface model (DSM) is used. Prior to region growing, the seeds are statistically tested for outliers on the basis of their spectral characteristics. Then, the border pixels of building regions are compared in spectrum to the seed points by calculating spectral dissimilarity. From this spectral dissimilarity the weights for weighted and constrained least squares (LS) adjustment are derived. We used the Spectral Angle Mapper (SAM) for spectral similarity measure, but the proposed boundary estimation method could benefit from soft classification or spectral unmixing results. The method was tested on a HSI image with spatial resolution of 4 m, and buildings of rectangular shape. The importance of constraints to the relations between building parts, e.g. perpendicularity is shown on example with a building with inner yards. The adjusted building boundaries are compared to the laser DSM, and have a relative accuracy of boundaries 1/4 of a pixel

Fusion of Hyperspectral Imags and Digital Surface Models for Urban Object Extraction

The topic of this thesis is the usage of remote sensing hyperspectral images and digital surface models for urban object extraction. A method for rectilinear building polygon extraction is proposed, which accounts for edge probabilities from both datasets. The edge probabilities are detected in a linear scale space and combined by a Bayesian fusion. They are introduced as weights in the adjustment of building polygons. A new quality measure for the evaluation of the extracted building polygons, named PoLiS metric, is defined and compared to the community accepted measures

Fusion of Hyperspectral Images and Height Models Using Edge Probability

We propose a method for data fusion of hyperspectral images (HSI) and digital surface models (DSM) basing on the edge probabilities from both datasets. A height discontinuity in DEM and change in material in HSI represent the high probability of an edge. Edge probabilities are computed in scale-space and combined according to the Gaussian mixture model. The reliability of the datasets can be included into this model as a prior knowledge. The method is tested on an urban area, where building boundaries represent the high probabilities of an edge in both datasets. Our results show, that the probabilistic fusion technique is dvantageous where boundary detected from only one dataset are unreliable