Transformation Model With Constraints for High Accuracy of 2D-3D Building Registration in Aerial Imagery

This paper proposes a novel rigorous transformation model for 2D-3D registration to address the difficult problem of obtaining a sufficient number of well-distributed ground control points (GCPs) in urban areas with tall buildings. The proposed model applies two types of geometric constraints, co-planarity and perpendicularity, to the conventional photogrammetric collinearity model. Both types of geometric information are directly obtained from geometric building structures, with which the geometric constraints are automatically created and combined into the conventional transformation model. A test field located in downtown Denver, Colorado, is used to evaluate the accuracy and reliability of the proposed method. The comparison analysis of the accuracy achieved by the proposed method and the conventional method is conducted. Experimental results demonstrated that: (1) the theoretical accuracy of the solved registration parameters can reach 0.47 pixels, whereas the other methods reach only 1.23 and 1.09 pixels; (2) the RMS values of 2D-3D registration achieved by the proposed model are only two pixels along the x and y directions, much smaller than the RMS values of the conventional model, which are approximately 10 pixels along the x and y directions. These results demonstrate that the proposed method is able to significantly improve the accuracy of 2D-3D registration with much fewer GCPs in urban areas with tall buildings

CNN-Based Initial Localization Improved by Data Augmentation

Image-based localization or camera re-localization is a fundamental task in computer vision and mandatory in the fields of navigation for robotics and autonomous driving or for virtual and augmented reality. Such image pose regression in 6 Degrees of Freedom (DoF) is recently solved by Convolutional Neural Networks (CNNs). However, already well-established methods based on feature matching still score higher accuracies so far. Therefore, we want to investigate how data augmentation could further improve CNN-based pose regression. Data augmentation is a valuable technique to boost performance on training based methods and wide spread in the computer vision community. Our aim in this paper is to show the benefit of data augmentation for pose regression by CNNs. For this purpose images are rendered from a 3D model of the actual test environment. This model again is generated by the original training data set, whereas no additional information nor data is required. Furthermore we introduce different training sets composed of rendered and real images. It is shown that the enhanced training of CNNs by utilizing 3D models of the environment improves the image localization accuracy. The accuracy of pose regression could be improved up to 69.37% for the position component and 61.61% for the rotation component on our investigated data set

UAS Navigation with SqueezePoseNet—Accuracy Boosting for Pose Regression by Data Augmentation

The navigation of Unmanned Aerial Vehicles (UAVs) nowadays is mostly based on Global Navigation Satellite Systems (GNSSs). Drawbacks of satellite-based navigation are failures caused by occlusions or multi-path interferences. Therefore, alternative methods have been developed in recent years. Visual navigation methods such as Visual Odometry (VO) or visual Simultaneous Localization and Mapping (SLAM) aid global navigation solutions by closing trajectory gaps or performing loop closures. However, if the trajectory estimation is interrupted or not available, a re-localization is mandatory. Furthermore, the latest research has shown promising results on pose regression in 6 Degrees of Freedom (DoF) based on Convolutional Neural Networks (CNNs). Additionally, existing navigation methods can benefit from these networks. In this article, a method for GNSS-free and fast image-based pose regression by utilizing a small Convolutional Neural Network is presented. Therefore, a small CNN SqueezePoseNet) is utilized, transfer learning is applied and the network is tuned for pose regression. Furthermore, recent drawbacks are overcome by applying data augmentation on a training dataset utilizing simulated images. Experiments with small CNNs show promising results for GNSS-free and fast localization compared to larger networks. By training a CNN with an extended data set including simulated images, the accuracy on pose regression is improved up to 61.7% for position and up to 76.0% for rotation compared to training on a standard not-augmented data set

Automatic 3-D Building Model Reconstruction from Very High Resolution Stereo Satellite Imagery

Recent advances in the availability of very high-resolution (VHR) satellite data together with efficient data acquisition and large area coverage have led to an upward trend in their applications for automatic 3-D building model reconstruction which require large-scale and frequent updates, such as disaster monitoring and urban management. Digital Surface Models (DSMs) generated from stereo satellite imagery suffer from mismatches, missing values, or blunders, resulting in rough building shape representations. To handle 3-D building model reconstruction using such low-quality DSMs, we propose a novel automatic multistage hybrid method using DSMs together with orthorectified panchromatic (PAN) and pansharpened data (PS) of multispectral (MS) satellite imagery. The algorithm consists of multiple steps including building boundary extraction and decomposition, image-based roof type classification, and initial roof parameter computation which are prior knowledge for the 3-D model fitting step. To fit 3-D models to the normalized DSM (nDSM) and to select the best one, a parameter optimization method based on exhaustive search is used sequentially in 2-D and 3-D. Finally, the neighboring building models in a building block are intersected to reconstruct the 3-D model of connecting roofs. All corresponding experiments are conducted on a dataset including four different areas of Munich city containing 208 buildings with different degrees of complexity. The results are evaluated both qualitatively and quantitatively. According to the results, the proposed approach can reliably reconstruct 3-D building models, even the complex ones with several inner yards and multiple orientations. Furthermore, the proposed approach provides a high level of automation by limiting the number of primitive roof types and by performing automatic parameter initialization

Methodologies for City-scale Microgeneration Viability Assessment

Over the last decade, increasing numbers of multi-national corporations, public institutions and individual property owners have become interested in installing solar photovoltaics and small wind turbines. To best inform this broad range of actors, this research aims to assess the financial viability of such investments across broad city regions whilst maintaining accuracy at individual properties. Publicly available digital representations of urban surfaces are central to meeting this aim because they can be used to assess the area, slope and orientation of potential solar photovoltaic (PV) installation sites and to define how vertical wind profiles are altered by urban areas. A first study utilised digital surface models (DSMs) across seven UK cities to assess the roof spaces available for solar PV and also incorporated socio-economic factors to define the propensity for cities to install the technology. Despite changes to financial incentives that had recently occurred, the technologies remained viable at a very large number of locations and could theoretically meet large percentages (16% to 43%) of the cities’ electricity demands. The accuracy of slope, orientation and available area estimation in roof geometry modelling was then improved through the development of a neighbouring buildings method. In 87% of 536 validated results, the method identified the correct roof shape and roof slope was estimated to a mean absolute error of 3.76° when compared to 182 measured roofs. Work was then undertaken to improve solar insolation modelling. A radiative transfer model was created that incorporated shading based on DSM data. It estimated the power output of 17 solar PV installations across four UK cities with +2.62% mean percentage error when its 2013 insolation estimates were converted to power outputs using a 0.8 performance ratio. The validation data showed that the RTS model outperformed the market-leading esri ArcMap solar radiation software which incurred a -15.97% mean percentage error. This method was then adapted to be deployable on a city scale and predicted solar insolation with a mean percentage error of -4.39% despite the process being made far more computationally efficient. A method to estimate long-term average wind speeds for urban areas was then developed that produced results of comparable accuracy to an existing model but with considerably reduced computational demand and complexity in deployment. The mean absolute error inwind speed estimation was just 1.75% greater using the simplified methodology than the existing model. Finally, the improved modelling of roof geometries, solar insolation and long-term mean wind speed were brought together to evaluate the city-scale potential for solar PV and small to medium wind microgeneration. The research has shown that wind and solar PV microgeneration at sites that pay back within nine years could theoretically meet 88.5% of annual domestic electricity demand in the city of Leeds, or would be the equivalent of providing electricity to 300,319 homes. Current financial contexts were used to define a baseline scenario from which hypothetical changes to a variety of factors influencing microgeneration viability were investigated. When the costs and revenues were defined from a pessimistic, but still realistic, perspective the percentage of the study area’s electricity demand that could theoretically be met by wind and solar PV microgeneration fell to 0.1%. This suggests that government policy will continue to play a key role in the future growth of UK wind and solar PV deployment

Photorealistic large-scale urban city model reconstruction

The rapid and efficient creation of virtual environments has become a crucial part of virtual reality applications. In particular, civil and defense applications often require and employ detailed models of operations areas for training, simulations of different scenarios, planning for natural or man-made events, monitoring, surveillance, games, and films. A realistic representation of the large-scale environments is therefore imperative for the success of such applications since it increases the immersive experience of its users and helps reduce the difference between physical and virtual reality. However, the task of creating such large-scale virtual environments still remains a time-consuming and manual work. In this work, we propose a novel method for the rapid reconstruction of photorealistic large-scale virtual environments. First, a novel, extendible, parameterized geometric primitive is presented for the automatic building identification and reconstruction of building structures. In addition, buildings with complex roofs containing complex linear and nonlinear surfaces are reconstructed interactively using a linear polygonal and a nonlinear primitive, respectively. Second, we present a rendering pipeline for the composition of photorealistic textures, which unlike existing techniques, can recover missing or occluded texture information by integrating multiple information captured from different optical sensors (ground, aerial, and satellite)