Enabling Neural Radiance Fields (NeRF) for Large-scale Aerial Images -- A Multi-tiling Approach and the Geometry Assessment of NeRF

Neural Radiance Fields (NeRF) offer the potential to benefit 3D reconstruction tasks, including aerial photogrammetry. However, the scalability and accuracy of the inferred geometry are not well-documented for large-scale aerial assets,since such datasets usually result in very high memory consumption and slow convergence.. In this paper, we aim to scale the NeRF on large-scael aerial datasets and provide a thorough geometry assessment of NeRF. Specifically, we introduce a location-specific sampling technique as well as a multi-camera tiling (MCT) strategy to reduce memory consumption during image loading for RAM, representation training for GPU memory, and increase the convergence rate within tiles. MCT decomposes a large-frame image into multiple tiled images with different camera models, allowing these small-frame images to be fed into the training process as needed for specific locations without a loss of accuracy. We implement our method on a representative approach, Mip-NeRF, and compare its geometry performance with threephotgrammetric MVS pipelines on two typical aerial datasets against LiDAR reference data. Both qualitative and quantitative results suggest that the proposed NeRF approach produces better completeness and object details than traditional approaches, although as of now, it still falls short in terms of accuracy.Comment: 9 Figur

Variable Resolution & Dimensional Mapping For 3d Model Optimization

Three-dimensional computer models, especially geospatial architectural data sets, can be visualized in the same way humans experience the world, providing a realistic, interactive experience. Scene familiarization, architectural analysis, scientific visualization, and many other applications would benefit from finely detailed, high resolution, 3D models. Automated methods to construct these 3D models traditionally has produced data sets that are often low fidelity or inaccurate; otherwise, they are initially highly detailed, but are very labor and time intensive to construct. Such data sets are often not practical for common real-time usage and are not easily updated. This thesis proposes Variable Resolution & Dimensional Mapping (VRDM), a methodology that has been developed to address some of the limitations of existing approaches to model construction from images. Key components of VRDM are texture palettes, which enable variable and ultra-high resolution images to be easily composited; texture features, which allow image features to integrated as image or geometry, and have the ability to modify the geometric model structure to add detail. These components support a primary VRDM objective of facilitating model refinement with additional data. This can be done until the desired fidelity is achieved as practical limits of infinite detail are approached. Texture Levels, the third component, enable real-time interaction with a very detailed model, along with the flexibility of having alternate pixel data for a given area of the model and this is achieved through extra dimensions. Together these techniques have been used to construct models that can contain GBs of imagery data

Review article: The use of remotely piloted aircraft systems (RPASs) for natural hazards monitoring and management

The number of scientific studies that consider possible applications of remotely piloted aircraft systems (RPASs) for the management of natural hazards effects and the identification of occurred damages strongly increased in the last decade. Nowadays, in the scientific community, the use of these systems is not a novelty, but a deeper analysis of the literature shows a lack of codified complex methodologies that can be used not only for scientific experiments but also for normal codified emergency operations. RPASs can acquire on-demand ultra-high-resolution images that can be used for the identification of active processes such as landslides or volcanic activities but can also define the effects of earthquakes, wildfires and floods. In this paper, we present a review of published literature that describes experimental methodologies developed for the study and monitoring of natural hazard

The Design Fabrication and Flight Testing of an Academic Research Platform for High Resolution Terrain Imaging

This thesis addresses the design, construction, and flight testing of an Unmanned Aircraft System (UAS) created to serve as a testbed for Intelligence, Surveillance, and Reconnaissance (ISR) research topics that require the rapid acquisition and processing of high resolution aerial imagery and are to be performed by academic research institutions. An analysis of the requirements of various ISR research applications and the practical limitations of academic research yields a consolidated set of requirements by which the UAS is designed. An iterative design process is used to transition from these requirements to cycles of component selection, systems integration, flight tests, diagnostics, and subsystem redesign. The resulting UAS is designed as an academic research platform to support a variety of ISR research applications ranging from human machine interaction with UAS technology to orthorectified mosaic imaging. The lessons learned are provided to enable future researchers to create similar systems

Scan4Façade: Automated As-Is Façade Modeling of Historic High-Rise Buildings Using Drones and AI

This paper presents an automated as-is façade modeling method for existing and historic high-rise buildings, named Scan4Façade. To begin with, a camera drone with a spiral path is employed to capture building exterior images, and photogrammetry is used to conduct three-dimensional (3D) reconstruction and create mesh models for the scanned building façades. High-resolution façade orthoimages are then generated from mesh models and pixelwise segmented by an artificial intelligence (AI) model named U-net. A combined data augmentation strategy, including random flipping, rotation, resizing, perspective transformation, and color adjustment, is proposed for model training with a limited number of labels. As a result, the U-net achieves an average pixel accuracy of 0.9696 and a mean intersection over union of 0.9063 in testing. Then, the developed twoStagesClustering algorithm, with a two-round shape clustering and a two-round coordinates clustering, is used to precisely extract façade elements’ dimensions and coordinates from façade orthoimages and pixelwise label. In testing with the Michigan Central Station (office tower), a historic high-rise building, the developed algorithm achieves an accuracy of 99.77% in window extraction. In addition, the extracted façade geometric information and element types are transformed into AutoCAD command and script files to create CAD drawings without manual interaction. Experimental results also show that the proposed Scan4Façade method can provide clear and accurate information to assist BIM feature creation in Revit. Future research recommendations are also stated in this paper

Deep Learning Methods for 3D Aerial and Satellite Data

Recent advances in digital electronics have led to an overabundance of observations from electro-optical (EO) imaging sensors spanning high spatial, spectral and temporal resolution. This unprecedented volume, variety, and velocity is overwhelming our capacity to manage and translate that data into actionable information. Although decades of image processing research have taken the human out of the loop for many important tasks, the human analyst is still an irreplaceable link in the image exploitation chain, especially for more complex tasks requiring contextual understanding, memory, discernment, and learning. If knowledge discovery is to keep pace with the growing availability of data, new processing paradigms are needed in order to automate the analysis of earth observation imagery and ease the burden of manual interpretation. To address this gap, this dissertation advances fundamental and applied research in deep learning for aerial and satellite imagery. We show how deep learning---a computational model inspired by the human brain---can be used for (1) tracking, (2) classifying, and (3) modeling from a variety of data sources including full-motion video (FMV), Light Detection and Ranging (LiDAR), and stereo photogrammetry. First we assess the ability of a bio-inspired tracking method to track small targets using aerial videos. The tracker uses three kinds of saliency maps: appearance, location, and motion. Our approach achieves the best overall performance, including being the only method capable of handling long-term occlusions. Second, we evaluate the classification accuracy of a multi-scale fully convolutional network to label individual points in LiDAR data. Our method uses only the 3D-coordinates and corresponding low-dimensional spectral features for each point. Evaluated using the ISPRS 3D Semantic Labeling Contest, our method scored second place with an overall accuracy of 81.6\%. Finally, we validate the prediction capability of our neighborhood-aware network to model the bare-earth surface of LiDAR and stereo photogrammetry point clouds. The network bypasses traditionally-used ground classifications and seamlessly integrate neighborhood features with point-wise and global features to predict a per point Digital Terrain Model (DTM). We compare our results with two widely used softwares for DTM extraction, ENVI and LAStools. Together, these efforts have the potential to alleviate the manual burden associated with some of the most challenging and time-consuming geospatial processing tasks, with implications for improving our response to issues of global security, emergency management, and disaster response