Continuous Modeling of 3D Building Rooftops From Airborne LIDAR and Imagery

In recent years, a number of mega-cities have provided 3D photorealistic virtual models to support the decisions making process for maintaining the cities' infrastructure and environment more effectively. 3D virtual city models are static snap-shots of the environment and represent the status quo at the time of their data acquisition. However, cities are dynamic system that continuously change over time. Accordingly, their virtual representation need to be regularly updated in a timely manner to allow for accurate analysis and simulated results that decisions are based upon. The concept of "continuous city modeling" is to progressively reconstruct city models by accommodating their changes recognized in spatio-temporal domain, while preserving unchanged structures. However, developing a universal intelligent machine enabling continuous modeling still remains a challenging task. Therefore, this thesis proposes a novel research framework for continuously reconstructing 3D building rooftops using multi-sensor data. For achieving this goal, we first proposes a 3D building rooftop modeling method using airborne LiDAR data. The main focus is on the implementation of an implicit regularization method which impose a data-driven building regularity to noisy boundaries of roof planes for reconstructing 3D building rooftop models. The implicit regularization process is implemented in the framework of Minimum Description Length (MDL) combined with Hypothesize and Test (HAT). Secondly, we propose a context-based geometric hashing method to align newly acquired image data with existing building models. The novelty is the use of context features to achieve robust and accurate matching results. Thirdly, the existing building models are refined by newly proposed sequential fusion method. The main advantage of the proposed method is its ability to progressively refine modeling errors frequently observed in LiDAR-driven building models. The refinement process is conducted in the framework of MDL combined with HAT. Markov Chain Monte Carlo (MDMC) coupled with Simulated Annealing (SA) is employed to perform a global optimization. The results demonstrates that the proposed continuous rooftop modeling methods show a promising aspects to support various critical decisions by not only reconstructing 3D rooftop models accurately, but also by updating the models using multi-sensor data

Extracting Agricultural Fields from Remote Sensing Imagery Using Graph-Based Growing Contours

Knowledge of the location and extent of agricultural fields is required for many applications, including agricultural statistics, environmental monitoring, and administrative policies. Furthermore, many mapping applications, such as object-based classification, crop type distinction, or large-scale yield prediction benefit significantly from the accurate delineation of fields. Still, most existing field maps and observation systems rely on historic administrative maps or labor-intensive field campaigns. These are often expensive to maintain and quickly become outdated, especially in regions of frequently changing agricultural patterns. However, exploiting openly available remote sensing imagery (e.g., from the European Union’s Copernicus programme) may allow for frequent and efficient field mapping with minimal human interaction. We present a new approach to extracting agricultural fields at the sub-pixel level. It consists of boundary detection and a field polygon extraction step based on a newly developed, modified version of the growing snakes active contours model we refer to as graph-based growing contours. This technique is capable of extracting complex networks of boundaries present in agricultural landscapes, and is largely automatic with little supervision required. The whole detection and extraction process is designed to work independently of sensor type, resolution, or wavelength. As a test case, we applied the method to two regions of interest in a study area in the northern Germany using multi-temporal Sentinel-2 imagery. Extracted fields were compared visually and quantitatively to ground reference data. The technique proved reliable in producing polygons closely matching reference data, both in terms of boundary location and statistical proxies such as median field size and total acreage

Signal-to-Noise Behavior for Matches to Gradient Direction Models of Corners in Images

Gradient direction models for corners of prescribed acuteness, leg length, and leg thickness are constructed by generating fields of unit vectors emanating from leg pixels that point normal to the edges. A novel FFT-based algorithm that quickly matches models of corners at all possible positions and orientations in the image to fields of gradient directions for image pixels is described. The signal strength of a corner is discussed in terms of the number of pixels along the edges of a corner in an image, while noise is characterized by the coherence of gradient directions along those edges. The detection-false alarm rate behavior of our corner detector is evaluated empirically by manually constructing maps of corner locations in typical overhead images, and then generating different ROC curves for matches to models of corners with different leg lengths and thicknesses. We then demonstrate how corners found with our detector can be used to quickly and automatically find families of polygons of arbitrary position, size and orientation in overhead images

Grounding semantics in robots for Visual Question Answering

In this thesis I describe an operational implementation of an object detection and description system that incorporates in an end-to-end Visual Question Answering system and evaluated it on two visual question answering datasets for compositional language and elementary visual reasoning

Automated Image Registration And Mosaicking For Multi-Sensor Images Acquired By A Miniature Unmanned Aerial Vehicle Platform

Algorithms for automatic image registration and mosaicking are developed for a miniature Unmanned Aerial Vehicle (MINI-UAV) platform, assembled by Air-O-Space International (AOSI) L.L.C.. Three cameras onboard this MINI-UAV platform acquire images in a single frame simultaneously at green (550nm), red (650 nm), and near infrared (820nm) wavelengths, but with shifting and rotational misalignment. The area-based method is employed in the developed algorithms for control point detection, which is applicable when no prominent feature details are present in image scenes. Because the three images to be registered have different spectral characteristics, region of interest determination and control point selection are the two key steps that ensure the quality of control points. Affine transformation is adopted for spatial transformation, followed by bilinear interpolation for image resampling. Mosaicking is conducted between adjacent frames after three-band co-registration. Pre-introducing the rotation makes the area-based method feasible when the rotational misalignment cannot be ignored. The algorithms are tested on three image sets collected at Stennis Space Center, Greenwood, and Oswalt in Mississippi. Manual evaluation confirms the effectiveness of the developed algorithms. The codes are converted into a software package, which is executable under the Microsoft Windows environment of personal computer platforms without the requirement of MATLAB or other special software support for commercial-off-the-shelf (COTS) product. The near real-time decision-making support is achievable with final data after its installation into the ground control station. The final products are color-infrared (CIR) composite and normalized difference vegetation index (NDVI) images, which are used in agriculture, forestry, and environmental monitoring

PyTrx : a python-based monoscopic terrestrial photogrammetry toolset for glaciology

This work was affiliated with the CRIOS project (Calving Rates and Impact On Sea Level), which was supported by the Conoco Phillips-Lundin Northern Area Program. PH was funded by a NERC Ph.D. studentship (reference number 1396698).Terrestrial time-lapse photogrammetry is a rapidly growing method for deriving measurements from glacial environments because it provides high spatio-temporal resolution records of change. Currently, however, the potential usefulness of time-lapse data is limited by the unavailability of user-friendly photogrammetry toolsets. Such data are used primarily to calculate ice flow velocities or to serve as qualitative records. PyTrx (available at https://github.com/PennyHow/PyTrx) is presented here as a Python-alternative toolset to widen the range of monoscopic photogrammetry (i.e., from a single viewpoint) toolsets on offer to the glaciology community. The toolset holds core photogrammetric functions for template generation, feature-tracking, camea calibration and optimization, image registration, and georectification (using a planar projective transformation model). In addition, PyTrx facilitates areal and line measurements, which can be detected from imagery using either an automated or manual approach. Examples of PyTrx's applications are demonstrated using time-lapse imagery from Kronebreen and Tunabreen, two tidewater glaciers in Svalbard. Products from these applications include ice flow velocities, surface areas of supraglacial lakes and meltwater plumes, and glacier terminus profiles.Publisher PDFPeer reviewe

Direct occlusion handling for high level image processing algorithms

Many high-level computer vision algorithms suffer in the presence of occlusions caused by multiple objects overlapping in a view. Occlusions remove the direct correspondence between visible areas of objects and the objects themselves by introducing ambiguity in the interpretation of the shape of the occluded object. Ignoring this ambiguity allows the perceived geometry of overlapping objects to be deformed or even fractured. Supplementing the raw image data with a vectorized structural representation which predicts object completions could stabilize high-level algorithms which currently disregard occlusions. Studies in the neuroscience community indicate that the feature points located at the intersection of junctions may be used by the human visual system to produce these completions. Geiger, Pao, and Rubin have successfully used these features in a purely rasterized setting to complete objects in a fashion similar to what is demonstrated by human perception. This work proposes using these features in a vectorized approach to solving the mid-level computer vision problem of object stitching. A system has been implemented which is able extract L and T-junctions directly from the edges of an image using scale-space and robust statistical techniques. The system is sensitive enough to be able to isolate the corners on polygons with 24 sides or more, provided sufficient image resolution is available. Areas of promising development have been identified and several directions for further research are proposed

Building Footprint Extraction from LiDAR Data and Imagery Information

This study presents an automatic method for regularisation of building outlines. Initially, building segments are extracted using a new fusion method. Data- and model-driven approaches are then combined to generate approximate building polygons. The core part of the method includes a novel data-driven algorithm based on likelihood equation derived from the geometrical properties of a building. Finally, the Gauss-Helmert and Gauss-Markov models adjustment are implemented and modified for regularisation of building outlines considering orthogonality constraints