Real-time 3-D Reconstruction by Means of Structured Light Illumination

Structured light illumination (SLI) is the process of projecting a series of light striped patterns such that, when viewed at an angle, a digital camera can reconstruct a 3-D model of a target object\u27s surface. But by relying on a series of time multiplexed patterns, SLI is not typically associated with video applications. For this purpose of acquiring 3-D video, a common SLI technique is to drive the projector/camera pair at very high frame rates such that any object\u27s motion is small over the pattern set. But at these high frame rates, the speed at which the incoming video can be processed becomes an issue. So much so that many video-based SLI systems record camera frames to memory and then apply off-line processing. In order to overcome this processing bottleneck and produce 3-D point clouds in real-time, we present a lookup-table (LUT) based solution that in our experiments, using a 640 by 480 video stream, can generate intermediate phase data at 1063.8 frames per second and full 3-D coordinate point clouds at 228.3 frames per second. These achievements are 25 and 10 times faster than previously reported studies. At the same time, a novel dual-frequency pattern is developed which combines a high-frequency sinusoid component with a unit-frequency sinusoid component, where the high-frequency component is used to generate robust phase information and the unit-frequency component is used to reduce phase unwrapping ambiguities. Finally, we developed a gamma model for SLI, which can correct the non-linear distortion caused by the optical devices. For three-step phase measuring profilometry (PMP), analysis of the root mean squared error of the corrected phase showed a 60х reduction in phase error when the gamma calibration is performed versus 33х reduction without calibration

High-speed 3D imaging of liquid jets, surfaces and respiratory droplets

Sprays are commonly found in, among other, combustion, agriculture and food processing. For each of these applications, the understanding of spray liquid dynamics is crucial for optimization of efficiency, accuracy, and robustness of the spray­-system in use. Sprays are also found as a collection of respiratory droplets ejected when people are speaking, yelling, coughing etc. that is one of the main transmission routes for viral disease in the recent COVID­19 pandemic. The experimental research performed on these sprays is often in 2D and not seldom on average data. However, the spray dynamics of interest acts in 3D space, during very short timescales and are stochastically unique. Here, instantaneous high-­speed 3D imaging is required to fully characterize these events.This thesis applies and analyses three different laser­-based instantaneous high­-speed 3D imaging techniques on three different liquid dynamics. These include, (1) volumetric Laser Induced Fluorescence (LIF) imaging of liquid jets, (2) LIF structured illumination for surface 3D reconstruction of a liquid hollow cone sheet and (3) stereoscopic particle tracking velocimetry of respiratory droplets. The volumetric imaging was found to be challenging because of refractive effects at the liquid­-air interface. The structured illumination 3D reconstruction technique managed to reconstruct a transient 3D event where liquid breakups, ruptures, surface ­waves, and ejection angles were extracted. Simulations found that the used reconstruction was accurate to below 1% of the structure and could resolve small surface waves with a height up to 65% of the theoretical limit. Finally, the stereoscopic imaging extracted 3D tracks of respiratory droplets with found experimental average speed uncertainties around 0.3 m/s. In addition, this experiment enabled simultaneous estimation of speed and size of respiratory droplets that give valuable information on the risks of disease spreading.The presented instantaneous high­-speed 3D reconstruction techniques can provide data that paves the way towards a deeper understanding of liquid dynamics in general and sprays in particular. The data is advantageous partly since it can be directly applied by modellers to improve and validate their simulations. In the future, both more validation and application of the presented techniques are required which is enabled by the open-­sourced software and data that this thesis provides

Unsupervised Automatic Detection Of Transient Phenomena In InSAR Time-Series using Machine Learning

The detection and measurement of transient episodes of crustal deformation from global InSAR datasets are crucial for a wide range of solid earth and natural hazard applications. But the large volumes of unlabelled data captured by satellites preclude manual systematic analysis, and the small signal-to-noise ratio makes the task difficult. In this thesis, I present a state-of-the-art, unsupervised and event-agnostic deep-learning based approach for the automatic identification of transient deformation events in noisy time-series of unwrapped InSAR images. I adopt an anomaly detection framework that learns the ‘normal’ spatio-temporal pattern of noise in the data, and which therefore identifies any transient deformation phenomena that deviate from this pattern as ‘anomalies’. The deep-learning model is built around a bespoke autoencoder that includes convolutional and LSTM layers, as well as a neural network which acts as a bridge between the encoder and decoder. I train our model on real InSAR data from northern Turkey and find it has an overall accuracy and true positive rate of around 85% when trying to detect synthetic deformation signals of length-scale > 350 m and magnitude > 4 cm. Furthermore, I also show the method can detect (1) a real Mw 5.7 earthquake in InSAR data from an entirely different region- SW Turkey, (2) a volcanic deformation in Domuyo, Argentina, (3) a synthetic slow-slip event and (4) an interseismic deformation around NAF in a descending frame in northern Turkey. Overall I show that my method is suitable for automated analysis of large, global InSAR datasets, and for robust detection and separation of deformation signals from nuisance signals in InSAR data

Elevation and Deformation Extraction from TomoSAR

3D SAR tomography (TomoSAR) and 4D SAR differential tomography (Diff-TomoSAR) exploit multi-baseline SAR data stacks to provide an essential innovation of SAR Interferometry for many applications, sensing complex scenes with multiple scatterers mapped into the same SAR pixel cell. However, these are still influenced by DEM uncertainty, temporal decorrelation, orbital, tropospheric and ionospheric phase distortion and height blurring. In this thesis, these techniques are explored. As part of this exploration, the systematic procedures for DEM generation, DEM quality assessment, DEM quality improvement and DEM applications are first studied. Besides, this thesis focuses on the whole cycle of systematic methods for 3D & 4D TomoSAR imaging for height and deformation retrieval, from the problem formation phase, through the development of methods to testing on real SAR data. After DEM generation introduction from spaceborne bistatic InSAR (TanDEM-X) and airborne photogrammetry (Bluesky), a new DEM co-registration method with line feature validation (river network line, ridgeline, valley line, crater boundary feature and so on) is developed and demonstrated to assist the study of a wide area DEM data quality. This DEM co-registration method aligns two DEMs irrespective of the linear distortion model, which improves the quality of DEM vertical comparison accuracy significantly and is suitable and helpful for DEM quality assessment. A systematic TomoSAR algorithm and method have been established, tested, analysed and demonstrated for various applications (urban buildings, bridges, dams) to achieve better 3D & 4D tomographic SAR imaging results. These include applying Cosmo-Skymed X band single-polarisation data over the Zipingpu dam, Dujiangyan, Sichuan, China, to map topography; and using ALOS L band data in the San Francisco Bay region to map urban building and bridge. A new ionospheric correction method based on the tile method employing IGS TEC data, a split-spectrum and an ionospheric model via least squares are developed to correct ionospheric distortion to improve the accuracy of 3D & 4D tomographic SAR imaging. Meanwhile, a pixel by pixel orbit baseline estimation method is developed to address the research gaps of baseline estimation for 3D & 4D spaceborne SAR tomography imaging. Moreover, a SAR tomography imaging algorithm and a differential tomography four-dimensional SAR imaging algorithm based on compressive sensing, SAR interferometry phase (InSAR) calibration reference to DEM with DEM error correction, a new phase error calibration and compensation algorithm, based on PS, SVD, PGA, weighted least squares and minimum entropy, are developed to obtain accurate 3D & 4D tomographic SAR imaging results. The new baseline estimation method and consequent TomoSAR processing results showed that an accurate baseline estimation is essential to build up the TomoSAR model. After baseline estimation, phase calibration experiments (via FFT and Capon method) indicate that a phase calibration step is indispensable for TomoSAR imaging, which eventually influences the inversion results. A super-resolution reconstruction CS based study demonstrates X band data with the CS method does not fit for forest reconstruction but works for reconstruction of large civil engineering structures such as dams and urban buildings. Meanwhile, the L band data with FFT, Capon and the CS method are shown to work for the reconstruction of large manmade structures (such as bridges) and urban buildings

A real-time multi-sensor 3D surface shape measurement system using fringe analysis

This thesis presents a state-of-the-art multi-sensor, 3D surface shape measurement system that is based upon fringe projection/analysis and which operates at speeds approaching real-time. The research programme was carried out as part of MEGURATH (www.megurath.org), a collaborative research project with the aim of improving the treatment of cancer by radiotherapy. The aim of this research programme was to develop a real-time, multi-sensor 3D surface shape measurement system that is based on fringe analysis, which provides the flexibility to choose from amongst several different fringe profilometry methods and to manipulate their settings interactively. The system has been designed specifically to measure dynamic 3D human body surface shape and to act as an enabling technology for the purpose of performing Metrology Guided Radiotherapy (MGRT). However, the system has a wide variety of other potential applications, including 3D modelling and visualisation, verbatim replication, reverse engineering and industrial inspection. It can also be used as a rapid prototyping tool for algorithm development and testing, within the field of fringe pattern profilometry. The system that has been developed provides single, or multi-sensor, measurement modes that are adaptable to the specific requirements of a desired application. The multi-sensor mode can be useful for covering a larger measurement area, by providing a multi-viewpoint measurement. The overall measurement accuracy of the system is better than O.5mm, with measurement speeds of up to 3 million XYZ points/second using the single-sensor mode and rising to up to 4.6 million XYZ points/second when measuring in parallel using the three sensor multi-sensor mode. In addition the system provides a wide-ranging catalogue of fringe profilometry methods and techniques, that enables the reconstruction of 3D information through an interactive user selection of 183 possible different paths of main combinations. The research aspects behind the development of the system are presented in this thesis, along with the author's contribution to this field of research, which has included the provision of a comprehensive framework for producing such a novel optical profilometry system, and the specific techniques that were developed to fulfil the aims of this research programme. This mainly included the following advanced methods: a transversal calibration method for the optical system, an adaptive filtering technique for the Fourier Transform Profilometry (FTP) method, and a method to synthetically restore the locations of the triangulation spots. Similarly, potential applications for the system have been presented and feasibility and accuracy analyses have been conducted, presenting both qualitative and quantitative measurement results. To this end, the high robustness levels exhibited by the system have been demonstrated (in terms of adaptability, accuracy and measurement capability) by performing extensive real experiments and laboratory testing. Finally, a number of potential future system developments are described, with the intention of further extending the system capabilities