Holoscopic Elemental-Image-Based Disparity Estimation Using Multi-Scale, Multi-Window Semi-Global Block Matching

Data Availability Statement: The data presented in this study are available on request from the corresponding author, Bodor Almatrouk, at [email protected]. The data are not publicly available due to commercial privacy.In Holoscopic imaging, a single aperture is used to acquire full-colour spatial images like a fly’s eye by gently altering angles between nearby lenses with a micro-lens array. Due to its simple data collection and visualisation methods, which provide robust and scalable spatial information, and its motion parallax, binocular disparity, and convergence, this technique may be able to overcome traditional 2D imaging issues like depth, scalability, and multi-perspective problems. A novel disparity-map-generating method uses angular information from a single Holoscopic image’s micro-images, or Elemental Images (EIs), to create a scene’s disparity map. Not much research has used EIs instead of Viewpoint Images (VPIs) for disparity estimation. This study investigates whether angular perspective data may replace spatial orthographic data. Using noise reduction and contrast enhancement, EIs with a low resolution and lack of texture are pre-processed to calculate the disparity. The Semi-Global Block Matching (SGBM) technique is used to calculate the disparity between EI pixels. A multi-resolution approach overcomes EIs’ resolution constraints, and a content-aware analysis dynamically modifies the SGBM window size settings to generate disparities across different texture and complexity levels. A background mask and nearby EIs with accurate backgrounds detect and rectify EIs with erroneous backgrounds. Our method generates disparity maps that outperform two state-of-the-art deep learning algorithms and VPIs in real images.This research received no external funding

3D cloud envelope and cloud development velocity from simulated CLOUD (C3IEL) stereo images

A method to derive the 3D cloud envelope and the cloud development velocity from high spatial and temporal resolution satellite imagery is presented. The CLOUD instrument of the recently proposed C3IEL mission lends itself well to observing at high spatial and temporal resolutions the development of convective cells. Space-borne visible cameras simultaneously image, under multiple view angles, the same surface domain every 20 s over a time interval of 200 s. In this paper, we present a method for retrieving cloud development velocity from simulated multi-angular, high-resolution top of the atmosphere (TOA) radiance cloud fields. The latter are obtained via the image renderer Mitsuba for a cumulus case generated via the atmospheric research model SAM and via the radiative transfer model 3DMCPOL, coupled with the outputs of an orbit, attitude, and camera simulator for a deep convective cloud case generated via the atmospheric research model Meso-NH. Matching cloud features are found between simulations via block matching. Image coordinates of tie points are mapped to spatial coordinates via 3D stereo reconstruction of the external cloud envelope for each acquisition. The accuracy of the retrieval of cloud topography is quantified in terms of RMSE and bias that are, respectively, less than 25 and 5 m for the horizontal components and less than 40 and 25 m for the vertical components. The inter-acquisition 3D velocity is then derived for each pair of tie points separated by 20 s. An independent method based on minimising the RMSE for a continuous horizontal shift of the cloud top, issued from the atmospheric research model, allows for the obtainment of a ground estimate of the velocity from two consecutive acquisitions. The mean values of the distributions of the stereo and ground velocities exhibit small biases. The width of the distributions is significantly different, with higher a distribution width for the stereo-retrieved velocity. An alternative way to derive an average velocity over 200 s, which relies on tracking clusters of points via image feature matching over several acquisitions, was also implemented and tested. For each cluster of points, mean stereo and ground positions were derived every 20 s over 200 s. The mean stereo and ground velocities, obtained as the slope of the line of best fit to the mean positions, are in good agreement.</p

Dense Point-Cloud Representation of a Scene using Monocular Vision

We present a three-dimensional (3-D) reconstruction system designed to support various autonomous navigation applications. The system presented focuses on the 3-D reconstruction of a scene using only a single moving camera. Utilizing video frames captured at different points in time allows us to determine the depths of a scene. In this way, the system can be used to construct a point-cloud model of its unknown surroundings. We present the step-by-step methodology and analysis used in developing the 3-D reconstruction technique. We present a reconstruction framework that generates a primitive point cloud, which is computed based on feature matching and depth triangulation analysis. To populate the reconstruction, we utilized optical flow features to create an extremely dense representation model. With the third algorithmic modification, we introduce the addition of the preprocessing step of nonlinear single-image super resolution. With this addition, the depth accuracy of the point cloud, which relies on precise disparity measurement, has significantly increased. Our final contribution is an additional postprocessing step designed to filter noise points and mismatched features unveiling the complete dense point-cloud representation (DPR) technique. We measure the success of DPR by evaluating the visual appeal, density, accuracy, and computational expense and compare with two state-of-the-art techniques

Locally Adaptive Stereo Vision Based 3D Visual Reconstruction

abstract: Using stereo vision for 3D reconstruction and depth estimation has become a popular and promising research area as it has a simple setup with passive cameras and relatively efficient processing procedure. The work in this dissertation focuses on locally adaptive stereo vision methods and applications to different imaging setups and image scenes. Solder ball height and substrate coplanarity inspection is essential to the detection of potential connectivity issues in semi-conductor units. Current ball height and substrate coplanarity inspection tools are expensive and slow, which makes them difficult to use in a real-time manufacturing setting. In this dissertation, an automatic, stereo vision based, in-line ball height and coplanarity inspection method is presented. The proposed method includes an imaging setup together with a computer vision algorithm for reliable, in-line ball height measurement. The imaging setup and calibration, ball height estimation and substrate coplanarity calculation are presented with novel stereo vision methods. The results of the proposed method are evaluated in a measurement capability analysis (MCA) procedure and compared with the ground-truth obtained by an existing laser scanning tool and an existing confocal inspection tool. The proposed system outperforms existing inspection tools in terms of accuracy and stability. In a rectified stereo vision system, stereo matching methods can be categorized into global methods and local methods. Local stereo methods are more suitable for real-time processing purposes with competitive accuracy as compared with global methods. This work proposes a stereo matching method based on sparse locally adaptive cost aggregation. In order to reduce outlier disparity values that correspond to mis-matches, a novel sparse disparity subset selection method is proposed by assigning a significance status to candidate disparity values, and selecting the significant disparity values adaptively. An adaptive guided filtering method using the disparity subset for refined cost aggregation and disparity calculation is demonstrated. The proposed stereo matching algorithm is tested on the Middlebury and the KITTI stereo evaluation benchmark images. A performance analysis of the proposed method in terms of the I0 norm of the disparity subset is presented to demonstrate the achieved efficiency and accuracy.Dissertation/ThesisDoctoral Dissertation Electrical Engineering 201

Disaster debris estimation using high-resolution polarimetric stereo-SAR

AbstractThis paper addresses the problem of debris estimation which is one of the most important initial challenges in the wake of a disaster like the Great East Japan Earthquake and Tsunami. Reasonable estimates of the debris have to be made available to decision makers as quickly as possible. Current approaches to obtain this information are far from being optimal as they usually rely on manual interpretation of optical imagery. We have developed a novel approach for the estimation of tsunami debris pile heights and volumes for improved emergency response. The method is based on a stereo-synthetic aperture radar (stereo-SAR) approach for very high-resolution polarimetric SAR. An advanced gradient-based optical-flow estimation technique is applied for optimal image coregistration of the low-coherence non-interferometric data resulting from the illumination from opposite directions and in different polarizations. By applying model based decomposition of the coherency matrix, only the odd bounce scattering contributions are used to optimize echo time computation. The method exclusively considers the relative height differences from the top of the piles to their base to achieve a very fine resolution in height estimation. To define the base, a reference point on non-debris-covered ground surface is located adjacent to the debris pile targets by exploiting the polarimetric scattering information. The proposed technique is validated using in situ data of real tsunami debris taken on a temporary debris management site in the tsunami affected area near Sendai city, Japan. The estimated height error is smaller than 0.6m RMSE. The good quality of derived pile heights allows for a voxel-based estimation of debris volumes with a RMSE of 1099m3. Advantages of the proposed method are fast computation time, and robust height and volume estimation of debris piles without the need for pre-event data or auxiliary information like DEM, topographic maps or GCPs

Characteristics of flight simulator visual systems

The physical parameters of the flight simulator visual system that characterize the system and determine its fidelity are identified and defined. The characteristics of visual simulation systems are discussed in terms of the basic categories of spatial, energy, and temporal properties corresponding to the three fundamental quantities of length, mass, and time. Each of these parameters are further addressed in relation to its effect, its appropriate units or descriptors, methods of measurement, and its use or importance to image quality

Aerospace medicine and biology: A continuing bibliography with indexes (supplement 323)

This bibliography lists 125 reports, articles and other documents introduced into the NASA Scientific and Technical Information System during April, 1989. Subject coverage includes; aerospace medicine and psychology, life support systems and controlled environments, safety equipment exobiology and extraterrestrial life, and flight crew behavior and performance

High-accuracy digital elevation model generation and ship monitoring from synthetic aperture radar images: innovative techniques and experimental results.

In this Thesis several state-of-the-art and innovative techniques for Digital Elevation Model (DEM) generation from Synthetic Aperture Radar (SAR) images are deeply analyzed, with a special focus on the methods which allow the improvement of the accuracy of the DEM product, which is directly related to the geolocation accuracy of geocoded images and is considered as an enabling factor for a large series of civilian and Defence applications. Furthermore, some of the proposed techniques, which are based both on phase and amplitude information, are experimented on real data, i.e. COSMO-SkyMed (CSK) data, assessing the achievable performances compared with the state-of-the-art, and pointing out and quantitatively highlighting the acquisition and processing strategies which would allow to maximize the quality of the results. Moreover, a critical analysis is performed about the main errors affecting the applied techniques, as well as the limitations of the orbital configurations, identifying several complementary techniques which would allow to overcome or mitigate the observed drawbacks. An innovative procedure for on-demand DEM production from CSK SAR data is elaborated and proposed, as well as an auto-validation technique which would enable the validation of the produced DEM also where vertical ground truths are not available. Based on the obtained results and on the consequent critical analysis, several interferometric specifications for new generation SAR satellites are identified. Finally, a literature review is proposed about the main state-of-the-art ship monitoring techniques, considered as one of the main fields of application which takes benefit from SAR data, based on single/multi-platform multi-channel SAR data, with a focus on TanDEM-X (TDX). In particular, in Chapter 1 the main concepts concerning SAR operating principles are introduced and the main characteristics and performances of CSK and TDX satellite systems are described; in Chapter 2 a review is proposed about the state-of-the-art SAR interferometric techniques for DEM generation, analyzing all the relevant processing steps and deepening the study of the main solutions recently proposed in the literature to increase the accuracy of the interferometric processing; in Chapter 3 complementary and innovative techniques respect to the interferometric processing are analyzed to mitigate disadvantages and to improve performances; in Chapter 4 experimental results are presented, obtained in the generation of high accuracy DEM by applying to a dataset of CSK images properly selected state-of-the-art interferometric techniques and innovative methods to improve DEM accuracy, exploring relevant limitations, and pointing out innovative acquisition and processing strategies. In Chapter 5, the basic principles of Ground Moving Target Indication (GMTI) are described, focusing on Displaced Phase Center Antenna (DPCA) and Along-Track Interferometry (ATI) techniques

Aeronautical engineering: A continuing bibliography with indexes (supplement 254)

This bibliography lists 538 reports, articles, and other documents introduced into the NASA scientific and technical information system in June 1990. Subject coverage includes: design, construction and testing of aircraft and aircraft engines; aircraft components, equipment and systems; ground support systems; and theoretical and applied aspects of aerodynamics and general fluid dynamics