Investigation of Sea Ice Using Multiple Synthetic Aperture Radar Acquisitions

The papers of this thesis are not available in Munin. Paper I: Yitayew, T. G., Ferro-Famil, L., Eltoft, T. & Tebaldini, S. (2017). Tomographic imaging of fjord ice using a very high resolution ground-based SAR system. Available in IEEE Transactions on Geoscience and Remote Sensing, 55 (2):698-714. Paper II: Yitayew, T. G., Ferro-Famil, L., Eltoft, T. & Tebaldini, S. (2017). Lake and fjord ice imaging using a multifrequency ground-based tomographic SAR system. Available in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(10):4457-4468. Paper III: Yitayew, T. G., Divine, D. V., Dierking, W., Eltoft, T., Ferro-Famil, L., Rosel, A. & Negrel, J. Validation of Sea ice Topographic Heights Derived from TanDEMX Interferometric SAR Data with Results from Laser Profiler and Photogrammetry. (Manuscript).The thesis investigates imaging in the vertical direction of different types of ice in the arctic using synthetic aperture radar (SAR) tomography and SAR interferometry. In the first part, the magnitude and the positions of the dominant scattering contributions within snow covered fjord and lake ice layers are effectively identified by using a very high resolution ground-based tomographic SAR system. Datasets collected at multiple frequencies and polarizations over two test sites in Tromsø area, northern Norway, are used for characterizing the three-dimensional response of snow and ice. The presented experimental results helped to improve our understanding of the interaction between radar waves and snow and ice layers. The reconstructed radar responses are also used for estimating the refractive indices and the vertical positions of the different sub-layers of snow and ice. The second part of the thesis deals with the retrieval of the surface topography of multi-year sea ice using SAR interferometry. Satellite acquisitions from TanDEM-X over the Svalbard area are used for analysis. The retrieved surface height is validated by using overlapping helicopter-based stereo camera and laser profiler measurements, and a very good agreement has been found. The work contributes to an improved understanding regarding the potential of SAR tomography for imaging the vertical scattering distribution of snow and ice layers, and for studying the influence of both sensor parameters such as its frequency and polarization and scene properties such as layer stratification, air bubbles and small-scale roughness of the interfaces on snow and ice backscattered signal. Moreover, the presented results reveal the potential of SAR interferometry for retrieving the surface topography of sea ice

3D display size matters: Compensating for the perceptual effects of S3D display scaling

Abstract Introduction In recent years the consumer electronics market has been flooded with a variety of S3D products, which rely on a variety of display and image segregation technologies. For each display system, the ideal viewing conditions (eg. viewing angle) can be defined in order to obtain the desired 3D experience. SMPTE and THX [1, 2] have provided specific standards and guidelines for the ideal viewing angle for theatre and television. However, screen dimension 1 is an uncontrolled variable since the same content could be displayed on a mobile autostereoscopic device, 3D monitor, HD 3DTV or in a 3D movie theatre. Adapting a S3D film to a variety of screen sizes is necessary for most, if not all, popular movies if the distributors are to maximize their exposure and therefore earnings. However, unlike 2D film the S3D scaling process is complicated by a variety of 1 The range of viewing distances typically used are correlated with the size of the display, with audiences moving closer as screens get smaller. If field of view is constant it is often the distance that is more important. Since they normally co-vary here we will focus on screen size and related disparity scaling issues, but will point out the role of viewing distance in particular when it is warranted. computational and perceptual issues that can significantly impact the audience experience. As outlined below, the existing approaches to scaling S3D content for a variety of delivery form factors can be divided into two main categories: those applied during acquisition and those applied during postproduction or display. The most common strategy is some combination of pre and post-production approaches. However, inevitably some degree of perceptual and geometric distortion will remain. A better understanding of these distortions and their perceptual consequences will provide S3D content creators with insight and context for using sophisticated scaling approaches based on both acquisition and post-production techniques. This paper will review the principal issues related to S3D content scaling, some of the technical solutions available to content makers/ providers and the perceptual consequences for audiences. Stereoscopic Geometry As was shown by Spottiswood in the early 1950's [3], displaying stereoscopic 3D content at different sizes may dramatically influence the audience's S3D experience. Given the interdependence of acquisition and display parameters; most filmmakers, while trying to protect for different screen dimensions will have a target viewing condition when they begin filming. Figures 1 and 2 depict stereoscopic viewing and acquisition geometry, respectivel

NOVEL DENSE STEREO ALGORITHMS FOR HIGH-QUALITY DEPTH ESTIMATION FROM IMAGES

This dissertation addresses the problem of inferring scene depth information from a collection of calibrated images taken from different viewpoints via stereo matching. Although it has been heavily investigated for decades, depth from stereo remains a long-standing challenge and popular research topic for several reasons. First of all, in order to be of practical use for many real-time applications such as autonomous driving, accurate depth estimation in real-time is of great importance and one of the core challenges in stereo. Second, for applications such as 3D reconstruction and view synthesis, high-quality depth estimation is crucial to achieve photo realistic results. However, due to the matching ambiguities, accurate dense depth estimates are difficult to achieve. Last but not least, most stereo algorithms rely on identification of corresponding points among images and only work effectively when scenes are Lambertian. For non-Lambertian surfaces, the brightness constancy assumption is no longer valid. This dissertation contributes three novel stereo algorithms that are motivated by the specific requirements and limitations imposed by different applications. In addressing high speed depth estimation from images, we present a stereo algorithm that achieves high quality results while maintaining real-time performance. We introduce an adaptive aggregation step in a dynamic-programming framework. Matching costs are aggregated in the vertical direction using a computationally expensive weighting scheme based on color and distance proximity. We utilize the vector processing capability and parallelism in commodity graphics hardware to speed up this process over two orders of magnitude. In addressing high accuracy depth estimation, we present a stereo model that makes use of constraints from points with known depths - the Ground Control Points (GCPs) as referred to in stereo literature. Our formulation explicitly models the influences of GCPs in a Markov Random Field. A novel regularization prior is naturally integrated into a global inference framework in a principled way using the Bayes rule. Our probabilistic framework allows GCPs to be obtained from various modalities and provides a natural way to integrate information from various sensors. In addressing non-Lambertian reflectance, we introduce a new invariant for stereo correspondence which allows completely arbitrary scene reflectance (bidirectional reflectance distribution functions - BRDFs). This invariant can be used to formulate a rank constraint on stereo matching when the scene is observed by several lighting configurations in which only the lighting intensity varies

Self-supervised monocular depth estimation from oblique UAV videos

UAVs have become an essential photogrammetric measurement as they are affordable, easily accessible and versatile. Aerial images captured from UAVs have applications in small and large scale texture mapping, 3D modelling, object detection tasks, DTM and DSM generation etc. Photogrammetric techniques are routinely used for 3D reconstruction from UAV images where multiple images of the same scene are acquired. Developments in computer vision and deep learning techniques have made Single Image Depth Estimation (SIDE) a field of intense research. Using SIDE techniques on UAV images can overcome the need for multiple images for 3D reconstruction. This paper aims to estimate depth from a single UAV aerial image using deep learning. We follow a self-supervised learning approach, Self-Supervised Monocular Depth Estimation (SMDE), which does not need ground truth depth or any extra information other than images for learning to estimate depth. Monocular video frames are used for training the deep learning model which learns depth and pose information jointly through two different networks, one each for depth and pose. The predicted depth and pose are used to reconstruct one image from the viewpoint of another image utilising the temporal information from videos. We propose a novel architecture with two 2D CNN encoders and a 3D CNN decoder for extracting information from consecutive temporal frames. A contrastive loss term is introduced for improving the quality of image generation. Our experiments are carried out on the public UAVid video dataset. The experimental results demonstrate that our model outperforms the state-of-the-art methods in estimating the depths.Comment: Submitted to ISPRS Journal of Photogrammetry and Remote Sensin

Projection-based Registration Using a Multi-view Camera for

Abstrac

3D Reconstruction using Active Illumination

In this thesis we present a pipeline for 3D model acquisition. Generating 3D models of real-world objects is an important task in computer vision with many applications, such as in 3D design, archaeology, entertainment, and virtual or augmented reality. The contribution of this thesis is threefold: we propose a calibration procedure for the cameras, we describe an approach for capturing and processing photometric normals using gradient illuminations in the hardware set-up, and finally we present a multi-view photometric stereo 3D reconstruction method. In order to obtain accurate results using multi-view and photometric stereo reconstruction, the cameras are calibrated geometrically and photometrically. For acquiring data, a light stage is used. This is a hardware set-up that allows to control the illumination during acquisition. The procedure used to generate appropriate illuminations and to process the acquired data to obtain accurate photometric normals is described. The core of the pipeline is a multi-view photometric stereo reconstruction method. In this method, we first generate a sparse reconstruction using the acquired images and computed normals. In the second step, the information from the normal maps is used to obtain a dense reconstruction of an object’s surface. Finally, the reconstructed surface is filtered to remove artifacts introduced by the dense reconstruction step

A data-fusion approach to motion-stereo

This paper introduces a novel method for performing motion--stereo, based on dynamic integration of depth (or its proxy) measures obtained by pairwise stereo matching of video frames. The focus is on the data fusion issue raised by the motion--stereo approach, which is solved within a Kalman filtering framework. Integration occurs along the temporal and spatial dimension, so that the final measure for a pixel results from the combination of measures of the same pixel in time and whose of its neighbors. The method has been validated on both synthetic and natural images, using the simplest stereo matching strategy and a range of different confidence measures, and has been compared to baseline and optimal strategies

Automatically Recovering Geometry and Texture from Large Sets of Calibrated Images

Three-dimensional models which contain both geometry and texture have numerous applications such as urban planning, physical simulation, and virtual environments. A major focus of computer vision (and recently graphics) research is the automatic recovery of three-dimensional models from two-dimensional images. After many years of research this goal is yet to be achieved. Most practical modeling systems require substantial human input and unlike automatic systems are not scalable. This thesis presents a novel method for automatically recovering dense surface patches using large sets (1000's) of calibrated images taken from arbitrary positions within the scene. Physical instruments, such as Global Positioning System (GPS), inertial sensors, and inclinometers, are used to estimate the position and orientation of each image. Essentially, the problem is to find corresponding points in each of the images. Once a correspondence has been established, calculating its three-dimensional position is simply a matter of geometry. Long baseline images improve the accuracy. Short baseline images and the large number of images greatly simplifies the correspondence problem. The initial stage of the algorithm is completely local and scales linearly with the number of images. Subsequent stages are global in nature, exploit geometric constraints, and scale quadratically with the complexity of the underlying scene. We describe techniques for: 1) detecting and localizing surface patches; 2) refining camera calibration estimates and rejecting false positive surfels; and 3) grouping surface patches into surfaces and growing the surface along a two-dimensional manifold. We also discuss a method for producing high quality, textured three-dimensional models from these surfaces. Some of the most important characteristics of this approach are that it: 1) uses and refines noisy calibration estimates; 2) compensates for large variations in illumination; 3) tolerates significant soft occlusion (e.g. tree branches); and 4) associates, at a fundamental level, an estimated normal (i.e. no frontal-planar assumption) and texture with each surface patch