Underwater 3D Reconstruction Based on Physical Models for Refraction and Underwater Light Propagation

In recent years, underwater imaging has gained a lot of popularity partly due to the availability of off-the-shelf consumer cameras, but also due to a growing interest in the ocean floor by science and industry. Apart from capturing single images or sequences, the application of methods from the area of computer vision has gained interest as well. However, water affects image formation in two major ways. First, while traveling through the water, light is attenuated and scattered, depending on the light's wavelength causing the typical strong green or blue hue in underwater images. Second, cameras used in underwater scenarios need to be confined in an underwater housing, viewing the scene through a flat or dome-shaped glass port. The inside of the housing is filled with air. Consequently, the light entering the housing needs to pass a water-glass interface, then a glass-air interface, thus is refracted twice, affecting underwater image formation geometrically. In classic Structure-from-Motion (SfM) approaches, the perspective camera model is usually assumed, however, it can be shown that it becomes invalid due to refraction in underwater scenarios. Therefore, this thesis proposes an adaptation of the SfM algorithm to underwater image formation with flat port underwater housings, i.e. introduces a method where refraction at the underwater housing is modeled explicitly. This includes a calibration approach, algorithms for relative and absolute pose estimation, an efficient, non-linear error function that is utilized in bundle adjustment, and a refractive plane sweep algorithm. Finally, if calibration data for an underwater light propagation model exists, the dense depth maps can be used to correct texture colors. Experiments with a perspective and the proposed refractive approach to 3D reconstruction revealed that the perspective approach does indeed suffer from a systematic model error depending on the distance between camera and glass and a possible tilt of the glass with respect to the image sensor. The proposed method shows no such systematic error and thus provides more accurate results for underwater image sequences

The Bubble Box: Towards an Automated Visual Sensor for 3D Analysis and Characterization of Marine Gas Release Sites

Several acoustic and optical techniques have been used for characterizing natural and anthropogenic gas leaks (carbon dioxide, methane) from the ocean floor. Here, single-camera based methods for bubble stream observation have become an important tool, as they help estimating flux and bubble sizes under certain assumptions. However, they record only a projection of a bubble into the camera and therefore cannot capture the full 3D shape, which is particularly important for larger, non-spherical bubbles. The unknown distance of the bubble to the camera (making it appear larger or smaller than expected) as well as refraction at the camera interface introduce extra uncertainties. In this article, we introduce our wide baseline stereo-camera deep-sea sensor bubble box that overcomes these limitations, as it observes bubbles from two orthogonal directions using calibrated cameras. Besides the setup and the hardware of the system, we discuss appropriate calibration and the different automated processing steps deblurring, detection, tracking, and 3D fitting that are crucial to arrive at a 3D ellipsoidal shape and rise speed of each bubble. The obtained values for single bubbles can be aggregated into statistical bubble size distributions or fluxes for extrapolation based on diffusion and dissolution models and large scale acoustic surveys. We demonstrate and evaluate the wide baseline stereo measurement model using a controlled test setup with ground truth information

Refractive 3D reconstruction on underwater images

Highlights • Complete 3D reconstruction system from images and videos. • Refractive Structure from Motion for flat port underwater cameras. • Eliminates systematic modeling error caused by using perspective camera model. Cameras can be considered measurement devices complementary to acoustic sensors when it comes to surveying marine environments. When calibrated and used correctly, these visual sensors are well-suited for automated detection, quantification, mapping, and monitoring applications and when aiming at high-accuracy 3D models or change detection. In underwater scenarios, cameras are often set up in pressure housings with a flat glass window, a flat port, which allows them to observe the environment. In this contribution, a geometric model for image formation is discussed that explicitly considers refraction at the interface under realistic assumptions like a slightly misaligned camera (w.r.t. the glass normal) and thick glass ports as common for deep sea applications. Then, starting from camera calibration, a complete, fully automated 3D reconstruction system is discussed that takes an image sequence and produces a 3D model. Newly derived refractive estimators for sparse two-view geometry, pose estimation, bundle adjustment, and dense depth estimation are discussed and evaluated in detail

Refractive Structure-from-Motion on Underwater Images

In underwater environments, cameras need to be confined in an underwater housing, viewing the scene through a piece of glass. In case of flat port underwater housings, light rays entering the camera housing are refracted twice, due to different medium densities of water, glass, and air. This causes the usually linear rays of light to bend and the commonly used pinhole camera model to be invalid. When using the pinhole camera model without explicitly modeling refraction in Structure-from-Motion (SfM) methods, a systematic model error occurs. Therefore, in this paper, we propose a system for computing camera path and 3D points with explicit incorporation of refraction using new methods for pose estimation. Additionally, a new error function is introduced for non-linear optimization, especially bundle adjustment. The proposed method allows to increase reconstruction accuracy and is evaluated in a set of experiments, where the proposed method’s performance is compared to SfM with the perspective camera model. 1

Refractive Plane Sweep for Underwater Images

Abstract. In underwater imaging, refraction changes the geometry of image formation, causing the perspective camera model to be invalid. Hence, a systematic model error occurs when computing 3D models using the perspective camera model. This paper deals with the problem of computing dense depth maps of underwater scenes with explicit incorporation of refraction of light at the underwater housing. It is assumed that extrinsic, intrinsic, and housing parameters have been calibrated for all cameras. Due to the refractive camera’s characteristics it is not possible to directly apply epipolar geometry or rectification to images because the single-view-point model and, consequently, homographies are invalid. Additionally, the projection of 3D points into the camera cannot be computed efficiently, but requires solving a 12 th degree polynomial. Therefore, the method proposed is an adapted plane sweep algorithm that is based on the idea of back-projecting rays for each pixel and view onto the 3D-hypothesis planes using the GPU. This allows to efficiently warp all images onto the plane, where they can be compared. Consequently, projections of 3D points and homographies are not utilized.

Perspectives in visual imaging for marine biology and ecology: from acquisition to understanding

http://www.godac.jamstec.go.jp/darwin/cruise/kairei/kr12-07/