Advanced perception, navigation and planning for autonomous in-water ship hull inspection

Inspection of ship hulls and marine structures using autonomous underwater vehicles has emerged as a unique and challenging application of robotics. The problem poses rich questions in physical design and operation, perception and navigation, and planning, driven by difficulties arising from the acoustic environment, poor water quality and the highly complex structures to be inspected. In this paper, we develop and apply algorithms for the central navigation and planning problems on ship hulls. These divide into two classes, suitable for the open, forward parts of a typical monohull, and for the complex areas around the shafting, propellers and rudders. On the open hull, we have integrated acoustic and visual mapping processes to achieve closed-loop control relative to features such as weld-lines and biofouling. In the complex area, we implemented new large-scale planning routines so as to achieve full imaging coverage of all the structures, at a high resolution. We demonstrate our approaches in recent operations on naval ships.United States. Office of Naval Research (Grant N00014-06-10043)United States. Office of Naval Research (Grant N00014-07-1-0791

Visibility in underwater robotics: Benchmarking and single image dehazing

Dealing with underwater visibility is one of the most important challenges in autonomous underwater robotics. The light transmission in the water medium degrades images making the interpretation of the scene difficult and consequently compromising the whole intervention. This thesis contributes by analysing the impact of the underwater image degradation in commonly used vision algorithms through benchmarking. An online framework for underwater research that makes possible to analyse results under different conditions is presented. Finally, motivated by the results of experimentation with the developed framework, a deep learning solution is proposed capable of dehazing a degraded image in real time restoring the original colors of the image.Una de las dificultades más grandes de la robótica autónoma submarina es lidiar con la falta de visibilidad en imágenes submarinas. La transmisión de la luz en el agua degrada las imágenes dificultando el reconocimiento de objetos y en consecuencia la intervención. Ésta tesis se centra en el análisis del impacto de la degradación de las imágenes submarinas en algoritmos de visión a través de benchmarking, desarrollando un entorno de trabajo en la nube que permite analizar los resultados bajo diferentes condiciones. Teniendo en cuenta los resultados obtenidos con este entorno, se proponen métodos basados en técnicas de aprendizaje profundo para mitigar el impacto de la degradación de las imágenes en tiempo real introduciendo un paso previo que permita recuperar los colores originales

Low-Cost Vision Based Autonomous Underwater Vehicle for Abyssal Ocean Ecosystem Research

The oceans have a major impact on the planet: they store 28% of the CO 2 pro- duced by humans, they act as the world’s thermal damper for temperature changes, and more than 17, 000 species call the deep oceans their home. Scientific drivers, like climate change, and commercial applications, like deep sea fisheries and underwater mining, are pushing the need to know more about oceans at depths beyond 1000 meters. However, the high cost associated with autonomous underwater vehicles (AUVs) capable of operating beyond the depth of 1000 meters has limited the study of the deep ocean. Traditional AUVs used for deep-sea navigation are large and typically weigh up- wards of 1000-kgs, thus requiring careful planning before deployment and multi- person teams to operate. This thesis proposes the use of a new vehicle design based around a low-cost oceanographic glass sphere as the main pressure enclosure to reduce its size and cost while maintaining the ability for deep-sea operation. This novel housing concept, together with a minimal sensor suite, enables environmental research at depths previously inaccessible at this price point. The key characteristic that enables the cost reduction of this platform is the removal of the Doppler velocity log (DVL) sensor, which is replaced by optical cameras. Cameras allow the vehicle to estimate its motion in the water, but also enable scientific applications such as identification of habitat types or population density estimation of benthic species. After each survey, images can be further processed to produce full, dense 3D models of the survey area. While underwater optical cameras are frequently placed inside pressure housings behind flat or domed viewports and used for visual navigation or 3D reconstructions, the underlying assumptions for those algorithms do not hold in the underwater domain. Refraction at the housing viewport, together with wavelength-dependent attenuation of light in water, render the ubiquitous pinhole camera model invalid. This thesis presents a quantitative evaluation of the errors introduced by underwater effects for 3D reconstruction applications, comparing low- and high-cost camera systems to quantify the trade-off between equipment cost and performance. Although the distortion effects created by underwater refraction of light have been extensively studied for more traditional viewports, the novel design proposed necessitates new research into modeling the lensing effect of this off-axis domed viewport. A novel calibration method is presented that explicitly models the effect of the glass interface on image formation based on the characterization of optical distortions. The method is capable of accurately finding the position of the camera within the dome and further enables the use of deconvolution to improve the quality of the taken image. Finally, this thesis presents the validation of the designed vehicle for optical surveying tasks and introduces a end-to-end ocean mapping pipeline to streamline AUV deployments, enabling efficient use of time and resources.PHDNaval Architecture & Marine EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/155225/1/eiscar_1.pd

3D reconstruction and object recognition from 2D SONAR data

Accurate and meaningful representations of the environment are required for autonomy in underwater applications. Thanks to favourable propagation properties in water, acoustic sensors are commonly preferred to video cameras and lasers but do not provide direct 3D information. This thesis addresses the 3D reconstruction of underwater scenes from 2D imaging SONAR data as well as the recognition of objects of interest in the reconstructed scene. We present two 3D reconstruction methods and two model-based object recognition methods. We evaluate our algorithms on multiple scenarios including data gathered by an AUV. We show the ability to reconstruct underwater environments at centimetre-level accuracy using 2D SONARs of any aperture. We demonstrate the recognition of structures of interest on a medium-sized oil-field type environment providing accurate yet low memory footprint semantic world models. We conclude that accurate 3D semantic representations of partially-structured marine environments can be obtained from commonly embedded 2D SONARs, enabling online world modelling, relocalisation and model-based applications

Multisensor Microbathymetric Habitat Mapping with a Deep-Towed Ocean Floor Observation and Bathymetry System (OFOBS)

To describe the seafloor topography, a number of different bathymetric methods can be applied. These methods vary greatly in coverage, resolution, and topographic uncertainty. Satellite-based gravimetry and radar altimetry can give large-scale structural estimates of the seafloor topography, yet, with a very low resolution and without real depth measurements. Ship-based swath bathymetry systems greatly improve the topographic uncertainty and increase the knowledge on geomorphology and depth of the seafloor. In shallow waters, ship mounted echosounders can produce high-resolution data on a submeter level. However, in deep-sea environments, the resolution deteriorates due to large acoustic footprints and a reduced number of measurement points with respect to the mapped area. In order to conduct high-resolution habitat mapping and to resolve small-scale topographic seafloor features, subsea survey vehicles need to be employed. Next to remotely operated or autonomous underwater vehicles, towed camera systems present a comparatively cheap method, both financially and with regards to support requirements, to collect close-range optical seafloor data. Nonetheless, optical sensors have very limited coverage capabilities in the deep sea, due to the nature of the sensors and the high attenuation of light in the water column. Acoustic sensors on the other hand can achieve much wider survey swaths, depending on their operation frequency. The Ocean Floor Observation and Bathymetry System (OFOBS), developed at the Alfred Wegener Institute for Polar and Marine Research, Germany, offers a novel survey technology for deep-towed multisensor microbathymetric habitat mapping. To augment the traditional optical sensors, the OFOBS was equipped with additional acoustic and navigational sensors. A bathymetric side scan sonar collects lateral seafloor reflection intensity and bathymetry at ranges up to 100 m to both sides of the vehicle. A forward looking sonar records acoustic imagery ahead of the system, which can be used for hazardous obstacle avoidance in rough terrain. This thesis introduces the newly developed system along with processing workflows for the acquired datasets. Underwater photogrammetric methods are utilized for the optical data, to reconstruct the three dimensional morphology of the seabed. The camera pose estimations of the employed bundle adjustment algorithms are used for local navigation corrections of the acoustic datasets, to achieve best possible data alignment. The resulting multilayer product consists of wide-swath acoustic bathymetry (submeter resolution), multi-frequency side scan mosaics (subdecimeter resolution), photogrammetric microbathymetry (subcentimeter resolution), and geometrically corrected, georeferenced photo mosaics (submillimeter resolution). These results offer a wide variety of use cases in high-resolution habitat analyses by the associated scientific working groups. The data used for developing the presented workflow was collected during the RV Polarstern expedition PS101 in the extreme environment of the volcanic seamounts along the Langseth Ridge in the high Arctic (87°N, 60°E)

Hyperspectral benthic mapping from underwater robotic platforms

We live on a planet of vast oceans; 70% of the Earth's surface is covered in water. They are integral to supporting life, providing 99% of the inhabitable space on Earth. Our oceans and the habitats within them are under threat due to a variety of factors. To understand the impacts and possible solutions, the monitoring of marine habitats is critically important. Optical imaging as a method for monitoring can provide a vast array of information however imaging through water is complex. To compensate for the selective attenuation of light in water, this thesis presents a novel light propagation model and illustrates how it can improve optical imaging performance. An in-situ hyperspectral system is designed which comprised of two upward looking spectrometers at different positions in the water column. The downwelling light in the water column is continuously sampled by the system which allows for the generation of a dynamic water model. In addition to the two upward looking spectrometers the in-situ system contains an imaging module which can be used for imaging of the seafloor. It consists of a hyperspectral sensor and a trichromatic stereo camera. New calibration methods are presented for the spatial and spectral co-registration of the two optical sensors. The water model is used to create image data which is invariant to the changing optical properties of the water and changing environmental conditions. In this thesis the in-situ optical system is mounted onboard an Autonomous Underwater Vehicle. Data from the imaging module is also used to classify seafloor materials. The classified seafloor patches are integrated into a high resolution 3D benthic map of the surveyed site. Given the limited imaging resolution of the hyperspectral sensor used in this work, a new method is also presented that uses information from the co-registered colour images to inform a new spectral unmixing method to resolve subpixel materials

Examination of the spatial resolution and discrimination capability of various acoustic seafloor classification techniques based on MBES backscatter data

This thesis focuses on two major topics regarding acoustic seafloor classification techniques. The first topic is about acoustic class separation which affects the discriminative power of classification techniques and the quality of final results. The second topic is the spatial resolution of seafloor acoustic maps that is fundamentally coupled with acoustic class separation. The approach followed here, a) employs an advanced unsupervised classification technique and b) analyzes its implications on the angular response analysis (ARA) of acoustic backscatter. Moreover, a novel approach for improving the ARA technique is described. Applying an unsupervised Bayesian technique that performs an internal cluster validation test, we obtain objective classification of the entire backscatter dataset. This technique utilizes single-angle backscatter measurements from the middle range of the sonar swath offering better discrimination of acoustic classes. The main advantages of the Bayesian technique are that it does not require sonar calibration, it resolves along-swath seafloor variations and that it outputs ordinal categorical values for acoustic classes. Furthermore, the concept of the Hyper-Angular Cube (HAC) is applied and its results are compared with the Bayesian classification results. The HAC is built by several angular backscatter layers which can result either by interpolation of dense soundings or by normalization of backscatter mosaics at different incidence angles. The high dimensional data of the HAC is suitable for supervised classification using machine learning techniques and restricted amount of ground truth information. This approach takes angular dependence of backscatter into consideration and utilizes hydro-acoustic and ground truth data in a more efficient way than it was possible until now

Underwater Vehicles

For the latest twenty to thirty years, a significant number of AUVs has been created for the solving of wide spectrum of scientific and applied tasks of ocean development and research. For the short time period the AUVs have shown the efficiency at performance of complex search and inspection works and opened a number of new important applications. Initially the information about AUVs had mainly review-advertising character but now more attention is paid to practical achievements, problems and systems technologies. AUVs are losing their prototype status and have become a fully operational, reliable and effective tool and modern multi-purpose AUVs represent the new class of underwater robotic objects with inherent tasks and practical applications, particular features of technology, systems structure and functional properties