Digital Cognitive Companions for Marine Vessels : On the Path Towards Autonomous Ships

As for the automotive industry, industry and academia are making extensive efforts to create autonomous ships. The solutions for this are very technology-intense. Many building blocks, often relying on AI technology, need to work together to create a complete system that is safe and reliable to use. Even when the ships are fully unmanned, humans are still foreseen to guide the ships when unknown situations arise. This will be done through teleoperation systems.In this thesis, methods are presented to enhance the capability of two building blocks that are important for autonomous ships; a positioning system, and a system for teleoperation.The positioning system has been constructed to not rely on the Global Positioning System (GPS), as this system can be jammed or spoofed. Instead, it uses Bayesian calculations to compare the bottom depth and magnetic field measurements with known sea charts and magnetic field maps, in order to estimate the position. State-of-the-art techniques for this method typically use high-resolution maps. The problem is that there are hardly any high-resolution terrain maps available in the world. Hence we present a method using standard sea-charts. We compensate for the lower accuracy by using other domains, such as magnetic field intensity and bearings to landmarks. Using data from a field trial, we showed that the fusion method using multiple domains was more robust than using only one domain. In the second building block, we first investigated how 3D and VR approaches could support the remote operation of unmanned ships with a data connection with low throughput, by comparing respective graphical user interfaces (GUI) with a Baseline GUI following the currently applied interfaces in such contexts. Our findings show that both the 3D and VR approaches outperform the traditional approach significantly. We found the 3D GUI and VR GUI users to be better at reacting to potentially dangerous situations than the Baseline GUI users, and they could keep track of the surroundings more accurately. Building from this, we conducted a teleoperation user study using real-world data from a field-trial in the archipelago, where the users should assist the positioning system with bearings to landmarks. The users experienced the tool to give a good overview, and despite the connection with the low throughput, they managed through the GUI to significantly improve the positioning accuracy

A parallel hypothesis method of autonomous underwater vehicle navigation

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the Massachusetts Institute of Technology and the Woods Hole Oceanographic Institution June 2009This research presents a parallel hypothesis method for autonomous underwater vehicle navigation that enables a vehicle to expand the operating envelope of existing long baseline acoustic navigation systems by incorporating information that is not normally used. The parallel hypothesis method allows the in-situ identification of acoustic multipath time-of-flight measurements between a vehicle and an external transponder and uses them in real-time to augment the navigation algorithm during periods when direct-path time-of-flight measurements are not available. A proof of concept was conducted using real-world data obtained by the Woods Hole Oceanographic Institution Deep Submergence Lab's Autonomous Benthic Explorer (ABE) and Sentry autonomous underwater vehicles during operations on the Juan de Fuca Ridge. This algorithm uses a nested architecture to break the navigation solution down into basic building blocks for each type of available external information. The algorithm classifies external information as either line of position or gridded observations. For any line of position observation, the algorithm generates a multi-modal block of parallel position estimate hypotheses. The multimodal hypotheses are input into an arbiter which produces a single unimodal output. If a priori maps of gridded information are available, they are used within the arbiter structure to aid in the elimination of false hypotheses. For the proof of concept, this research uses ranges from a single external acoustic transponder in the hypothesis generation process and grids of low-resolution bathymetric data from a ship-based multibeam sonar in the arbitration process. The major contributions of this research include the in-situ identification of acoustic multipath time-of-flight measurements, the multiscale utilization of a priori low-resolution bathymetric data in a high-resolution navigation algorithm, and the design of a navigation algorithm with a exible architecture. This flexible architecture allows the incorporation of multimodal beliefs without requiring a complex mechanism for real-time hypothesis generation and culling, and it allows the real-time incorporation of multiple types of external information as they become available in situ into the overall navigation solution

Self consistent bathymetric mapping from robotic vehicles in the deep ocean

Submitted In partial fulfillment of the requirements for the degree of Doctor of Philosophy at the Massachusetts Institute of Technology and Woods Hole Oceanographic Institution June 2005Obtaining accurate and repeatable navigation for robotic vehicles in the deep ocean is difficult and consequently a limiting factor when constructing vehicle-based bathymetric maps. This thesis presents a methodology to produce self-consistent maps and simultaneously improve vehicle position estimation by exploiting accurate local navigation and utilizing terrain relative measurements. It is common for errors in the vehicle position estimate to far exceed the errors associated with the acoustic range sensor. This disparity creates inconsistency when an area is imaged multiple times and causes artifacts that distort map integrity. Our technique utilizes small terrain "submaps" that can be pairwise registered and used to additionally constrain the vehicle position estimates in accordance with actual bottom topography. A delayed state Kalman filter is used to incorporate these sub-map registrations as relative position measurements between previously visited vehicle locations. The archiving of previous positions in a filter state vector allows for continual adjustment of the sub-map locations. The terrain registration is accomplished using a two dimensional correlation and a six degree of freedom point cloud alignment method tailored for bathymetric data. The complete bathymetric map is then created from the union of all sub-maps that have been aligned in a consistent manner. Experimental results from the fully automated processing of a multibeam survey over the TAG hydrothermal structure at the Mid-Atlantic ridge are presented to validate the proposed method.This work was funded by the CenSSIS ERC of the Nation Science Foundation under grant EEC-9986821 and in part by the Woods Hole Oceanographic Institution through a grant from the Penzance Foundation

Benthic Habitat Analyses Using Micro-bathymetry Data and Subsea Photogrammetry

The very first map of the Arctic Ocean basin with a few lead line sounding changed the supposal of large continental land beneath of the ice. More resolution added over the decades, reveals the detail of the Arctic seafloor structure of seamounts and ridges below the frozen sea. Numerous methods of bathymetry and mapping were applied as the technology developed over the years for different purposes. While the airborne and satellite-based altimetry and gravimetry data provides a large-scale estimation of the seafloor topography by hundreds of meters resolution, the shipborne and submarine sonars focuses on certain features and areas with higher resolution. During the last century the knowledge of the Arctic seabed geomorphology increased dramatically by the development of acoustic technology combined with altimetry and gravimetry while the habitat characteristic of the polar region still contains lots of mysteries. The new developments of underwater survey vehicles are bringing new clarity and perspective from the deep sea to the questioners. The sub-meter resolution data of the seabed could be employed for very high-resolution micro topography as well as habitat mapping and feature detection. The Alfred Wegner Institute for Polar and Marine Research (AWI) developed the Ocean Floor Observation and Bathymetry System (OFOBS) for deep sea research, mostly in polar region. The tailored deep tow system of the AWI is equipped with optical and acoustic sensors in addition to underwater positioning systems. The OFOBS, first deployed during the PS101 expedition, provides a novel dataset of megafauna’s habitats at the Karasik seamount. This thesis is implementing geospatial data mining and knowledge discovery for feature detection by means of habitat mapping in the study area with a focus on the central mount of Karasik seamount where an imperial assemblage of the Geodia sponges are dominating the seafloor. The main datasets for this study are based on the optical sensor of the OFOBS, including video and still images collected during the dives, while the feature detection within the sonar dataset is in the second place. During this work study, the development of the OFOBS is also considered in order to improve the capability of the dataset for further expeditions

Underwater Vehicles

For the latest twenty to thirty years, a significant number of AUVs has been created for the solving of wide spectrum of scientific and applied tasks of ocean development and research. For the short time period the AUVs have shown the efficiency at performance of complex search and inspection works and opened a number of new important applications. Initially the information about AUVs had mainly review-advertising character but now more attention is paid to practical achievements, problems and systems technologies. AUVs are losing their prototype status and have become a fully operational, reliable and effective tool and modern multi-purpose AUVs represent the new class of underwater robotic objects with inherent tasks and practical applications, particular features of technology, systems structure and functional properties

Geomagnetic gradient-assisted evolutionary algorithm for long-range underwater navigation

Extensive research results have shown that animals like pigeons and turtles can use geomagnetic information for long-distance migration and homing. This article studies the bionic navigation method inspired by magnetotaxis behavior without prior knowledge. The problem of bionic geomagnetic navigation is generalized as an autonomous search of navigation path under the excitation of geomagnetic environment. The geomagnetic gradient-assisted evolutionary algorithm for long-range underwater navigation is proposed. In order to optimize the navigation path, the heading angle predicted by the geomagnetic gradient is used to constrain the sample space in the evolutionary algorithm. Then, according to the principle of multiparameter simultaneous convergence, the evaluation function is improved to enhance the reliability and accuracy of the navigation path. Simulations of the algorithm before and after improvement are carried out based on the data retrieved from the enhanced magnetic model (EMM). The performance of the improved method is evaluated and verified in the case of the area with normal geomagnetic field (GF), geomagnetic anomaly area, and multiple destinations. The simulation results show that the search efficiency and the straightness of the navigation path are greatly improved. The reason is that the constraint of sample space reduces the randomness in the process of navigation path search, and the improved evaluation function can evaluate the quality of samples more accurately. The improved algorithm also has good performance in the geomagnetic anomaly area, which indicates the potential application in the future

Virtual aids to navigation

There are many examples of master, bridge crew and pilot errors in navigation causing grounding under adverse circumstances that were known and published in official notices and records. Also dangerous are hazards to navigation resulting from dynamic changes within the marine environment, inadequate surveys and charts. This research attempts to reduce grounding and allision incidents and increase safety of navigation by expanding mariner situational awareness at and below the waterline using new technology and developing methods for the creation, implementation and display of Virtual Aids to Navigation (AtoN) and related navigation information. This approach has widespread significance beyond commonly encountered navigation situations. Increased vessel navigation activity in the Arctic and sub-Arctic regions engenders risk due, in part, to the inability to place navigational aids and buoys in constantly changing ice conditions. Similar conditions exist in tropical regions where sinker placement to moor buoys in sensitive environmental areas with coral reefs is problematic. Underdeveloped regions also lack assets and infrastructure needed to provide adequate navigation services, and infrastructure can also rapidly perish in developed regions during times of war and natural disaster. This research exploits rapidly developing advances in environmental sensing technology, evolving capabilities and improved methods for reporting real time environmental data that can substantially expand electronic navigation aid availability and improve knowledge of undersea terrain and imminent hazards to navigation that may adversely affect ship operations. This is most needed in areas where physical aids to navigation are scarce or non-existent as well as in areas where vessel traffic is congested. Research to expand related vessel capabilities is accomplished to overcome limitations in existing and planned electronic aids, expanding global capabilities and resources at relatively low-cost. New methods for sensor fusion are also explored to vi reduce overall complexity and improve integration with other navigation systems with the goal of simplifying navigation tasks. An additional goal is to supplement training program content by expanding technical resources and capabilities within the confines of existing International Convention on Standards for Training, Certification and Watchkeeping for Seafarers (STCW) requirements, while improving safety by providing new techniques to enhance situational awareness

Underwater iceberg profiling and motion estimation using autonomous underwater vehicles

Icebergs originating from high latitude glaciers have drawn much attention from scientists and offshore operators in the North Atlantic. Scientists are curious about the iceberg drift and deterioration, while the offshore industry is concerned about the potential risks and damages on offshore oil platforms and infrastructures. In order to provide information to improve the iceberg drift and deterioration model constructed by scientists, and to assess the threats posed by icebergs to offshore platforms, iceberg shapes need to be measured. For the above water portion, optical instruments such as a camera and a laser scanner/LIDAR can be used. However, measuring the underwater portion of an iceberg is more challenging due to navigational constraints and sensor limitations. One approach, commonly used, is to deploy a horizontal plane scanning sonar from a support vessel at several locations around the iceberg. There are many drawbacks to this method, including the cost, sensing trade-offs in resolution and coverage, as well as constraints because of weather conditions limiting safe operations. The technology of Autonomous Underwater Vehicles (AUVs) has been developing rapidly in the last two decades. AUVs are commonly chosen to carry scientific sensors for various oceanographic applications. Without human intervention, AUVs can accomplish pre-programmed missions autonomously and deliver scientific data upon the users’ request. With these advantages, AUVs are considered as potential candidates in underwater iceberg sensing operations because they can operate close to icebergs to measure shapes and collect environmental data of the surrounding water. Sonar is usually used for underwater mapping applications. Since AUVs are typically quieter acoustically than manned surface vessels, a low noise to signal ratio can be achieved on sonars carried by AUVs. In this research, a technology of AUV-based underwater iceberg-profiling is evaluated. An iceberg-profiling simulator is constructed to analyse underwater iceberg-profiling missions. With the simulator, the accuracy of AUV-based operation is compared with conventional methods of deploying sonar profilers around icebergs. Beyond the simulation, a guidance, navigation, and control (GNC) system is designed with an objective of guiding the vehicle traveling around the iceberg at a standoff distance. The GNC uses measurements from a mechanical scanning sonar to construct a vehicleattached occupancy map (VOM) that the probability of occupancy of the cells in the VOM is updated based on a dynamic inverse-sonar model. Using the occupancy information about the cells in the VOM, the line-of-sight (LOS) guidance law is used to compute the desired heading for the existing heading controller in the AUV. The GNC is first calibrated and validated in a simulated environment. Then, an AUV equipped with a forward side-looking mechanical scanning sonar is deployed in the field. The GNC guides the vehicle circumnavigated an iceberg autonomously, and underwater shape of the target iceberg is represented using the sonar samples. The point cloud may deviate from the original iceberg shape due to the iceberg movement. A motion estimation algorithm is developed to estimate the iceberg motion for converting the point cloud into an iceberg-centered coordinate system. Two point clouds measured at different times, inputs of the motion estimation algorithm, are presumed to be identical in the iceberg-centered coordinate system. Then, the algorithm iteratively updates the motion estimates based on the translational matrix and rotational matrix from an iterative closest point (ICP) algorithm to match the point clouds. The hypothesis that two point clouds are identical in the iceberg-centered coordinate system is valid when the motion estimates are converged in the updating process. Once the iceberg motion is resolved, the point cloud in the inertial coordinate can be converted in to the iceberg-centered coordinate to present the true iceberg shape. The algorithm for estimating iceberg motion is applied to data collected from the simulation environment and the field trials in Newfoundland

Virtual reality based multi-modal teleoperation using mixed autonomy

The thesis presents a multi modal teleoperation interface featuring an integrated virtual reality based simulation aumented by sensors and image processing capabilities onboard the remotely operated vehicle. The virtual reality interface fuses an existing VR model with live video feed and prediction states, thereby creating a multi modal control interface. Virtual reality addresses the typical limitations of video-based teleoperation caused by signal lag and limited field of view thereby allowing the operator to navigate in a continuous fashion. The vehicle incorporates an on-board computer and a stereo vision system to facilitate obstacle detection. A vehicle adaptation system with a priori risk maps and real state tracking system enables temporary autonomous operation of the vehicle for local navigation around obstacles and automatic re-establishment of the vehicle\u27s teleoperated state. As both the vehicle and the operator share absolute autonomy in stages, the operation is referred to as mixed autonomous. Finally, the system provides real time update of the virtual environment based on anomalies encountered by the vehicle. The system effectively balances the autonomy between the human operator and on board vehicle intelligence. The reliability results of individual components along with overall system implementation and the results of the user study helps show that the VR based multi modal teleoperation interface is more adaptable and intuitive when compared to other interfaces