Internet of Underwater Things and Big Marine Data Analytics -- A Comprehensive Survey

The Internet of Underwater Things (IoUT) is an emerging communication ecosystem developed for connecting underwater objects in maritime and underwater environments. The IoUT technology is intricately linked with intelligent boats and ships, smart shores and oceans, automatic marine transportations, positioning and navigation, underwater exploration, disaster prediction and prevention, as well as with intelligent monitoring and security. The IoUT has an influence at various scales ranging from a small scientific observatory, to a midsized harbor, and to covering global oceanic trade. The network architecture of IoUT is intrinsically heterogeneous and should be sufficiently resilient to operate in harsh environments. This creates major challenges in terms of underwater communications, whilst relying on limited energy resources. Additionally, the volume, velocity, and variety of data produced by sensors, hydrophones, and cameras in IoUT is enormous, giving rise to the concept of Big Marine Data (BMD), which has its own processing challenges. Hence, conventional data processing techniques will falter, and bespoke Machine Learning (ML) solutions have to be employed for automatically learning the specific BMD behavior and features facilitating knowledge extraction and decision support. The motivation of this paper is to comprehensively survey the IoUT, BMD, and their synthesis. It also aims for exploring the nexus of BMD with ML. We set out from underwater data collection and then discuss the family of IoUT data communication techniques with an emphasis on the state-of-the-art research challenges. We then review the suite of ML solutions suitable for BMD handling and analytics. We treat the subject deductively from an educational perspective, critically appraising the material surveyed.Comment: 54 pages, 11 figures, 19 tables, IEEE Communications Surveys & Tutorials, peer-reviewed academic journa

Cooperative localisation in underwater robotic swarms for ocean bottom seismic imaging.

Spatial information must be collected alongside the data modality of interest in wide variety of sub-sea applications, such as deep sea exploration, environmental monitoring, geological and ecological research, and samples collection. Ocean-bottom seismic surveys are vital for oil and gas exploration, and for productivity enhancement of an existing production facility. Ocean-bottom seismic sensors are deployed on the seabed to acquire those surveys. Node deployment methods used in industry today are costly, time-consuming and unusable in deep oceans. This study proposes the autonomous deployment of ocean-bottom seismic nodes, implemented by a swarm of Autonomous Underwater Vehicles (AUVs). In autonomous deployment of ocean-bottom seismic nodes, a swarm of sensor-equipped AUVs are deployed to achieve ocean-bottom seismic imaging through collaboration and communication. However, the severely limited bandwidth of underwater acoustic communications and the high cost of maritime assets limit the number of AUVs that can be deployed for experiments. A holistic fuzzy-based localisation framework for large underwater robotic swarms (i.e. with hundreds of AUVs) to dynamically fuse multiple position estimates of an autonomous underwater vehicle is proposed. Simplicity, exibility and scalability are the main three advantages inherent in the proposed localisation framework, when compared to other traditional and commonly adopted underwater localisation methods, such as the Extended Kalman Filter. The proposed fuzzy-based localisation algorithm improves the entire swarm mean localisation error and standard deviation (by 16.53% and 35.17% respectively) at a swarm size of 150 AUVs when compared to the Extended Kalman Filter based localisation with round-robin scheduling. The proposed fuzzy based localisation method requires fuzzy rules and fuzzy set parameters tuning, if the deployment scenario is changed. Therefore a cooperative localisation scheme that relies on a scalar localisation confidence value is proposed. A swarm subset is navigationally aided by ultra-short baseline and a swarm subset (i.e. navigation beacons) is configured to broadcast navigation aids (i.e. range-only), once their confidence values are higher than a predetermined confidence threshold. The confidence value and navigation beacons subset size are two key parameters for the proposed algorithm, so that they are optimised using the evolutionary multi-objective optimisation algorithm NSGA-II to enhance its localisation performance. Confidence value-based localisation is proposed to control the cooperation dynamics among the swarm agents, in terms of aiding acoustic exteroceptive sensors. Given the error characteristics of a commercially available ultra-short baseline system and the covariance matrix of a trilaterated underwater vehicle position, dead reckoning navigation - aided by Extended Kalman Filter-based acoustic exteroceptive sensors - is performed and controlled by the vehicle's confidence value. The proposed confidence-based localisation algorithm has significantly improved the entire swarm mean localisation error when compared to the fuzzy-based and round-robin Extended Kalman Filter-based localisation methods (by 67.10% and 59.28% respectively, at a swarm size of 150 AUVs). The proposed fuzzy-based and confidence-based localisation algorithms for cooperative underwater robotic swarms are validated on a co-simulation platform. A physics-based co-simulation platform that considers an environment's hydrodynamics, industrial grade inertial measurement unit and underwater acoustic communications characteristics is implemented for validation and optimisation purposes

Collaborative Unmanned Vehicles for Inspection, Maintenance, and Repairs of Offshore Wind Turbines

Operations and maintenance of Offshore Wind Turbines (OWTs) are challenging, with manual operators constantly exposed to hazardous environments. Due to the high task complexity associated with the OWT, the transition to unmanned solutions remains stagnant. Efforts toward unmanned operations have been observed using Unmanned Aerial Vehicles (UAVs) and Unmanned Underwater Vehicles (UUVs) but are limited mostly to visual inspections only. Collaboration strategies between unmanned vehicles have introduced several opportunities that would enable unmanned operations for the OWT maintenance and repair activities. There have been many papers and reviews on collaborative UVs. However, most of the past papers reviewed collaborative UVs for surveillance purposes, search and rescue missions, and agricultural activities. This review aims to present the current capabilities of Unmanned Vehicles (UVs) used in OWT for Inspection, Maintenance, and Repair (IMR) operations. Strategies to implement collaborative UVs for complex tasks and their associated challenges are discussed together with the strategies to solve localization and navigation issues, prolong operation time, and establish effective communication within the OWT IMR operations. This paper also briefly discusses the potential failure modes for collaborative approaches and possible redundancy strategies to manage them. The collaborative strategies discussed herein will be of use to researchers and technology providers in identifying significant gaps that have hindered the implementation of full unmanned systems which have significant impacts towards the net zero strategy.</jats:p

Adaptive Sampling with Mobile Sensor Networks

Mobile sensor networks have unique advantages compared with wireless sensor networks. The mobility enables mobile sensors to flexibly reconfigure themselves to meet sensing requirements. In this dissertation, an adaptive sampling method for mobile sensor networks is presented. Based on the consideration of sensing resource constraints, computing abilities, and onboard energy limitations, the adaptive sampling method follows a down sampling scheme, which could reduce the total number of measurements, and lower sampling cost. Compressive sensing is a recently developed down sampling method, using a small number of randomly distributed measurements for signal reconstruction. However, original signals cannot be reconstructed using condensed measurements, as addressed by Shannon Sampling Theory. Measurements have to be processed under a sparse domain, and convex optimization methods should be applied to reconstruct original signals. Restricted isometry property would guarantee signals can be recovered with little information loss. While compressive sensing could effectively lower sampling cost, signal reconstruction is still a great research challenge. Compressive sensing always collects random measurements, whose information amount cannot be determined in prior. If each measurement is optimized as the most informative measurement, the reconstruction performance can perform much better. Based on the above consideration, this dissertation is focusing on an adaptive sampling approach, which could find the most informative measurements in unknown environments and reconstruct original signals. With mobile sensors, measurements are collect sequentially, giving the chance to uniquely optimize each of them. When mobile sensors are about to collect a new measurement from the surrounding environments, existing information is shared among networked sensors so that each sensor would have a global view of the entire environment. Shared information is analyzed under Haar Wavelet domain, under which most nature signals appear sparse, to infer a model of the environments. The most informative measurements can be determined by optimizing model parameters. As a result, all the measurements collected by the mobile sensor network are the most informative measurements given existing information, and a perfect reconstruction would be expected. To present the adaptive sampling method, a series of research issues will be addressed, including measurement evaluation and collection, mobile network establishment, data fusion, sensor motion, signal reconstruction, etc. Two dimensional scalar field will be reconstructed using the method proposed. Both single mobile sensors and mobile sensor networks will be deployed in the environment, and reconstruction performance of both will be compared.In addition, a particular mobile sensor, a quadrotor UAV is developed, so that the adaptive sampling method can be used in three dimensional scenarios

Cooperative Localization in Mobile Underwater Acoustic Sensor Networks

Die großflächige Erkundung und Überwachung von Tiefseegebieten gewinnt mehr und mehr an Bedeutung für Industrie und Wissenschaft. Diese schwer zugänglichen Areale in der Tiefsee können nur mittels Teams unbemannter Tauchbote effizient erkundet werden. Aufgrund der hohen Kosten, war bisher ein Einsatz von mehreren autonomen Unterwasserfahrzeugen (AUV) wirtschaftlich undenkbar, wodurch AUV-Teams nur in Simulationen erforscht werden konnten. In den letzten Jahren konnte jedoch eine Entwicklung hin zu günstigeren und robusteren AUVs beobachtet werden. Somit wird der Einsatz von AUV-Teams in Zukunft zu einer realen Option. Die wachsende Nachfrage nach Technologien zur Unterwasseraufklärung und Überwachung konnte diese Entwicklung noch zusätzlich beschleunigen. Eine der größten technischen Hürden für tief tauchende AUVs ist die Unterwasserlokalisierug. Satelitengestützte Navigation ist in der Tiefe nicht möglich, da Radiowellen bereits nach wenigen Metern im Wasser stark an Intensität verlieren. Daher müssen neue Ansätze für die Unterwasserlokalisierung entwickelt werden die sich auch für Fahrzeugenverbände skalieren lassen. Der Einsatz von AUV-Teams ermöglicht nicht nur völlig neue Möglichkeiten der Kooperation, sondern erlaubt auch jedem einzelnen AUV von den Navigationsdaten der anderen Fahrzeuge im Verband zu profitieren, um die eigene Lokalisierung zu verbessern. In dieser Arbeit wird ein kooperativer Lokalisierungsansatz vorgestellt, welcher auf dem Nachrichtenaustausch durch akustische Ultra-Short Base-Line (USBL) Modems basiert. Ein akustisches Modem ermöglicht die Übertragung von Datenpaketen im Wasser, wärend ein USBL-Sensor die Richtung einer akustischen Quelle bestimmen kann. Durch die Kombination von Modem und Sensor entsteht ein wichtiges Messinstrument für die Unterwasserlokalisierung. Wenn ein Fahrzeug ein Datenpaket mit seiner eignen Position aussendet, können andere Fahrzeuge mit einem USBL-Modem diese Nachricht empfangen. In Verbindung mit der Richtungsmessung zur Quelle, können diese Daten von einem Empfangenden AUV verwendet werden, um seine eigene Positionsschatzung zu verbessern. Diese Arbeit schlägt einen Ansatz zur Fusionierung der empfangenen Nachricht mit der Richtungsmessung vor, welcher auch die jeweiligen Messungenauigkeiten berücksichtigt. Um die Messungenauigkeit des komplexen USBL-Sensors bestimmen zu können, wurde zudem ein detailliertes Sensormodell entwickelt. Zunächst wurden existierende Ansätze zur kooperativen Lokalisierung (CL) untersucht, um daraus eine Liste von erwünschten Eigenschaften für eine CL abzuleiten. Darauf aufbauend wurde der Deep-Sea Network Lokalisation (DNL) Ansatz entwickelt. Bei DNL handelt es sich um eine CL Methode, bei der die Skalierbarkeit sowie die praktische Anwendbarkeit im Fokus stehen. DNL ist als eine Zwischenschicht konzipiert, welche USBL-Modem und Navigationssystem miteinander verbindet. Es werden dabei Messwerte und Kommunikationsdaten des USBL zu einer Standortbestimmung inklusive Richtungsschätzung fusioniert und an das Navigationssystem weiter geleitet, ähnlich einem GPS-Sensor. Die Funktionalität von USBL-Modell und DNL konnten evaluiert werden anhand von Messdaten aus Seeerprobungen in der Ostsee sowie im Mittelatlantik. Die Qualität einer CL hangt häufig von vielen unterschiedlichen Faktoren ab. Die Netzwerktopologie muss genauso berücksichtig werden wie die Lokalisierungsfähigkeiten jedes einzelnen Teilnehmers. Auch das Kommunikationsverhalten der einzelnen Teilnehmer bestimmt, welche Informationen im Netzwerk vorhanden sind und hat somit einen starken Einfluss auf die CL. Um diese Einflussfaktoren zu untersuchen, wurden eine Reihe von Szenarien simuliert, in denen Kommunikationsverhalten und Netzwerktopologie für eine Gruppe von AUVs variiert wurden. In diesen Experimenten wurden die AUVs durch ein Oberflächenfahrzeug unterstützt, welches seine geo-referenzierte Position über DNL an die getauchten Fahrzeuge weiter leitete. Anhand der untersuchten Topologie können die Experimente eingeteilt werden in Single-Hop und Multi-Hop. Single-Hop bedeutet, dass jedes AUV sich in der Sendereichweite des Oberflächenfahrzeugs befindet und dessen Positionsdaten auf direktem Wege erhält. Wie die Ergebnisse der Single-Hop Experimente zeigen, kann der Lokalisierungsfehler der AUVs eingegrenzt werden, wenn man DNL verwendet. Dabei korreliert der Lokalisierungsfehler mit der kombinierten Ungenauigkeit von USBL-Messung und Oberflächenfahrzeugposition. Bei den Multi-Hop Experimenten wurde die Topologie so geändert, dass sich nur eines der AUVs in direkter Sendereichweite des Oberflächenfahrzeugs befindet. Dieses AUV verbessert seine Position mit den empfangen Daten des Oberflächenfahrzeugs und sendet wiederum seine verbesserte Position an die anderen AUVs. Auch hier konnte gezeigt werden, dass sich der Lokalisierungfehler der Gruppe mit DNL einschränken lässt. Ändert man nun das Schema der Kommunikation so, dass alle AUVs zyklisch ihre Position senden, zeigte sich eine Verschlechterung der Lokalisierungsqualität der Gruppe. Dieses unerwartet Ergebnis konnte auf einen Teil des DNL-Algorithmus zurück geführt werden. Da die verwendete USBL-Klasse nur die Richtung eines Signals misst, nicht jedoch die Entfernung zum Sender, wird in der DNL-Schicht eine Entfernungsschatzung vorgenommen. Wenn die Kommunikation nicht streng unidirektional ist, entsteht eine Ruckkopplungsschleife, was zu fehlerhaften Entfernungsschatzungen führt. Im letzten Experiment wird gezeigt wie sich dieses Problem vermeiden lasst, mithilfe einer relativ neue USBL-Klasse, die sowohl Richtung als auch Entfernung zum Sender misst. Die zwei wesentlichen Beiträge dieser Arbeit sind das USBL-Model zum einen und zum Anderen, der neue kooperative Lokalisierungsansatz DNL. Mithilfe des Sensormodels lassen sich nicht nur Messabweichungen einer USBL-Messung bestimmen, es kann auch dazu genutzt werden, einige Fehlereinflüsse zu korrigieren. Mit DNL wurde eine skalierbare CL-Methode entwickelt, die sich gut für den den Einsatz bei mobilen Unterwassersensornetzwerken eignet. Durch das Konzept als Zwischenschicht, lasst sich DNL einfach in bestehende Navigationslösungen integrieren, um die Langzeitstabilität der Navigation für große Verbände von tiefgetauchten Fahrzeugen zu gewährleisten. Sowohl USBL-Model als auch DNL sind dabei so ressourcenschonend, dass sie auf dem Computer eines Standard USBL laufen können, ohne die ursprüngliche Funktionalität einzuschränken, was den praktischen Einsatz zusätzlich vereinfacht

BathyBoat: An Autonomous Surface Vessel for Stand-alone Survey and Underwater Vehicle Network Supervision

Exploration of remote environments, once the domain of intrepid adventurers, can now be conducted in relative safety using unmanned vehicles. This article describes the joint University of Michigan (UMich) and Michigan Tech Research Institute’s project to design and to build a new autonomous surface vessel (ASV) for use in research, education, and resource management as well as in the commercial sector. Originally designed to assist with bathymetric surveys in the wilderness of northern Alaska, the BathyBoat has become a test-bed platform for new research in collaborative heterogeneous underwater robotic search and survey missions in ports, harbors, lakes, and rivers. The UMich Marine Hydrodynamics Laboratories are actively researching autonomous technologies such as cooperative navigation, surface vessel control, and multivehicle search and survey using the BathyBoat and the UMich Perceptual Robotics Laboratory’s Iver2 autonomous underwater vehicles. This article presents an overview of these research topics and highlights relevant real-world testing and recent missions involving the BathyBoat ASV on Alaska’s North Slope, the harbors of Illinois, and various riverine environments in Michigan.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/83223/1/2010e_MTS_Journal.pd

Collaborative autonomy in heterogeneous multi-robot systems

As autonomous mobile robots become increasingly connected and widely deployed in different domains, managing multiple robots and their interaction is key to the future of ubiquitous autonomous systems. Indeed, robots are not individual entities anymore. Instead, many robots today are deployed as part of larger fleets or in teams. The benefits of multirobot collaboration, specially in heterogeneous groups, are multiple. Significantly higher degrees of situational awareness and understanding of their environment can be achieved when robots with different operational capabilities are deployed together. Examples of this include the Perseverance rover and the Ingenuity helicopter that NASA has deployed in Mars, or the highly heterogeneous robot teams that explored caves and other complex environments during the last DARPA Sub-T competition. This thesis delves into the wide topic of collaborative autonomy in multi-robot systems, encompassing some of the key elements required for achieving robust collaboration: solving collaborative decision-making problems; securing their operation, management and interaction; providing means for autonomous coordination in space and accurate global or relative state estimation; and achieving collaborative situational awareness through distributed perception and cooperative planning. The thesis covers novel formation control algorithms, and new ways to achieve accurate absolute or relative localization within multi-robot systems. It also explores the potential of distributed ledger technologies as an underlying framework to achieve collaborative decision-making in distributed robotic systems. Throughout the thesis, I introduce novel approaches to utilizing cryptographic elements and blockchain technology for securing the operation of autonomous robots, showing that sensor data and mission instructions can be validated in an end-to-end manner. I then shift the focus to localization and coordination, studying ultra-wideband (UWB) radios and their potential. I show how UWB-based ranging and localization can enable aerial robots to operate in GNSS-denied environments, with a study of the constraints and limitations. I also study the potential of UWB-based relative localization between aerial and ground robots for more accurate positioning in areas where GNSS signals degrade. In terms of coordination, I introduce two new algorithms for formation control that require zero to minimal communication, if enough degree of awareness of neighbor robots is available. These algorithms are validated in simulation and real-world experiments. The thesis concludes with the integration of a new approach to cooperative path planning algorithms and UWB-based relative localization for dense scene reconstruction using lidar and vision sensors in ground and aerial robots

Sensor Fusion for Mobile Robot Localization using UWB and ArUco Markers

Uma das principais características para considerar um robô autónomo é o facto de este ser capaz de se localizar, em tempo real, no seu ambiente, ou seja saber a sua posição e orientação. Esta é uma área desafiante que tem sido estudada por diversos investigadores em todo o mundo. Para obter a localização de um robô é possível recorrer a diferentes metodologias. No entanto há metodologias que apresentam problemas em diferentes circunstâncias, como é o caso da odometria que sofre de acumulação de erros com a distância percorrida pelo robô. Outro problema existente em diversas metodologias é a incerteza na deteção do robô devido a ruído presente nos sensores. Com o intuito de obter uma localização mais robusta do robô e mais tolerante a falhas é possível combinar diversos sistemas de localização, combinando assim as vantagens de cada um deles. Neste trabalho, será utilizado o sistema Pozyx, uma solução de baixo custo que fornece informação de posicionamento com o auxílio da tecnologia Ultra-WideBand Time-of-Flight (UWB ToF). Também serão utilizados marcadores ArUco colocados no ambiente que através da sua identificação por uma câmara é também possível obter informação de posicionamento. Estas duas soluções irão ser estudadas e implementadas num robô móvel, através de um esquema de localização baseada em marcadores. Primeiramente, irá ser feita uma caracterização do erro de ambos os sistemas, uma vez que as medidas não são perfeitas, havendo sempre algum ruído nas medições. De seguida, as medidas fornecidas pelos sistemas irão ser filtradas e fundidas com os valores da odometria do robô através da implementação de um Filtro de Kalman Extendido (EKF). Assim, é possível obter a pose do robô (posição e orientação), pose esta que é comparada com a pose fornecida por um sistema de Ground-Truth igualmente desenvolvido para este trabalho com o auxílio da libraria ArUco, percebendo assim a precisão do algoritmo desenvolvido. O trabalho desenvolvido mostrou que com a utilização do sistema Pozyx e dos marcadores ArUco é possível melhorar a localização do robô, o que significa que é uma solução adequada e eficaz para este fim.One of the main characteristics to consider a robot truly autonomous is the fact that it is able to locate itself, in real time, in its environment, that is, to know its position and orientation. This is a challenging area that has been studied by several researchers around the world. To obtain the localization of a robot it is possible to use different methodologies. However, there are methodologies that present problems in different circumstances, as is the case of odometry that suffers from error accumulation with the distance traveled by the robot. Another problem existing in several methodologies is the uncertainty in the sensing of the robot due to noise present in the sensors. In order to obtain a more robust localization of the robot and more fault tolerant it is possible to combine several localization systems, thus combining the advantages of each one. In this work, the Pozyx system will be used, a low-cost solution that provides positioning information through Ultra-WideBand Time-of-Flight (UWB ToF) technology. It will also be used ArUco markers placed in the environment that through their identification by a camera it is also possible to obtain positioning information. These two solutions will be studied and implemented in a mobile robot, through a beacon-based localization scheme. First, an error characterization of both systems will be performed, since the measurements are not perfect, and there is always some noise in the measurements. Next, the measurements provided by the systems will be filtered and fused with the robot's odometry values by the implementation of an Extended Kalman Filter (EKF). In this way, it is possible to obtain the robot's pose, i.e position and orientation, which is compared with the pose provided by a Ground-Truth system also developed for this work with the aid of the ArUco library, thus realizing the accuracy of the developed algorithm. The developed work showed that with the use of the Pozyx system and ArUco markers it is possible to improve the robot localization, meaning that it is an adequate and effective solution for this purpose