Underwater localization using imaging sonars in 3D environments

This work proposes a localization method using a mechanically scanned imaging sonar (MSIS), which stands out by its low cost and weight. The proposed method implements a Particle Filter, a Bayesian Estimator, and introduces a measurement model based on sonar simulation theory. To the best of author’s knowledge, there is no similar approach in the literature, as sonar simulation current methods target in syntethic data generation, mostly for object recognition . This stands as the major contribution of the thesis as allows the introduction of the computation of intensity values provided by imaging sonars, while maitaining compability with the already used methods, such as range extraction. Simulations shows the efficiency of the method as well its viability to the utilization of imaging sonar in underwater localization. The new approach make possible, under certain constraints, the extraction of 3D information from a sensor considered, in the literature, as 2D and also in situations where there is no reference at the same horizontal plane of the sensor transducer scanning axis. The localization in complex 3D environment is also an advantage provided by the proposed method.Este trabalho propõe um método de localização utilizando um sonar do tipo MSIS (Mechanically Scanned Imaging Sonar ), o qual se destaca por seu baixo custo e peso. O método implementa um filtro de partículas, um estimador Bayesiano, e introduz um modelo de medição baseado na teoria de simulação de sonares. No conhecimento do autor não há uma abordagem similar na literatura, uma vez que os métodos atuais de simulação de sonar visam a geração de dados sintéticos para o reconhecimento de objetos. Esta é a maior contribuição da tese pois permite a a computação dos valores de intensidade fornecidos pelos sonares do tipo imaging e ao mesmo tempo é compatível com os métodos já utilizados, como extração de distância. Simulações mostram o bom desempenho do método, assim como sua viabilidade para o uso de imaging sonars na localização submarina. A nova abordagem tornou possível, sob certas restrições, a extração de informações 3D de um sensor considerado, na literatura, como somente 2D e também em situações em que não há nehnuma referência no mesmo plano horizontal do eixo de escaneamento do transdutor. A localização em ambientes 3D complexos é também uma vantagem proporcionada pelo método proposto

Underwater simulation and mapping using imaging sonar through ray theory and Hilbert maps

Mapping, sometimes as part of a SLAM system, is an active topic of research and has remarkable solutions using laser scanners, but most of the underwater mapping is focused on 2D maps, treating the environment as a floor plant, or on 2.5D maps of the seafloor. The reason for the problematic of underwater mapping originates in its sensor, i.e. sonars. In contrast to lasers (LIDARs), sonars are unprecise high-noise sensors. Besides its noise, imaging sonars have a wide sound beam effectuating a volumetric measurement. The first part of this dissertation develops an underwater simulator for highfrequency single-beam imaging sonars capable of replicating multipath, directional gain and typical noise effects on arbitrary environments. The simulation relies on a ray theory based method and explanations of how this theory follows from first principles under short-wavelegnth assumption are provided. In the second part of this dissertation, the simulator is combined to a continous map algorithm based on Hilbert Maps. Hilbert maps arise as a machine learning technique over Hilbert spaces, using features maps, applied to the mapping context. The embedding of a sonar response in such a map is a contribution. A qualitative comparison between the simulator ground truth and the reconstucted map reveal Hilbert maps as a promising technique to noisy sensor mapping and, also, indicates some hard to distinguish characteristics of the surroundings, e.g. corners and non smooth features.O mapeamento, às vezes como parte de um sistema SLAM, é um tema de pesquisa ativo e tem soluções notáveis usando scanners a laser, mas a maioria do mapeamento subaquático é focada em mapas 2D, que tratam o ambiente como uma planta, ou mapas 2.5D do fundo do mar. A razão para a dificuldade do mapeamento subaquático origina-se no seu sensor, i.e. sonares. Em contraste com lasers (LIDARs), os sonares são sensores imprecisos e com alto nível de ruído. Além do seu ruído, os sonares do tipo imaging têm um feixe sonoro muito amplo e, com isso, efetuam uma medição volumétrica, ou seja, sobre todo um volume. Na primeira parte dessa dissertação se desenvolve um simulador para sonares do tipo imaging de feixo único de alta frequência capaz de replicar os efeitos típicos de multicaminho, ganho direcional e ruído de fundo em ambientes arbitrários. O simulador implementa um método baseado na teoria geométrica de raios, com todo seu desenvolvimento partindo da acústica subaquática. Na segunda parte dessa dissertação, o simulador é incorporado em um algoritmo de reconstrução de mapas contínuos baseado em Hilbert Maps. Hilbert Maps surge como uma técnica de aprendizado de máquina sobre espaços de Hilbert, usando mapas de características, aplicadas ao contexto de mapeamento. A incorporação de uma resposta de sonar em um tal mapa é uma contribuição desse trabalho. Uma comparação qualitativa entre o ambiente de referência fornecido ao simulador e o mapa reconstruído pela técnica proposta, revela Hilbert Maps como uma técnica promissora para mapeamento atráves de sensores ruidosos e, também, aponta para algumas características do ambiente difíceis de se distinguir, e.g. cantos e regiões não suaves

Quantum-inspired computational imaging

Computational imaging combines measurement and computational methods with the aim of forming images even when the measurement conditions are weak, few in number, or highly indirect. The recent surge in quantum-inspired imaging sensors, together with a new wave of algorithms allowing on-chip, scalable and robust data processing, has induced an increase of activity with notable results in the domain of low-light flux imaging and sensing. We provide an overview of the major challenges encountered in low-illumination (e.g., ultrafast) imaging and how these problems have recently been addressed for imaging applications in extreme conditions. These methods provide examples of the future imaging solutions to be developed, for which the best results are expected to arise from an efficient codesign of the sensors and data analysis tools.Y.A. acknowledges support from the UK Royal Academy of Engineering under the Research Fellowship Scheme (RF201617/16/31). S.McL. acknowledges financial support from the UK Engineering and Physical Sciences Research Council (grant EP/J015180/1). V.G. acknowledges support from the U.S. Defense Advanced Research Projects Agency (DARPA) InPho program through U.S. Army Research Office award W911NF-10-1-0404, the U.S. DARPA REVEAL program through contract HR0011-16-C-0030, and U.S. National Science Foundation through grants 1161413 and 1422034. A.H. acknowledges support from U.S. Army Research Office award W911NF-15-1-0479, U.S. Department of the Air Force grant FA8650-15-D-1845, and U.S. Department of Energy National Nuclear Security Administration grant DE-NA0002534. D.F. acknowledges financial support from the UK Engineering and Physical Sciences Research Council (grants EP/M006514/1 and EP/M01326X/1). (RF201617/16/31 - UK Royal Academy of Engineering; EP/J015180/1 - UK Engineering and Physical Sciences Research Council; EP/M006514/1 - UK Engineering and Physical Sciences Research Council; EP/M01326X/1 - UK Engineering and Physical Sciences Research Council; W911NF-10-1-0404 - U.S. Defense Advanced Research Projects Agency (DARPA) InPho program through U.S. Army Research Office; HR0011-16-C-0030 - U.S. DARPA REVEAL program; 1161413 - U.S. National Science Foundation; 1422034 - U.S. National Science Foundation; W911NF-15-1-0479 - U.S. Army Research Office; FA8650-15-D-1845 - U.S. Department of the Air Force; DE-NA0002534 - U.S. Department of Energy National Nuclear Security Administration)Accepted manuscrip

Emerging Technologies and Approaches for In Situ, Autonomous Observing in the Arctic

Understanding and predicting Arctic change and its impacts on global climate requires broad, sustained observations of the atmosphere-ice-ocean system, yet technological and logistical challenges severely restrict the temporal and spatial scope of observing efforts. Satellite remote sensing provides unprecedented, pan-Arctic measurements of the surface, but complementary in situ observations are required to complete the picture. Over the past few decades, a diverse range of autonomous platforms have been developed to make broad, sustained observations of the ice-free ocean, often with near-real-time data delivery. Though these technologies are well suited to the difficult environmental conditions and remote logistics that complicate Arctic observing, they face a suite of additional challenges, such as limited access to satellite services that make geolocation and communication possible. This paper reviews new platform and sensor developments, adaptations of mature technologies, and approaches for their use, placed within the framework of Arctic Ocean observing needs

Interoperability Among Unmanned Maritime Vehicles: Review and First In-field Experimentation

Complex maritime missions, both above and below the surface, have traditionally been carried out by manned surface ships and submarines equipped with advanced sensor systems. Unmanned Maritime Vehicles (UMVs) are increasingly demonstrating their potential for improving existing naval capabilities due to their rapid deployability, easy scalability, and high reconfigurability, offering a reduction in both operational time and cost. In addition, they mitigate the risk to personnel by leaving the man far-from-the-risk but in-the-loop of decision making. In the long-term, a clear interoperability framework between unmanned systems, human operators, and legacy platforms will be crucial for effective joint operations planning and execution. However, the present multi-vendor multi-protocol solutions in multi-domain UMVs activities are hard to interoperate without common mission control interfaces and communication protocol schemes. Furthermore, the underwater domain presents significant challenges that cannot be satisfied with the solutions developed for terrestrial networks. In this paper, the interoperability topic is discussed blending a review of the technological growth from 2000 onwards with recent authors' in-field experience; finally, important research directions for the future are given. Within the broad framework of interoperability in general, the paper focuses on the aspect of interoperability among UMVs not neglecting the role of the human operator in the loop. The picture emerging from the review demonstrates that interoperability is currently receiving a high level of attention with a great and diverse deal of effort. Besides, the manuscript describes the experience from a sea trial exercise, where interoperability has been demonstrated by integrating heterogeneous autonomous UMVs into the NATO Centre for Maritime Research and Experimentation (CMRE) network, using different robotic middlewares and acoustic modem technologies to implement a multistatic active sonar system. A perspective for the interoperability in marine robotics missions emerges in the paper, through a discussion of current capabilities, in-field experience and future advanced technologies unique to UMVs. Nonetheless, their application spread is slowed down by the lack of human confidence. In fact, an interoperable system-of-systems of autonomous UMVs will require operators involved only at a supervisory level. As trust develops, endorsed by stable and mature interoperability, human monitoring will be diminished to exploit the tremendous potential of fully autonomous UMVs

Active and Passive Acoustic Methods for In-situ Monitoring of the Ocean Status

Recent European strategic plans for the successful monitoring of the status of the ocean push on the development of an integrated observing system able to further link existing instruments and techniques with the aim to complement each other and answer open issues. A more intensive use of acoustic devices could contribute to the knowledge of oceanographic processes exploiting the characteristic of sound to travel in the ocean for a wide area than in the atmosphere. In this context, the installation of passive acoustic instruments, able to listen to ambient noise on fixed or mobile platforms, could contribute to provide information on sound budget and to enhance the monitoring capacity of meteorological phenomena also in the open ocean. Instead, the deployment of active acoustic instruments can be of benefit for monitoring biological activities through the analysis of backscatter data as well as for monitoring ocean waves

A future for intelligent autonomous ocean observing systems

Ocean scientists have dreamed of and recently started to realize an ocean observing revolution with autonomous observing platforms and sensors. Critical questions to be answered by such autonomous systems are where, when, and what to sample for optimal information, and how to optimally reach the sampling locations. Definitions, concepts, and progress towards answering these questions using quantitative predictions and fundamental principles are presented. Results in reachability and path planning, adaptive sampling, machine learning, and teaming machines with scientists are overviewed. The integrated use of differential equations and theory from varied disciplines is emphasized. The results provide an inference engine and knowledge base for expert autonomous observing systems. They are showcased using a set of recent at-sea campaigns and realistic simulations. Real-time experiments with identical autonomous underwater vehicles (AUVs) in the Buzzards Bay and Vineyard Sound region first show that our predicted time-optimal paths were faster than shortest distance paths. Deterministic and probabilistic reachability and path forecasts issued and validated for gliders and floats in the northern Arabian Sea are then presented. Novel Bayesian adaptive sampling for hypothesis testing and optimal learning are finally shown to forecast the observations most informative to estimate the accuracy of model formulations, the values of ecosystem parameters and dynamic fields, and the presence of Lagrangian Coherent Structures

Multipurpose acoustic networks in the integrated arctic ocean observing system

The dramatic reduction of sea ice in the Arctic Ocean will increase human activities in the coming years. This activity will be driven by increased demand for energy and the marine resources of an Arctic Ocean accessible to ships. Oil and gas exploration, fisheries, mineral extraction, marine transportation, research and development, tourism, and search and rescue will increase the pressure on the vulnerable Arctic environment. Technologies that allow synoptic in situ observations year-round are needed to monitor and forecast changes in the Arctic atmosphere-ice-ocean system at daily, seasonal, annual, and decadal scales. These data can inform and enable both sustainable development and enforcement of international Arctic agreements and treaties, while protecting this critical environment. In this paper, we discuss multipurpose acoustic networks, including subsea cable components, in the Arctic. These networks provide communication, power, underwater and under-ice navigation, passive monitoring of ambient sound (ice, seismic, biologic, and anthropogenic), and acoustic remote sensing (tomography and thermometry), supporting and complementing data collection from platforms, moorings, and vehicles. We support the development and implementation of regional to basin-wide acoustic networks as an integral component of a multidisciplinary in situ Arctic Ocean observatory

A Multi-Sensor Fusion-Based Underwater Slam System

This dissertation addresses the problem of real-time Simultaneous Localization and Mapping (SLAM) in challenging environments. SLAM is one of the key enabling technologies for autonomous robots to navigate in unknown environments by processing information on their on-board computational units. In particular, we study the exploration of challenging GPS-denied underwater environments to enable a wide range of robotic applications, including historical studies, health monitoring of coral reefs, underwater infrastructure inspection e.g., bridges, hydroelectric dams, water supply systems, and oil rigs. Mapping underwater structures is important in several fields, such as marine archaeology, Search and Rescue (SaR), resource management, hydrogeology, and speleology. However, due to the highly unstructured nature of such environments, navigation by human divers could be extremely dangerous, tedious, and labor intensive. Hence, employing an underwater robot is an excellent fit to build the map of the environment while simultaneously localizing itself in the map. The main contribution of this dissertation is the design and development of a real-time robust SLAM algorithm for small and large scale underwater environments. SVIn – a novel tightly-coupled keyframe-based non-linear optimization framework fusing Sonar, Visual, Inertial and water depth information with robust initialization, loop-closing, and relocalization capabilities has been presented. Introducing acoustic range information to aid the visual data, shows improved reconstruction and localization. The availability of depth information from water pressure enables a robust initialization and refines the scale factor, as well as assists to reduce the drift for the tightly-coupled integration. The complementary characteristics of these sensing v modalities provide accurate and robust localization in unstructured environments with low visibility and low visual features – as such make them the ideal choice for underwater navigation. The proposed system has been successfully tested and validated in both benchmark datasets and numerous real world scenarios. It has also been used for planning for underwater robot in the presence of obstacles. Experimental results on datasets collected with a custom-made underwater sensor suite and an autonomous underwater vehicle (AUV) Aqua2 in challenging underwater environments with poor visibility, demonstrate performance never achieved before in terms of accuracy and robustness. To aid the sparse reconstruction, a contour-based reconstruction approach utilizing the well defined edges between the well lit area and darkness has been developed. In particular, low lighting conditions, or even complete absence of natural light inside caves, results in strong lighting variations, e.g., the cone of the artificial video light intersecting underwater structures and the shadow contours. The proposed method utilizes these contours to provide additional features, resulting into a denser 3D point cloud than the usual point clouds from a visual odometry system. Experimental results in an underwater cave demonstrate the performance of our system. This enables more robust navigation of autonomous underwater vehicles using the denser 3D point cloud to detect obstacles and achieve higher resolution reconstructions