Exploratory dives are traditionally the first step for marine scientists to
acquire information on a previously unknown area of scientific interest. Manned
submersibles have been the platform of choice for such exploration, as they allow
a high level of environmental perception by the scientist on-board, and the ability
to take informed decisions on what to explore next. However, manned submersibles
have extremely high operation costs and provide very limited bottom time. Remotely
operated vehicles (ROVs) can partially address these two issues, but have operational
and cost constraints that restrict their usage.
This paper discusses new capabilities to assist scientists operating lightweight hybrid
remotely operated vehicles (HROV) in exploratory missions of mapping and
surveying. The new capabilities, under development within the Spanish National
project OMNIUS, provide a new layer of autonomy for HROVs by exploring three key
concepts: Omni-directional optical sensing for collaborative immersive exploration,
Proximity safety awareness and Online mapping during mission time.Peer Reviewe

Bosch Alay, Josep

Campos Dausà, Ricard

Elibol, Armagan

Estrela Gracias, Nuno Ricardo

Garcia Campos, Rafael

Neumann, Laszlo

Nicosevici, Tudor

Prados Gutiérrez, Ricard

Quintana Plana, Josep

English

Exploratory dives are traditionally the first step for marine scientists to
acquire information on a previously unknown area of scientific interest. Manned
submersibles have been the platform of choice for such exploration, as they allow
a high level of environmental perception by the scientist on-board, and the ability
to take informed decisions on what to explore next. However, manned submersibles
have extremely high operation costs and provide very limited bottom time. Remotely
operated vehicles (ROVs) can partially address these two issues, but have operational
and cost constraints that restrict their usage.
This paper discusses new capabilities to assist scientists operating lightweight hybrid
remotely operated vehicles (HROV) in exploratory missions of mapping and
surveying. The new capabilities, under development within the Spanish National
project OMNIUS, provide a new layer of autonomy for HROVs by exploring three key
concepts: Omni-directional optical sensing for collaborative immersive exploration,
Proximity safety awareness and Online mapping during mission time

Revistes Catalanes amb Accés Obert

Omnidirectional underwater surveying and telepresence

Exploratory dives are traditionally the first step for marine scientists to

acquire information on a previously unknown area of scientific interest. Manned

submersibles have been the platform of choice for such exploration, as they allow

a high level of environmental perception by the scientist on-board, and the ability

to take informed decisions on what to explore next. However, manned submersibles

have extremely high operation costs and provide very limited bottom time. Remotely

operated vehicles (ROVs) can partially address these two issues, but have operational

and cost constraints that restrict their usage.

This paper discusses new capabilities to assist scientists operating lightweight hybrid

remotely operated vehicles (HROV) in exploratory missions of mapping and

surveying. The new capabilities, under development within the Spanish National

project OMNIUS, provide a new layer of autonomy for HROVs by exploring three key

concepts: Omni-directional optical sensing for collaborative immersive exploration,

Proximity safety awareness and Online mapping during mission time.Peer Reviewe

UPCommons

UPCommons. Portal del coneixement obert de la UPC

75instrumentation viewpOint- 19 - MARTECH 16 instrumentation viewpOint - 19 - MARTECH 16ID33- OMNIDIRECTIONAL UNDERWATER SURVEYING AND TELEPRESENCENuno Gracias25, Rafael Garcia26, Ricard Campos27, Ricard Prados28, Josep Bosch22, Armagan Elibol46, Tudor Nicosevici29, Laszlo Neumann24, Josep Quintana23Abstract – Exploratory dives are traditionally the first step for marine scientists to acquire information on a previously unknown area of scientific interest. Manned submersibles have been the platform of choice for such exploration, as they allow a high level of environmental perception by the scientist on-board, and the ability to take informed decisions on what to explore next. However, manned submersibles have extremely high operation costs and provide very limited bottom time. Remote-ly operated vehicles (ROVs) can partially address these two issues, but have opera-tional and cost constraints that restrict their usage. This paper discusses new capabilities to assist scientists operating lightweight hy-brid remotely operated vehicles (HROV) in exploratory missions of mapping and surveying. The new capabilities, under development within the Spanish National project OMNIUS, provide a new layer of autonomy for HROVs by exploring three key concepts: Omni-directional optical sensing for collaborative immersive exploration, Proximity safety awareness and Online mapping during mission time.Keywords - HROVs, telepresence, collaborative underwater exploration, advanced assisted 2D/3D mapping, autonomous safety awareness, mapping of challenging environments, mapping quality control.I. CONCEPTThe study of marine habitats is a key component for understanding and man-aging ocean resources. In addition to their scientific and economical interest, underwater studies can be used for assessing sites that are important from a conservation point of view, determining areas sensitive to disturbance and pol-lution, development of coastal and marine protected areas management plans.Currently, such studies face serious challenges, mainly resulting from the harsh-ness and the hazard of the underwater environment. The use of divers drasti-cally limits the depth range and the duration of the studies, while the use of manned submersibles involves high operational costs.Within the Spanish national project OMNIUS, a new paradigm is being pursued which provides the basis for more direct, interactive and efficient underwater studies, while reducing the associated costs. From the hardware point of view, the proposal has three main elements (Fig. 1): (i) an intelligent ROV that will be used to acquire and transmit data such as: panoramic visual data, acoustic imaging, and other mission-specific data; (ii) a data processing unit that will process the data acquired by the ROV, creating 2D/3D multimodal representations of the environment; and (iii) a telepresence environment, in which the scientists will have a 360 degrees panoramic real-time view of the area currently surveyed by the ROV, complemented with other navigation sensor information.The starting hypothesis is that lightweight hybrid remotely operated vehicles (HROV), similarly to the Girona500 [1], can be used by a much wider group of marine scientists without requiring specialized personnel, if such platforms are endowed with suitable mapping and navigation capabilities.The goal of this effort is to develop new capabilities to assist scientists operating a lightweight hybrid remotely operated vehicle in exploratory missions of map-ping and surveying, to ensure the creation of high quality map products at the end. The new capabilities provide a new layer of autonomy for HROVs that will assist scientists in using the platform as an exploratory tool.Regarding research and innovation, the general objectives are to devise and validate experimentally three new capabilities of high operational impact: (1) advanced assisted mapping, (2) spatial awareness and safety, and (3) user im-mersion/telepresence.1 - Advanced assisted mapping - It addresses the need to provide the scientists with high-level information on the mapping data that is being acquired, during exploratory and mapping surveys. This high-level information addresses two common concerns – what areas in the surrounding regions contain interesting features to be mapped, and once the scientist has decided to start mapping, if that area is being properly mapped. This information will assist the scientist in taking informed decisions on what to map and inspect, while taking full control of the exploration process.2 - Spatial awareness and safety - It aims at providing the vehicle with proximity awareness capabilities when navigating in complex 3D and unstructured envi-ronments. Examples of such environments are confined spaces or very steep slopes. The aim is to release human operators from the pressing need to monitor vehicle safety when operating in very close proximity to obstacles in the sur-roundings. The vehicle will know when it is unsafe to navigate, and override or correct human control input if needed. This will allow the inspection and map-ping of difficult areas that, otherwise would not be reachable due to the risk of damaging or entangling the vehicle.3 - Immersion and Telepresence - aims at validating and demonstrating the ex-pected impact of using an omnidirectional optical sensor as part of a collab-orative visualization and exploration tool. This effort addresses the following question: Can a low-cost, customized Hybrid ROV/AUV be used to replicate the experience of a group of scientists on a manned submersible for purely interac-tive (and reactive) surveying?II. ONGOING WORK AND ACHIEVEMENTSSeveral technical advances are being pursuit within the project. Among others, underwater omnidirectional imaging is a key sensing technol-ogy. An Omnidirectional Multi-camera System (OMS), based on a commercially available 6-camera system, was adapted for underwater use (Fig. 2). A custom-made compact housing inspired the development of a new 3-stage calibration procedure due to its non-conventional geometry [6]. This camera system was used at sea with the Girona500 AUV (Fig. 3) for tracking a formation of AUVs navigating at both close distance to each other and to the benthos [4], and also Figure 1 - Concept of the OMNIUS acquisition system. A tethered HROV (a) performs data acquisition using an omnidirectional camera (b) that provide the scientists with 360 degrees of visual feedback (c). This feedback is used to control and drive the acquisition76 instrumentation viewpOint- 19 - MARTECH 16Figure 2- Omnidirectional sensor based on a 6-camera system. A custom housing was designed and built, comprising a cylindrical viewport with a hemispherical cap, and a metal container for a small form factor pc.Figure 3 - Side view of the Girona 500 AUV/HROV with the omnidirectional camera installed.to create virtual panoramic tours for public outreach, in a similar concept to Google StreetView [5,3] (Figs. 4 and 5).An important aspect in exploratory missions is the ability by the scientists to adequately perceive the visual information from the surroundings, since they have to decide on-the-fly where to navigate next. As such it become important to improve the live imagery in real-time, to counteract part of the image deg-radation that the underwater medium creates. Towards this goal a new method for color correction of underwater images was proposed [7], using the Lab color space and having a low computational cost suitable for on-line operation. Another key aspect is the ability to build optical maps during mission. Several steps have been taken to perform optical mapping using computationally ef-ficient approaches, towards the goal of on-line operation [8]. The case of multi-camera mapping has been addressed in the context of habitat mapping in the Azores [9], along with the problem of creating high quality 3D surfaces from noisy point data [2]. An important operational concern relates with the quality of the navigation data that is used in the mapping process. While altitude, depth and heading can be assumed to be highly reliable in most conditions, the same is not necessary true for latitude and longitude, which depend on the quality of inertial systems and/or on the presence of an acoustic positioning infrastructure. In this sense effort was devoted for the creation of optical maps when only the image mea-surements are assumed to be reliable (i.e. in absence of navigation data). These methods make use of graph theory to devise efficient approaches for image overlap prediction and joint optimization of the image registration, on large im-ages sets [10,11,12] III. FUTURE DIRECTIONSThe technologies developed within the OMNIUS paradigm aim at the study of the benthos immersively and  in real time, and at the remote interaction with the vehicle in a safe and intuitive manner. Within this paradigm, several future directions can be identified as be-ing of high potential impact for marine science. Two new integrated capabilities are pointed out due to their high operational relevance. The first one is the ability to identify benthic structures of interest automatically and during mission time. An example of such is the detection of a particular invasive species, such as the Caulerpa webbinana in the Azores, or small man-made objects such as marine munitions or accident debris. Such detection is particularly well suited for the large field of view of the omnidirectional camera, and will serve to complement (and improve upon) the visual detection conduct-ed by the scientist piloting the vehicle. The second is the automated detection and characterization of temporal changes, for areas that are visited repeatedly over time. This ability is central to a number of ecology studies, and traditionally involves extensive manual work. The automated change detection and interpretation underwater requires tech-niques, such as pre-processing, registration and classification, that have not yet reached the required maturity and performance level. Figure 4 - Location and sample image of a shipwreck virtual tour in Google Maps 77instrumentation viewpOint- 19 - MARTECH 16 instrumentation viewpOint - 19 - MARTECH 16REFERENCES[1] D.  Ribas,  N.  Palomeras,  P.  Ridao,  M.  Carreras,  and  A.  Mallios,  Girona  500  AUV: From Survey to Intervention, Mechatronics, IEEE/ASME Transactions on , vol. 17, no. 1, pp. 46--53, Feb 2012 [2] R. Campos, R. Garcia, P. Alliez, M. Yvinec. "A Surface Reconstruction Method for In-Detail Underwater 3D Optical Mapping". The International Journal of Robotics Research, vol. 34 no. 1 64-89, January 2015. eISSN: 1741-3176, ISSN: 0278-3649. [3] J. Bosch, P. Ridao, R. Garcia, N. Gracias, Towards omnidirectional immersion for ROV teleoperation, In Proc. Jornadas de Automática, Madrid, Spain, September 2016[4] J. Bosch, N. Gracias, P. Ridao, K. Istenič,  D. Ribas. Close-Range Tracking of Un-derwater Vehicles Using Light Beacons. Sensors. 2016; 16(4), 429; DOI: 10.3390/s16040429[5] J. Bosch, P. Ridao, D. Ribas, N. Gracias. Creating 360◦ Underwater Virtual Tours Using an Omnidirectional Camera Integrated in an AUV.  Proceedings of the Oceans MTS/IEEE OCEANS conference, Genova, Italy, May 2015[6] J. Bosch, N. Gracias, P. Ridao, D. Ribas. Omnidirectional Underwater Camera De-sign and Calibration. Sensors. 2015; 15(3):6033-6065[7] G. Bianco, M. Muzzupappa, F. Bruno, R. Garcia, L. Neumann. "A new color cor-rection method for underwater imaging". ISPRS/CIPA Workshop on Underwater 3D Recording and Modelling, Piano di Sorrento (Napoli), Italy. 16-17 April 2015. [8] T. Nicosevici, R. Prados and R. Garcia. "3D Data Processing for Underwater Inspec-tion and Monitoring". Instrumentation Viewpoint, pp. 72-73, 2015. ISSN: 1886-4864. [9] R. Campos, N. Gracias, P. Ridao. Underwater Multi-Vehicle Trajectory Alignment and Mapping using Acoustic and Optical Constraints. Sensors 2016, 16(3), 387; DOI: 10.3390/s16030387[10] A. Elibol, N. Gracias, R. Garcia, J. Kim. "Graph Theory Approach for Match Reduc-tion in Image Mosaicing", Journal of the Optical Society of America A. vol. 31, no. 4, pp. 773-782. ISSN: 1084-7529. DOI: 10.1364/JOSAA.31.000773[11] A. Elibol, J. Kim, N. Gracias, R. Garcia. "Towards automatic identification of mis-matched image pairs through loop constraints", in Proceedings of SPIE Vol. 9025, 2014. doi:10.1117/12.2040778[12] A. Elibol, J. Kim, N. Gracias, R. Garcia. "Efficient Image Mosaicing for Multi-robot Visual Underwater Mapping", Pattern Recognition Letters. Vol. 46, pp. 20–26, 2014. DOI: 10.1016/j.patrec.2014.04.020Figure 5 – Panoramic image using the equirectangular projection of a shipwreck survey in Horta, Azores Islands in September 2014.

https://upcommons.upc.edu/bitstream/2117/100101/1/ID33.pdf

Omnidirectional underwater surveying and telepresence

Abstract

Similar works

Full text

Available Versions

Revistes Catalanes amb Accés Obert

UPCommons

UPCommons. Portal del coneixement obert de la UPC