Repeatable semantic reef-mapping through photogrammetry and label-augmentation

In an endeavor to study natural systems at multiple spatial and taxonomic resolutions, there is an urgent need for automated, high-throughput frameworks that can handle plethora of information. The coalescence of remote-sensing, computer-vision, and deep-learning elicits a new era in ecological research. However, in complex systems, such as marine-benthic habitats, key ecological processes still remain enigmatic due to the lack of cross-scale automated approaches (mms to kms) for community structure analysis. We address this gap by working towards scalable and comprehensive photogrammetric surveys, tackling the profound challenges of full semantic segmentation and 3D grid definition. Full semantic segmentation (where every pixel is classified) is extremely labour-intensive and difficult to achieve using manual labeling. We propose using label-augmentation, i.e., propagation of sparse manual labels, to accelerate the task of full segmentation of photomosaics. Photomosaics are synthetic images generated from a projected point-of-view of a 3D model. In the lack of navigation sensors (e.g., a diver-held camera), it is difficult to repeatably determine the slope-angle of a 3D map. We show this is especially important in complex topographical settings, prevalent in coral-reefs. Specifically, we evaluate our approach on benthic habitats, in three different environments in the challenging underwater domain. Our approach for label-augmentation shows human-level accuracy in full segmentation of photomosaics using labeling as sparse as 0.1%, evaluated on several ecological measures. Moreover, we found that grid definition using a leveler improves the consistency in community-metrics obtained due to occlusions and topology (angle and distance between objects), and that we were able to standardise the 3D transformation with two percent error in size measurements. By significantly easing the annotation process for full segmentation and standardizing the 3D grid definition we present a semantic mapping methodology enabling change-detection, which is practical, swift, and cost-effective. Our workflow enables repeatable surveys without permanent markers and specialized mapping gear, useful for research and monitoring, and our code is available online. Additionally, we release the Benthos data-set, fully manually labeled photomosaics from three oceanic environments with over 4500 segmented objects useful for research in computer-vision and marine ecology

Hierarchical Classification of Scientific Taxonomies with Autonomous Underwater Vehicles

Autonomous Underwater Vehicles (AUVs) have catalysed a significant shift in the way marine habitats are studied. It is now possible to deploy an AUV from a ship, and capture tens of thousands of georeferenced images in a matter of hours. There is a growing body of research investigating ways to automatically apply semantic labels to this data, with two goals. The task of manually labelling a large number of images is time consuming and error prone. Further, there is the potential to change AUV surveys from being geographically defined (based on a pre-planned route), to permitting the AUV to adapt the mission plan in response to semantic observations. This thesis focusses on frameworks that permit a unified machine learning approach with applicability to a wide range of geographic areas, and diverse areas of interest for marine scientists. This can be addressed through the use of hierarchical classification; in which machine learning algorithms are trained to predict not just a binary or multi-class outcome, but a hierarchy of related output labels which are not mutually exclusive, such as a scientific taxonomy. In order to investigate classification on larger hierarchies with greater geographic diversity, the BENTHOZ-2015 data set was assembled as part of a collaboration between five Australian research groups. Existing labelled data was re-mapped to the CATAMI hierarchy, in total more than 400,000 point labels, conforming to a hierarchy of around 150 classes. The common hierarchical classification approach of building a network of binary classifiers was applied to the BENTHOZ-2015 data set, and a novel application of Bayesian Network theory and probability calibration was used as a theoretical foundation for the approach, resulting in improved classifier performance. This was extended to a more complex hidden node Bayesian Network structure, which permits inclusion of additional sensor modalities, and tuning for better performance in particular geographic regions

Autonomous marine environmental monitoring: Application in decommissioned oil fields

Hundreds of Oil & Gas Industry structures in the marine environment are approaching decommissioning. In most areas decommissioning operations will need to be supported by environmental assessment and monitoring, potentially over the life of any structures left in place. This requirement will have a considerable cost for industry and the public. Here we review approaches for the assessment of the primary operating environments associated with decommissioning — namely structures, pipelines, cuttings piles, the general seabed environment and the water column — and show that already available marine autonomous systems (MAS) offer a wide range of solutions for this major monitoring challenge. Data of direct relevance to decommissioning can be collected using acoustic, visual, and oceanographic sensors deployed on MAS. We suggest that there is considerable potential for both cost savings and a substantial improvement in the temporal and spatial resolution of environmental monitoring. We summarise the trade-offs between MAS and current conventional approaches to marine environmental monitoring. MAS have the potential to successfully carry out much of the monitoring associated with decommissioning and to offer viable alternatives where a direct match for the conventional approach is not possible

High-resolution underwater robotic vision-based mapping and three-dimensional reconstruction for archaeology

Documenting underwater archaeological sites is an extremely challenging problem. Sites covering large areas are particularly daunting for traditional techniques. In this paper, we present a novel approach to this problem using both an autonomous underwater vehicle (AUV) and a diver-controlled stereo imaging platform to document the submerged Bronze Age city at Pavlopetri, Greece. The result is a three-dimensional (3D) reconstruction covering 26,600 m2 at a resolution of 2 mm/pixel, the largest-scale underwater optical 3D map, at such a resolution, in the world to date. We discuss the advances necessary to achieve this result, including i) an approach to color correct large numbers of images at varying altitudes and over varying bottom types; ii) a large-scale bundle adjustment framework that is capable of handling upward of 400,000 stereo images; and iii) a novel approach to the registration and rapid documentation of an underwater excavations area that can quickly produce maps of site change. We present visual and quantitative comparisons to the authors' previous underwater mapping approaches

Photogrammetric surveys and geometric processes to analyse and monitor red coral colonies

This article describes the set of photogrammetric tools developed for the monitoring of Mediterranean red coral Corallium rubrum populations. The description encompasses the full processing chain: from the image acquisition to the information extraction and data interpretation. The methods applied take advantage of existing tools and new, innovative and specific developments in order to acquire data on relevant ecological information concerning the structure and functioning of a red coral population. The tools presented here are based on: (i) automatic orientation using coded quadrats; (ii) use of non-photorealistic rendering (NPR) and 3D skeletonization techniques; (iii) computation of distances between colonies from a same site; and (iv) the use of a plenoptic approach in an underwater environment. © 2018 by the authors.This work is partially done in the framework of the PERfECT project, funded by the Foundation TOTAL, project 2014/257. The plenoptic camera was bought in the frame of the DGA RAPID LORI project (LOcalisation et Reconnaissance d’objets Immergés

Caracterização da comunidade epibentônica em recifes de corais das ilhas Fiji por vídeo-imagem

TCC(graduação) - Universidade Federal de Santa Catarina. Centro de Ciências Biológicas. Biologia.A profundidade é um dos indicadores mais bem estabelecidos para o estudo da distribuição de comunidades bentônicas nos ecossistemas marinhos por estar diretamente relacionada com a zona fótica disponível. Todavia, a maioria dos estudos analisa ambientes recifais até 30 metros dado o limite do SCUBA. Consequentemente, sabemos pouco sobre estrutura de comunidades ao longo de um gradiente de profundidade entre os recifes rasos e mesofóticos. Os veículos automatizados são exemplos de tecnologias disponíveis para investigação de recifes em ambientes mais profundos. No entanto, são necessárias adequações das metodologias operacionais e amostrais para a coleta dos dados A alta quantidade de dados gerada precisa ser otimizada. Os dados podem ser integrados com resultados de outras pesquisas para ampliação do conhecimento dos processos ecológicos. Esquemas metodológicos de identificação de organismos bentônicos em imagens subaquáticas vêm sendo desenvolvidos a fim de poderem ser adaptados globalmente. O CATAMI (Collaborative and Automated Tools for Analysis of Marine Imagery) é um exemplo disso, que propõe um esquema com uma abordagem morfofuncional taxonômica hierárquica. Neste estudo utilizou-se veículos remotamente operados (ROV’s) e adotou-se a classificação hierárquica baseada no CATAMI. Modelos de distribuição de espécies foram utilizados para avaliar o efeito da profundidade na composição de comunidades bentônicas de 10 à 130 metros, em recifes de corais na área de Vatu-i-Ra, ilhas Fiji. Observou-se que a profundidade foi significantemente relacionada com a presença e abundância de três dos quatro grupos epibêntonicos investigados. A abundância de corais pétreos diminuiu com a profundidade, enquanto a abundância de corais negros, octocorais e macroalgas aumentou até os 50 metros, e então diminuiu significantemente nas profundidades subsequentes. Esponjas e ascídias foram relativamente abundantes (>30%) ao longo de toda profundidade investigada, assim como o grupo de macroalgas (>40%). Este estudo demonstra que imagens originadas de ROVs podem ser utilizadas para caracterizar a composição da comunidade epibentônica ao longo de uma ampla escala de profundidade, e assim contribui para nosso conhecimento sobre recifes de corais mesofóticos

Underwater Hyperspectral Imaging (UHI): a review of systems and applications for proximal seafloor ecosystem studies

Marine ecosystem monitoring requires observations of its attributes at different spatial and temporal scales that traditional sampling methods (e.g., RGB imaging, sediment cores) struggle to efficiently provide. Proximal optical sensing methods can fill this observational gap by providing observations of, and tracking changes in, the functional features of marine ecosystems non-invasively. Underwater hyperspectral imaging (UHI) employed in proximity to the seafloor has shown a further potential to monitor pigmentation in benthic and sympagic phototrophic organisms at small spatial scales (mm–cm) and for the identification of minerals and taxa through their finely resolved spectral signatures. Despite the increasing number of studies applying UHI, a review of its applications, capabilities, and challenges for seafloor ecosystem research is overdue. In this review, we first detail how the limited band availability inherent to standard underwater cameras has led to a data analysis “bottleneck” in seafloor ecosystem research, in part due to the widespread implementation of underwater imaging platforms (e.g., remotely operated vehicles, time-lapse stations, towed cameras) that can acquire large image datasets. We discuss how hyperspectral technology brings unique opportunities to address the known limitations of RGB cameras for surveying marine environments. The review concludes by comparing how different studies harness the capacities of hyperspectral imaging, the types of methods required to validate observations, and the current challenges for accurate and replicable UHI research

Coral Colony-Scale Rugosity Metrics and Applications for Assessing Temporal Trends in the Structural Complexity of Coral Reefs.

Globally, coral reefs are experiencing reductions in structural complexity, primarily due to a loss of key reef building taxa. Monitoring these changes is difficult due to the time-consuming nature of in-situ measurements and lack of data concerning coral genus-specific contributions to reef structure. This research aimed to develop a new technique that uses coral colony level data to quantify reef rugosity (a 3-dimensional measure of reef structure) from three sources of coral survey data: 2D video imagery, line intercept data and UAV imagery. A database of coral colony rugosity data, comparing coral colony planar and contour length for 40 coral genera, 14 morphotypes and 9 abiotic reef substrates, was created using measurements from the Great Barrier Reef and Natural History Museum. Mean genus rugosity was identified as a key trait related to coral life history strategy. Linear regression analyses (y = mx) revealed statistically significant (p < 0.05) relationships between coral colony size and rugosity for every coral genus, morphotype and substrate. The gradient governing these relationships was unique for each coral taxa, ranging from mean = 1.23, for (encrusting) Acanthastrea, to m = 3.84, for (vase-shape) Merulina. These gradients were used as conversion factors to calculate reef rugosity from linear distances measured in video transects of both artificial reefs, used as a control test, and in-situ natural coral reefs, using Kinovea software. This calculated, ‘virtual’ rugosity had a strong, positive relationship with in-situ microscale rugosity (r2 = 0.96) measured from the control transects, but not with that measured at the meso-scale in natural, highly heterogeneous reef environments (r2 < 0.2). This showed that the technique can provide accurate rugosity information when considered at the coral colony level. The conversion factors were also applied to historic line intercept data from the Seychelles, where temporal changes in calculated rugosity were consistent with changes in coral cover between 2008 and 2017. Finally, on application to 2,283 corals digitised from UAV imagery of the Maldives, the conversion factors enabled calculation of rugosity for three 100 m2 reef areas and prediction of how this rugosity will decrease during two future scenarios of coral reef degradation and community change. The study highlights that the application of genera-specific coral rugosity data to both new and existing coral reef survey datasets could be a valuable tool for monitoring reef structural complexity over large spatial scales

Low-Cost Vision Based Autonomous Underwater Vehicle for Abyssal Ocean Ecosystem Research

The oceans have a major impact on the planet: they store 28% of the CO 2 pro- duced by humans, they act as the world’s thermal damper for temperature changes, and more than 17, 000 species call the deep oceans their home. Scientific drivers, like climate change, and commercial applications, like deep sea fisheries and underwater mining, are pushing the need to know more about oceans at depths beyond 1000 meters. However, the high cost associated with autonomous underwater vehicles (AUVs) capable of operating beyond the depth of 1000 meters has limited the study of the deep ocean. Traditional AUVs used for deep-sea navigation are large and typically weigh up- wards of 1000-kgs, thus requiring careful planning before deployment and multi- person teams to operate. This thesis proposes the use of a new vehicle design based around a low-cost oceanographic glass sphere as the main pressure enclosure to reduce its size and cost while maintaining the ability for deep-sea operation. This novel housing concept, together with a minimal sensor suite, enables environmental research at depths previously inaccessible at this price point. The key characteristic that enables the cost reduction of this platform is the removal of the Doppler velocity log (DVL) sensor, which is replaced by optical cameras. Cameras allow the vehicle to estimate its motion in the water, but also enable scientific applications such as identification of habitat types or population density estimation of benthic species. After each survey, images can be further processed to produce full, dense 3D models of the survey area. While underwater optical cameras are frequently placed inside pressure housings behind flat or domed viewports and used for visual navigation or 3D reconstructions, the underlying assumptions for those algorithms do not hold in the underwater domain. Refraction at the housing viewport, together with wavelength-dependent attenuation of light in water, render the ubiquitous pinhole camera model invalid. This thesis presents a quantitative evaluation of the errors introduced by underwater effects for 3D reconstruction applications, comparing low- and high-cost camera systems to quantify the trade-off between equipment cost and performance. Although the distortion effects created by underwater refraction of light have been extensively studied for more traditional viewports, the novel design proposed necessitates new research into modeling the lensing effect of this off-axis domed viewport. A novel calibration method is presented that explicitly models the effect of the glass interface on image formation based on the characterization of optical distortions. The method is capable of accurately finding the position of the camera within the dome and further enables the use of deconvolution to improve the quality of the taken image. Finally, this thesis presents the validation of the designed vehicle for optical surveying tasks and introduces a end-to-end ocean mapping pipeline to streamline AUV deployments, enabling efficient use of time and resources.PHDNaval Architecture & Marine EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/155225/1/eiscar_1.pd

The Vezo communities and fisheries of the coral reef ecosystem in the Bay of Ranobe, Madagascar

Madagascar, a country whose extraordinary levels of endemism and biodiversity are celebrated globally by scientists and laymen alike, yet historically has received surprisingly little research attention, is the setting of the present dissertation. Here, I contribute to the need for applied research by: 1) focusing on the most intensely fished section of the Toliara Barrier Reef, the Bay of Ranobe; 2) characterizing the marine environment, the human population, and the fisheries; and 3) collecting the longest known time-series of data on fisheries of Madagascar, thereby providing a useful baseline for future analyses. In Chapter 1, the bathymetry of the Bay was characterized following a unique application of the boosted regression tree classifier to the RGB bands of IKONOS imagery. Derivation of water depths, based on DOS-corrected images, following a generic, log-transformed multiple linear regression approach produced a predictive accuracy of 1.28 m, whereas model fitting performed using the boosted regression tree classifier, allowing for interaction effects (tree complexity= 2), provided increased accuracy (RMSE= 1.01 m). Estimates of human population abundance, distribution, and dynamics were obtained following a dwelling-unit enumeration approach, using IKONOS Panchromatic and Google Earth images. Results indicated, in 2016, 31,850 people lived within 1 km of the shore, and 28,046 people lived within the 12 coastal villages of the Bay. Localized population growth rates within the villages, where birth rates and migration are combined, ranged from 2.96% - 6.83%, greatly exceeding official estimates of 2.78%. Annual pirogue counts demonstrated a shift in fishing effort from south to the north. Gear and boat (pirogue) profiles were developed, and the theoretical maximum number of fishermen predicted (n= 4,820), in 2013, from a regression model based on pirogue lengths (R2= 0.49). Spatial fishing effort distribution was mapped following a satellite-based enumeration of fishers-at-sea, resulting in a bay-wide estimate of intensity equaling 33.3 pirogue-meters km-2. Landings and CPUE were characterized, with respect to finfish, by family, species, gear, and village. Expansion of landings to bay-wide fisheries yields indicated 1,885.8 mt year-1 of mixed fisheries productivity, with an estimated wholesale value of 1.64 million USD per annum