Unlocking the potential of deep learning for marine ecology: overview, applications, and outlook

publishedVersio

Unlocking the potential of deep learning for marine ecology: overview, applications, and outlook

The deep learning (DL) revolution is touching all scientific disciplines and corners of our lives as a means of harnessing the power of big data. Marine ecology is no exception. New methods provide analysis of data from sensors, cameras, and acoustic recorders, even in real time, in ways that are reproducible and rapid. Off-the-shelf algorithms find, count, and classify species from digital images or video and detect cryptic patterns in noisy data. These endeavours require collaboration across ecological and data science disciplines, which can be challenging to initiate. To promote the use of DL towards ecosystem-based management of the sea, this paper aims to bridge the gap between marine ecologists and computer scientists. We provide insight into popular DL approaches for ecological data analysis, focusing on supervised learning techniques with deep neural networks, and illustrate challenges and opportunities through established and emerging applications of DL to marine ecology. We present case studies on plankton, fish, marine mammals, pollution, and nutrient cycling that involve object detection, classification, tracking, and segmentation of visualized data. We conclude with a broad outlook of the field’s opportunities and challenges, including potential technological advances and issues with managing complex data sets.publishedVersionPaid Open Acces

Sonar image interpretation for sub-sea operations

Mine Counter-Measure (MCM) missions are conducted to neutralise underwater explosives. Automatic Target Recognition (ATR) assists operators by increasing the speed and accuracy of data review. ATR embedded on vehicles enables adaptive missions which increase the speed of data acquisition. This thesis addresses three challenges; the speed of data processing, robustness of ATR to environmental conditions and the large quantities of data required to train an algorithm. The main contribution of this thesis is a novel ATR algorithm. The algorithm uses features derived from the projection of 3D boxes to produce a set of 2D templates. The template responses are independent of grazing angle, range and target orientation. Integer skewed integral images, are derived to accelerate the calculation of the template responses. The algorithm is compared to the Haar cascade algorithm. For a single model of sonar and cylindrical targets the algorithm reduces the Probability of False Alarm (PFA) by 80% at a Probability of Detection (PD) of 85%. The algorithm is trained on target data from another model of sonar. The PD is only 6% lower even though no representative target data was used for training. The second major contribution is an adaptive ATR algorithm that uses local sea-floor characteristics to address the problem of ATR robustness with respect to the local environment. A dual-tree wavelet decomposition of the sea-floor and an Markov Random Field (MRF) based graph-cut algorithm is used to segment the terrain. A Neural Network (NN) is then trained to filter ATR results based on the local sea-floor context. It is shown, for the Haar Cascade algorithm, that the PFA can be reduced by 70% at a PD of 85%. Speed of data processing is addressed using novel pre-processing techniques. The standard three class MRF, for sonar image segmentation, is formulated using graph-cuts. Consequently, a 1.2 million pixel image is segmented in 1.2 seconds. Additionally, local estimation of class models is introduced to remove range dependent segmentation quality. Finally, an A* graph search is developed to remove the surface return, a line of saturated pixels often detected as false alarms by ATR. The A* search identifies the surface return in 199 of 220 images tested with a runtime of 2.1 seconds. The algorithm is robust to the presence of ripples and rocks

SEABED MORPHOLOGICAL PREDICTION WITH APPLICATION TO MOBILITY AND BURIAL OF MUNITIONS

Hundreds of sites in littoral and inland waters across the United States have been reported as possibly containing underwater munitions, a safety hazard for the general public. Therefore, it is critical to determine and predict munition location and depth to implement remediation strategies. The mobility of munitions may be influenced by currents, waves, and seafloor slope. Burial depth may be affected by scour, sediment accretion, wave-induced liquefaction, and bedform migration. Here, environmental conditions and morphological evolution are investigated by examining observational data and modeling analyses to better understand the physical processes influencing the burial and mobility of munitions. The environmental models (Delft3D) are validated using observations from field experiments. A coupled Delft3D-object model capable of predicting the mobility and burial of objects on a sandy seafloor is presented. Although the object model limitation considers only cylindrical objects on flat seafloor, ignoring pitch and yaw movements, the coupled Delft3D-object model predictions agree well with observations. In addition, processes such as sediment transport, wave-induced liquefaction, and sand wave migration are examined. The findings show that environmental conditions from Delft3D can be used as a forcing term by other models (e.g., object and wave-induced liquefaction models), which is a valuable tool for predicting the fate of munitions.SERDPCapitao-de-Corveta, Brazilian NavyApproved for public release. Distribution is unlimited

The Influence of Data Resolution on Predicted Distribution and Estimates of Extent of Current Protection of Three 'Listed' Deep-Sea Habitats

Modelling approaches have the potential to significantly contribute to the spatial management of the deep-sea ecosystem in a cost effective manner. However, we currently have little understanding of the accuracy of such models, developed using limited data, of varying resolution. The aim of this study was to investigate the performance of predictive models constructed using non-simulated (real world) data of different resolution. Predicted distribution maps for three deep-sea habitats were constructed using MaxEnt modelling methods using high resolution multibeam bathymetric data and associated terrain derived variables as predictors. Model performance was evaluated using repeated 75/25 training/test data partitions using AUC and threshold-dependent assessment methods. The overall extent and distribution of each habitat, and the percentage contained within an existing MPA network were quantified and compared to results from low resolution GEBCO models. Predicted spatial extent for scleractinian coral reef and Syringammina fragilissima aggregations decreased with an increase in model resolution, whereas Pheronema carpenteri total suitable area increased. Distinct differences in predicted habitat distribution were observed for all three habitats. Estimates of habitat extent contained within the MPA network all increased when modelled at fine scale. High resolution models performed better than low resolution models according to threshold-dependent evaluation. We recommend the use of high resolution multibeam bathymetry data over low resolution bathymetry data for use in modelling approaches. We do not recommend the use of predictive models to produce absolute values of habitat extent, but likely areas of suitable habitat. Assessments of MPA network effectiveness based on calculations of percentage area protection (policy driven conservation targets) from low resolution models are likely to be fit for purpose

Mapping of complex marine environments using an unmanned surface craft

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2011.Cataloged from PDF version of thesis.Includes bibliographical references (p. 185-199).Recent technology has combined accurate GPS localization with mapping to build 3D maps in a diverse range of terrestrial environments, but the mapping of marine environments lags behind. This is particularly true in shallow water and coastal areas with man-made structures such as bridges, piers, and marinas, which can pose formidable challenges to autonomous underwater vehicle (AUV) operations. In this thesis, we propose a new approach for mapping shallow water marine environments, combining data from both above and below the water in a robust probabilistic state estimation framework. The ability to rapidly acquire detailed maps of these environments would have many applications, including surveillance, environmental monitoring, forensic search, and disaster recovery. Whereas most recent AUV mapping research has been limited to open waters, far from man-made surface structures, in our work we focus on complex shallow water environments, such as rivers and harbors, where man-made structures block GPS signals and pose hazards to navigation. Our goal is to enable an autonomous surface craft to combine data from the heterogeneous environments above and below the water surface - as if the water were drained, and we had a complete integrated model of the marine environment, with full visibility. To tackle this problem, we propose a new framework for 3D SLAM in marine environments that combines data obtained concurrently from above and below the water in a robust probabilistic state estimation framework. Our work makes systems, algorithmic, and experimental contributions in perceptual robotics for the marine environment. We have created a novel Autonomous Surface Vehicle (ASV), equipped with substantial onboard computation and an extensive sensor suite that includes three SICK lidars, a Blueview MB2250 imaging sonar, a Doppler Velocity Log, and an integrated global positioning system/inertial measurement unit (GPS/IMU) device. The data from these sensors is processed in a hybrid metric/topological SLAM state estimation framework. A key challenge to mapping is extracting effective constraints from 3D lidar data despite GPS loss and reacquisition. This was achieved by developing a GPS trust engine that uses a semi-supervised learning classifier to ascertain the validity of GPS information for different segments of the vehicle trajectory. This eliminates the troublesome effects of multipath on the vehicle trajectory estimate, and provides cues for submap decomposition. Localization from lidar point clouds is performed using octrees combined with Iterative Closest Point (ICP) matching, which provides constraints between submaps both within and across different mapping sessions. Submap positions are optimized via least squares optimization of the graph of constraints, to achieve global alignment. The global vehicle trajectory is used for subsea sonar bathymetric map generation and for mesh reconstruction from lidar data for 3D visualization of above-water structures. We present experimental results in the vicinity of several structures spanning or along the Charles River between Boston and Cambridge, MA. The Harvard and Longfellow Bridges, three sailing pavilions and a yacht club provide structures of interest, having both extensive superstructure and subsurface foundations. To quantitatively assess the mapping error, we compare against a georeferenced model of the Harvard Bridge using blueprints from the Library of Congress. Our results demonstrate the potential of this new approach to achieve robust and efficient model capture for complex shallow-water marine environments. Future work aims to incorporate autonomy for path planning of a region of interest while performing collision avoidance to enable fully autonomous surveys that achieve full sensor coverage of a complete marine environment.by Jacques Chadwick Leedekerken.Ph.D