Towards Sustainable Oceans: Deep Learning Models for Accurate COTS Detection in Underwater Images

Object detection is one of the main tasks in computer vision, which includes image classification and localization. The application of object detection is now widespread as it powers various applications such as self-driving cars, robotics, biometrics, surveillance, satellite image analysis, and in healthcare, to mention just a few. Deep learning has taken computer vision to a different horizon. One of the areas that will benefit immensely from deep learning computer vision is the detection of killer starfish, the crown-of-thorns starfish (COTS). For decades, this killer starfish has dealt a big blow to the Great Barrier Reef in Australia, the world’s largest system of reefs, and in other places too. In addition to impacting negatively environmentally, it affects revenue generation from reef tourism. Hence, reef managers and authorities want to control the populations of crown-of-thorns starfish, which have been observed to be the culprits. The deep learning technique offers real-time and robust detection of this creature more than earlier traditional methods that were used to detect these creatures. This thesis work is part of a competition for a deep learning approach to detect COTS in real time by building an object detector trained using underwater images. This offers a solution to control the outbreaks in the population of these animals. Deep learning methods of Artificial Intelligence (AI) have gained popularity today because of its speed and high accuracy in detection and have performed better than the earlier traditional methods. They can be used in real-time object detection, and they owe their speed to convolutional neural networks (CNN). The thesis gives a comprehensive literature review of the journey so far in the field of computer vision and how deep learning methods can be applied to detect COTS. It also outlines the steps involved in the implementation of the model using the state-of-the-art computer vision algorithm known for its speed and accuracy – YOLOv8. The COTS detection model was trained using the custom dataset provided by the organizers of the competition, harnessing the powers of deep learning methods such as transfer learning, data augmentation, and preprocessing of underwater images to achieve high accuracy. Evaluation of the results obtained from the training showed a mean average precision of 0.803mAP at IoU of 0.5-0.95, acknowledging the detector model’s versatility in making accurate detection at different confidence levels. This supports the hypothesis that when we use pre trained model, this enhances the performance of our model for better object detection tasks. Certainly, better detection accuracy is one way to detect killer starfish, the crown-of-thorns starfish (COTS), and help protect the oceans

Deep learning based deep-sea automatic image enhancement and animal species classification

The automatic classification of marine species based on images is a challenging task for which multiple solutions have been increasingly provided in the past two decades. Oceans are complex ecosystems, difficult to access, and often the images obtained are of low quality. In such cases, animal classification becomes tedious. Therefore, it is often necessary to apply enhancement or pre-processing techniques to the images, before applying classification algorithms. In this work, we propose an image enhancement and classification pipeline that allows automated processing of images from benthic moving platforms. Deep-sea (870 m depth) fauna was targeted in footage taken by the crawler “Wally” (an Internet Operated Vehicle), within the Ocean Network Canada (ONC) area of Barkley Canyon (Vancouver, BC; Canada). The image enhancement process consists mainly of a convolutional residual network, capable of generating enhanced images from a set of raw images. The images generated by the trained convolutional residual network obtained high values in metrics for underwater imagery assessment such as UIQM (~ 2.585) and UCIQE (2.406). The highest SSIM and PSNR values were also obtained when compared to the original dataset. The entire process has shown good classification results on an independent test data set, with an accuracy value of 66.44% and an Area Under the ROC Curve (AUROC) value of 82.91%, which were subsequently improved to 79.44% and 88.64% for accuracy and AUROC respectively. These results obtained with the enhanced images are quite promising and superior to those obtained with the non-enhanced datasets, paving the strategy for the on-board real-time processing of crawler imaging, and outperforming those published in previous papers.This work was developed at Deusto Seidor S.A. (01015, Vitoria-Gasteiz, Spain) within the framework of the Tecnoterra (ICM-CSIC/UPC) and the following project activities: ARIM (Autonomous Robotic sea-floor Infrastructure for benthopelagic Monitoring); MarTERA ERA-Net Cofund; Centro para el Desarrollo Tecnológico Industrial, CDTI; and RESBIO (TEC2017-87861-R; Ministerio de Ciencia, Innovación y Universidades). This work was supported by the Centro para el Desarrollo Tecnológico Industrial (CDTI) (Grant No. EXP 00108707 / SERA-20181020)

DeepSeaNet: Improving Underwater Object Detection using EfficientDet

Marine animals and deep underwater objects are difficult to recognize and monitor for safety of aquatic life. There is an increasing challenge when the water is saline with granular particles and impurities. In such natural adversarial environment, traditional approaches like CNN start to fail and are expensive to compute. This project involves implementing and evaluating various object detection models, including EfficientDet, YOLOv5, YOLOv8, and Detectron2, on an existing annotated underwater dataset, called the Brackish-Dataset. The dataset comprises annotated image sequences of fish, crabs, starfish, and other aquatic animals captured in Limfjorden water with limited visibility. The aim of this research project is to study the efficiency of newer models on the same dataset and contrast them with the previous results based on accuracy and inference time. Firstly, I compare the results of YOLOv3 (31.10% mean Average Precision (mAP)), YOLOv4 (83.72% mAP), YOLOv5 (97.6%), YOLOv8 (98.20%), EfficientDet (98.56% mAP) and Detectron2 (95.20% mAP) on the same dataset. Secondly, I provide a modified BiSkFPN mechanism (BiFPN neck with skip connections) to perform complex feature fusion in adversarial noise which makes modified EfficientDet robust to perturbations. Third, analyzed the effect on accuracy of EfficientDet (98.63% mAP) and YOLOv5 by adversarial learning (98.04% mAP). Last, I provide class activation map based explanations (CAM) for the two models to promote Explainability in black box models. Overall, the results indicate that modified EfficientDet achieved higher accuracy with five-fold cross validation than the other models with 88.54% IoU of feature maps

Deep learning with self-supervision and uncertainty regularization to count fish in underwater images

Effective conservation actions require effective population monitoring. However, accurately counting animals in the wild to inform conservation decision-making is difficult. Monitoring populations through image sampling has made data collection cheaper, wide-reaching and less intrusive but created a need to process and analyse this data efficiently. Counting animals from such data is challenging, particularly when densely packed in noisy images. Attempting this manually is slow and expensive, while traditional computer vision methods are limited in their generalisability. Deep learning is the state-of-the-art method for many computer vision tasks, but it has yet to be properly explored to count animals. To this end, we employ deep learning, with a density-based regression approach, to count fish in low-resolution sonar images. We introduce a large dataset of sonar videos, deployed to record wild Lebranche mullet schools (Mugil liza), with a subset of 500 labelled images. We utilise abundant unlabelled data in a self-supervised task to improve the supervised counting task. For the first time in this context, by introducing uncertainty quantification, we improve model training and provide an accompanying measure of prediction uncertainty for more informed biological decision-making. Finally, we demonstrate the generalisability of our proposed counting framework through testing it on a recent benchmark dataset of high-resolution annotated underwater images from varying habitats (DeepFish). From experiments on both contrasting datasets, we demonstrate our network outperforms the few other deep learning models implemented for solving this task. By providing an open-source framework along with training data, our study puts forth an efficient deep learning template for crowd counting aquatic animals thereby contributing effective methods to assess natural populations from the ever-increasing visual data

Whale Detection Enhancement through Synthetic Satellite Images

With a number of marine populations in rapid decline, collecting and analyzing data about marine populations has become increasingly important to develop effective conservation policies for a wide range of marine animals, including whales. Modern computer vision algorithms allow us to detect whales in images in a wide range of domains, further speeding up and enhancing the monitoring process. However, these algorithms heavily rely on large training datasets, which are challenging and time-consuming to collect particularly in marine or aquatic environments. Recent advances in AI however have made it possible to synthetically create datasets for training machine learning algorithms, thus enabling new solutions that were not possible before. In this work, we present a solution - SeaDroneSim2 benchmark suite, which addresses this challenge by generating aerial, and satellite synthetic image datasets to improve the detection of whales and reduce the effort required for training data collection. We show that we can achieve a 15% performance boost on whale detection compared to using the real data alone for training, by augmenting a 10% real data. We open source both the code of the simulation platform SeaDroneSim2 and the dataset generated through it

Detectability of dolphins and turtles from Unoccupied Aerial Vehicle (UAV) survey imagery

For many decades occupied aircraft with trained observers have conducted aerial surveys of marine megafauna to estimate population size and dynamics. Recent technological advances mean that unoccupied aerial vehicles (UAVs) now provide a potential alternative to occupied surveys, eliminating some of the disadvantages of occupied surveys such as risk to human life, weather constraints and cost. In this study, data collected from an occupied aircraft (at 500 ft) and a UAV (at 1400 ft) flown at the same time, deployed for counting dugongs, were compared for detecting dolphins and turtles within Shark Bay, Western Australia. The UAV images were manually reviewed post hoc to count the animals sighted and the environmental conditions (visibility, sea state, cloud cover and glare) had been classified by the occupied teams’ data for each image. The UAV captured more sightings (174 dolphins and 368 turtles) than were recorded by the flight team (93 dolphins and 312 turtles). Larger aggregations (>10 animals) were also found in the UAV images (5 aggregations of dolphins and turtles) compared to the occupied teams sightings (0 dolphins and 3 aggregations of turtles). A generalised linear mixed model determined that turtle detection was significantly affected by visibility, while cloud cover, sea state and visibility significantly affected dolphin detection in both platforms. An expert survey of 120 images was also conducted to determine the image ground sampling distance (GSD; four levels from 1.7 to 3.5 cm/pixel) needed to identify dolphin and turtles to species. At 3 cm/pixel only 40% of the dolphins and turtles were identified to species with a reasonable level of certainty (>75% certainty). This study demonstrated that UAVs can be successfully deployed for detecting dolphins and turtles and that a GSD of 1.7 – 3cm/pixel is too low resolution to effectively identify dolphin and turtle species. Overcoming the limitations imposed on UAVs such as aviator regulatory bodies and payload capabilities will make UAVs a pivotal tool for future research, conservation, and management

A Gated Cross-domain Collaborative Network for Underwater Object Detection

Underwater object detection (UOD) plays a significant role in aquaculture and marine environmental protection. Considering the challenges posed by low contrast and low-light conditions in underwater environments, several underwater image enhancement (UIE) methods have been proposed to improve the quality of underwater images. However, only using the enhanced images does not improve the performance of UOD, since it may unavoidably remove or alter critical patterns and details of underwater objects. In contrast, we believe that exploring the complementary information from the two domains is beneficial for UOD. The raw image preserves the natural characteristics of the scene and texture information of the objects, while the enhanced image improves the visibility of underwater objects. Based on this perspective, we propose a Gated Cross-domain Collaborative Network (GCC-Net) to address the challenges of poor visibility and low contrast in underwater environments, which comprises three dedicated components. Firstly, a real-time UIE method is employed to generate enhanced images, which can improve the visibility of objects in low-contrast areas. Secondly, a cross-domain feature interaction module is introduced to facilitate the interaction and mine complementary information between raw and enhanced image features. Thirdly, to prevent the contamination of unreliable generated results, a gated feature fusion module is proposed to adaptively control the fusion ratio of cross-domain information. Our method presents a new UOD paradigm from the perspective of cross-domain information interaction and fusion. Experimental results demonstrate that the proposed GCC-Net achieves state-of-the-art performance on four underwater datasets