Semantic segmentation based on Deep learning for the detection of Cyanobacterial Harmful Algal Blooms (CyanoHABs) using synthetic images

Cyanobacterial Harmful Algal Blooms (CyanoHABs) in lakes and reservoirs have increased substantially in recent decades due to different environmental factors. Its early detection is a crucial issue to minimize health effects, particularly in potential drinking and recreational water bodies. The use of Autonomous Surface Vehicles (ASVs) equipped with machine vision systems (cameras) onboard, represents a useful alternative at this time. In this regard, we propose an image Semantic Segmentation approach based on Deep Learning with Convolutional Neural Networks (CNNs) for the early detection of CyanoHABs considering an ASV perspective. The use of these models is justified by the fact that with their convolutional architecture, it is possible to capture both, spectral and textural information considering the context of a pixel and its neighbors. To train these models it is necessary to have data, but the acquisition of real images is a difficult task, due to the capricious appearance of the algae on water surfaces sporadically and intermittently over time and after long periods of time, requiring even years and the permanent installation of the image capture system. This justifies the generation of synthetic data so that sufficiently trained models are required to detect CyanoHABs patches when they emerge on the water surface. The data generation for training and the use of the semantic segmentation models to capture contextual information determine the need for the proposal, as well as its novelty and contribution. Three datasets of images containing CyanoHABs patches are generated: (a) the first contains real patches of CyanoHABs as foreground and images of lakes and reservoirs as background, but with a limited number of examples; (b) the second, contains synthetic patches of CyanoHABs generated with state-of-the-art Style-based Generative Adversarial Network Adaptive Discriminator Augmentation (StyleGAN2-ADA) and Neural Style Transfer as foreground and images of lakes and reservoirs as background, and (c) the third set, is the combination of the previous two. Four model architectures for semantic segmentation (UNet++, FPN, PSPNet, and DeepLabV3+), with two encoders as backbone (ResNet50 and EfficientNet-b6), are evaluated from each dataset on real test images and different distributions. The results show the feasibility of the approach and that the UNet++ model with EfficientNet-b6, trained on the third dataset, achieves good generalization and performance for the real test images.Depto. de Arquitectura de Computadores y AutomáticaFac. de InformáticaTRUEComunidad Autónoma de MadridSpanish Ministry of Science, Innovation and UniversitiesMinistry of Education of PeruSpanish Ministry of Universitiespu

Filter pruning for convolutional neural networks in semantic image segmentation

The remarkable performance of Convolutional Neural Networks (CNNs) has increased their use in real-time systems and devices with limited resources. Hence, compacting these networks while preserving accuracy has become necessary, leading to multiple compression methods. However, the majority require intensive iterative procedures and do not delve into the influence of the used data. To overcome these issues, this paper presents several contributions, framed in the context of explainable Artificial Intelligence (xAI): (a) two filter pruning methods for CNNs, which remove the less significant convolutional kernels; (b) a fine-tuning strategy to recover generalization; (c) a layer pruning approach for U-Net; and (d) an explanation of the relationship between performance and the used data. Filter and feature maps information are used in the pruning process: Principal Component Analysis (PCA) is combined with a next-convolution influence-metric, while the latter and the mean standard deviation are used in an importance score distribution-based method. The developed strategies are generic, and therefore applicable to different models. Experiments demonstrating their effectiveness are conducted over distinct CNNs and datasets, focusing mainly on semantic segmentation (using U-Net, DeepLabv3+, SegNet, and VGG-16 as highly representative models). Pruned U-Net on agricultural benchmarks achieves 98.7% parameters and 97.5% FLOPs drop, with a 0.35% gain in accuracy. DeepLabv3+ and SegNet on CamVid reach 46.5% and 72.4% parameters reduction and a 51.9% and 83.6% FLOPs drop respectively, with almost no decrease in accuracy. VGG-16 on CIFAR-10 obtains up to 86.5% parameter and 82.2% FLOPs decrease with a 0.78% accuracy gain.Sinérgicos Comunidad de MadridMinisterio de Ciencia, Innovación y Universidades de EspañaMinisterio de Universidades de EspañaMinisterio de Educación de PerúDepto. de Ingeniería de Software e Inteligencia Artificial (ISIA)Sección Deptal. de Arquitectura de Computadores y Automática (Físicas)Fac. de InformáticaFac. de Ciencias FísicasTRUEpu