Improved Residual Dense Network for Large Scale Super-Resolution via Generative Adversarial Network

Recent single image super resolution (SISR) studies were conducted extensively on small upscaling factors such as x2 and x4 on remote sensing images, while less work was conducted on large factors such as the factor x8 and x16. Owing to the high performance of the generative adversarial networks (GANs), in this paper, two GAN’s frameworks are implemented to study the SISR on the residual remote sensing image with large magnification under x8 scale factor, which is still lacking acceptable results. This work proposes a modified version of the residual dense network (RDN) and then it been implemented within GAN framework which named RDGAN. The second GAN framework has been built based on the densely sampled super resolution network (DSSR) and we named DSGAN. The used loss function for the training employs the adversarial, mean squared error (MSE) and the perceptual loss derived from the VGG19 model. We optimize the training by using Adam for number of epochs then switching to the SGD optimizer. We validate the frameworks on the proposed dataset of this work and other three remote sensing datasets: the UC Merced, WHU-RS19 and RSSCN7. To validate the frameworks, we use the following image quality assessment metrics: the PSNR and the SSIM on the RGB and the Y channel and the MSE. The RDGAN evaluation values on the proposed dataset were 26.02, 0.704, and 257.70 for PSNR, SSIM and the MSE, respectively, and the DSGAN evaluation on the same dataset yielded 26.13, 0.708 and 251.89 for the PSNR, the SSIM, and the MSE

Small-Object Detection in Remote Sensing Images with End-to-End Edge-Enhanced GAN and Object Detector Network

The detection performance of small objects in remote sensing images is not satisfactory compared to large objects, especially in low-resolution and noisy images. A generative adversarial network (GAN)-based model called enhanced super-resolution GAN (ESRGAN) shows remarkable image enhancement performance, but reconstructed images miss high-frequency edge information. Therefore, object detection performance degrades for small objects on recovered noisy and low-resolution remote sensing images. Inspired by the success of edge enhanced GAN (EEGAN) and ESRGAN, we apply a new edge-enhanced super-resolution GAN (EESRGAN) to improve the image quality of remote sensing images and use different detector networks in an end-to-end manner where detector loss is backpropagated into the EESRGAN to improve the detection performance. We propose an architecture with three components: ESRGAN, Edge Enhancement Network (EEN), and Detection network. We use residual-in-residual dense blocks (RRDB) for both the ESRGAN and EEN, and for the detector network, we use the faster region-based convolutional network (FRCNN) (two-stage detector) and single-shot multi-box detector (SSD) (one stage detector). Extensive experiments on a public (car overhead with context) and a self-assembled (oil and gas storage tank) satellite dataset show superior performance of our method compared to the standalone state-of-the-art object detectors.Comment: This paper contains 27 pages and accepted for publication in MDPI remote sensing journal. GitHub Repository: https://github.com/Jakaria08/EESRGAN (Implementation

Oil Palm USB (Unstripped Bunch) Detector Trained on Synthetic Images Generated by PGGAN

Identifying Unstriped Bunches (USB) is a pivotal challenge in palm oil production, contributing to reduced mill efficiency. Existing manual detection methods are proven time-consuming and prone to inaccuracies. Therefore, we propose an innovative solution harnessing computer vision technology. Specifically, we leverage the Faster R-CNN (Region-based Convolution Neural Network), a robust object detection algorithm, and complement it with Progressive Growing Generative Adversarial Networks (PGGAN) for synthetic image generation. Nevertheless, a scarcity of authentic USB images may hinder the application of Faster R-CNN. Herein, PGGAN is assumed to be pivotal in generating synthetic images of Empty Fruit Bunches (EFB) and USB. Our approach pairs synthetic images with authentic ones to train the Faster R-CNN. The VGG16 feature generator serves as the architectural backbone, fostering enhanced learning. According to our experimental results, USB detectors that were trained solely with authentic images resulted in an accuracy of 77.1%, which highlights the potential of this methodology. However, employing solely synthetic images leads to a slightly reduced accuracy of 75.3%. Strikingly, the fusion of authentic and synthetic images in a balanced ratio of 1:1 fuels a remarkable accuracy surge to 87.9%, signifying a 10.1% improvement. This innovative amalgamation underscores the potential of synthetic data augmentation in refining detection systems. By amalgamating authentic and synthetic data, we unlock a novel dimension of accuracy in USB detection, which was previously unattainable. This contribution holds significant implications for the industry, ensuring further exploration into advanced data synthesis techniques and refining detection models

Arbitrary Scale Super-Resolution Assisted Lunar Crater Detection in Satellite Images

Craters are one of the most studied planetary features used for different scientific analyses, such as estimation of surface age and surface processes. Satellite images utilized for crater detection often have low resolution (LR) due to hardware constraints and transmission time. Super-resolution (SR) is a practical and cost-effective solution; however, most SR approaches work on fixed integer scale factors, i.e., a single model can generate images of a specific resolution. In practical applications, SR on multiple scales provides various levels of detail, but training for each scale is resource-intensive. Therefore, this paper proposes a system for crater detection assisted with an arbitrary scale super-resolution approach (i.e., a single model can be used for multiple scale factors) for the lunar surface. Our work is composed of two subsystems. The first sub-system employs an arbitrary scale SR approach to generate super-resolved images of multiple resolutions. Subsequently, the second sub-system passes super-resolved images of multiple resolutions to a deep learning-based crater detection framework for identifying craters on the lunar surface. Employed arbitrary scale SR approach is based on a combination of convolution and transformer modules. For the crater detection sub-system, we utilize the Mask-RCNN framework. Using SR images of multiple resolutions, the proposed system detects 13.47% more craters from the ground truth than the craters detected using only the LR images. Further, in complex crater settings, specifically in overlapping and degraded craters, 11.84% and 15.01% more craters are detected as compared to the crater detection networks using only the LR images. The proposed system also leads to better localization performance, 3.19% IoU increment compared to the LR imagesComment: 15 pages, 8 figures, 8 Table