LIRA: Lifelong Image Restoration from Unknown Blended Distortions

Most existing image restoration networks are designed in a disposable way and catastrophically forget previously learned distortions when trained on a new distortion removal task. To alleviate this problem, we raise the novel lifelong image restoration problem for blended distortions. We first design a base fork-join model in which multiple pre-trained expert models specializing in individual distortion removal task work cooperatively and adaptively to handle blended distortions. When the input is degraded by a new distortion, inspired by adult neurogenesis in human memory system, we develop a neural growing strategy where the previously trained model can incorporate a new expert branch and continually accumulate new knowledge without interfering with learned knowledge. Experimental results show that the proposed approach can not only achieve state-of-the-art performance on blended distortions removal tasks in both PSNR/SSIM metrics, but also maintain old expertise while learning new restoration tasks.Comment: ECCV2020 accepte

Learning Disentangled Feature Representation for Hybrid-distorted Image Restoration

Hybrid-distorted image restoration (HD-IR) is dedicated to restore real distorted image that is degraded by multiple distortions. Existing HD-IR approaches usually ignore the inherent interference among hybrid distortions which compromises the restoration performance. To decompose such interference, we introduce the concept of Disentangled Feature Learning to achieve the feature-level divide-and-conquer of hybrid distortions. Specifically, we propose the feature disentanglement module (FDM) to distribute feature representations of different distortions into different channels by revising gain-control-based normalization. We also propose a feature aggregation module (FAM) with channel-wise attention to adaptively filter out the distortion representations and aggregate useful content information from different channels for the construction of raw image. The effectiveness of the proposed scheme is verified by visualizing the correlation matrix of features and channel responses of different distortions. Extensive experimental results also prove superior performance of our approach compared with the latest HD-IR schemes.Comment: Accepted by ECCV202

Development of a fusion adaptive algorithm for marine debris detection within the post-Sandy restoration framework

Recognition of marine debris represent a difficult task due to the extreme variability of the marine environment, the possible targets, and the variable skill levels of human operators. The range of potential targets is much wider than similar fields of research such as mine hunting, localization of unexploded ordnance or pipeline detection. In order to address this additional complexity, an adaptive algorithm is being developing that appropriately responds to changes in the environment, and context. The preliminary step is to properly geometrically and radiometrically correct the collected data. Then, the core engine manages the fusion of a set of statistically- and physically-based algorithms, working at different levels (swath, beam, snippet, and pixel) and using both predictive modeling (that is, a high-frequency acoustic backscatter model) and phenomenological (e.g., digital image processing techniques) approaches. The expected outcome is the reduction of inter-algorithmic cross-correlation and, thus, the probability of false alarm. At this early stage, we provide a proof of concept showing outcomes from algorithms that dynamically adapt themselves to the depth and average backscatter level met in the surveyed environment, targeting marine debris (modeled as objects of about 1-m size). The project relies on a modular software library, called Matador (Marine Target Detection and Object Recognition)

Image sequence restoration by median filtering

Median filters are non-linear filters that fit in the generic category of order-statistic filters. Median filters are widely used for reducing random defects, commonly characterized by impulse or salt and pepper noise in a single image. Motion estimation is the process of estimating the displacement vector between like pixels in the current frame and the reference frame. When dealing with a motion sequence, the motion vectors are the key for operating on corresponding pixels in several frames. This work explores the use of various motion estimation algorithms in combination with various median filter algorithms to provide noise suppression. The results are compared using two sets of metrics: performance-based and objective image quality-based. These results are used to determine the best motion estimation / median filter combination for image sequence restoration. The primary goals of this work are to implement a motion estimation and median filter algorithm in hardware and develop and benchmark a flexible software alternative restoration process. There are two unique median filter algorithms to this work. The first filter is a modification to a single frame adaptive median filter. The modification applied motion compensation and temporal concepts. The other is an adaptive extension to the multi-level (ML3D) filter, called adaptive multi-level (AML3D) filter. The extension provides adaptable filter window sizes to the multiple filter sets that comprise the ML3D filter. The adaptive median filter is capable of filtering an image in 26.88 seconds per frame and results in a PSNR improvement of 5.452dB. The AML3D is capable of filtering an image in 14.73 seconds per frame and results in a PSNR improvement of 6.273dB. The AML3D is a suitable alternative to the other median filters