Image restoration is a low-level vision task, most CNN methods are designed
as a black box, lacking transparency and internal aesthetics. Although some
methods combining traditional optimization algorithms with DNNs have been
proposed, they all have some limitations. In this paper, we propose a
three-granularity memory layer and contrast learning named MemoryNet,
specifically, dividing the samples into positive, negative, and actual three
samples for contrastive learning, where the memory layer is able to preserve
the deep features of the image and the contrastive learning converges the
learned features to balance. Experiments on Derain/Deshadow/Deblur task
demonstrate that these methods are effective in improving restoration
performance. In addition, this paper's model obtains significant PSNR, SSIM
gain on three datasets with different degradation types, which is a strong
proof that the recovered images are perceptually realistic. The source code of
MemoryNet can be obtained from https://github.com/zhangbaijin/MemoryNe