Most existing image restoration networks are designed in a disposable way and
catastrophically forget previously learned distortions when trained on a new
distortion removal task. To alleviate this problem, we raise the novel lifelong
image restoration problem for blended distortions. We first design a base
fork-join model in which multiple pre-trained expert models specializing in
individual distortion removal task work cooperatively and adaptively to handle
blended distortions. When the input is degraded by a new distortion, inspired
by adult neurogenesis in human memory system, we develop a neural growing
strategy where the previously trained model can incorporate a new expert branch
and continually accumulate new knowledge without interfering with learned
knowledge. Experimental results show that the proposed approach can not only
achieve state-of-the-art performance on blended distortions removal tasks in
both PSNR/SSIM metrics, but also maintain old expertise while learning new
restoration tasks.Comment: ECCV2020 accepte