Two-Stage Overfitting of Neural Network-Based Video Coding In-Loop Filter

Modern video coding standards like the Versatile Video Coding (VVC) produce compression artefacts, due to their block-based, lossy compression techniques. These artefacts are mitigated to an extent by the in-loop filters inside the coding process. Neural Network (NN) based in-loop filters are being explored for the denoising tasks, and in recent studies, these NN-based loop filters are overfitted on test content to achieve a content-adaptive nature, and further enhance the visual quality of the video frames, while balancing the trade-off between quality and bitrate. This loop filter is a relatively low-complexity Convolutional Neural Network (CNN) that is pretrained on a general video dataset and then fine-tuned on the video that needs to be encoded. Only a set of parameters inside the CNN architecture, named multipliers, are fine-tuned, thus the bitrate overhead, that is signalled to the decoder, is minimized. The created weight update is compressed using the Neural Network Compression and Representation (NNR) standard. In this project, an exploration of high-performing hyperparameters was conducted, and the two-stage training process was employed to, potentially, further increase the coding efficiency of the in-loop filter. A first-stage model was overfitted on the test video sequence, it explored on which patches of the dataset it could improve the quality of the unfiltered video data, and then the second-stage model was overfitted only on these patches that provided a gain. The model with best-found hyperparameters saved on average 1.01% (Y), 4.28% (Cb), and 3.61% (Cr) Bjontegaard Delta rate (BD-rate) compared to the Versatile Video Coding (VVC) Test Model (VTM) 11.0 NN-based Video Coding (NNVC) 5.0, Random Access (RA) Common Test Conditions (CTC). The second-stage model, although exceeded the VTM, it underperformed with about 0.20% (Y), 0.23% (Cb), and 0.18% (Cr) BD-rate with regards to the first-stage model, due to the high bitrate overhead created by the second-stage model

Image restoration with group sparse representation and low‐rank group residual learning

Image restoration, as a fundamental research topic of image processing, is to reconstruct the original image from degraded signal using the prior knowledge of image. Group sparse representation (GSR) is powerful for image restoration; it however often leads to undesirable sparse solutions in practice. In order to improve the quality of image restoration based on GSR, the sparsity residual model expects the representation learned from degraded images to be as close as possible to the true representation. In this article, a group residual learning based on low-rank self-representation is proposed to automatically estimate the true group sparse representation. It makes full use of the relation among patches and explores the subgroup structures within the same group, which makes the sparse residual model have better interpretation furthermore, results in high-quality restored images. Extensive experimental results on two typical image restoration tasks (image denoising and deblocking) demonstrate that the proposed algorithm outperforms many other popular or state-of-the-art image restoration methods

Fully Trainable and Interpretable Non-Local Sparse Models for Image Restoration

Non-local self-similarity and sparsity principles have proven to be powerful priors for natural image modeling. We propose a novel differentiable relaxation of joint sparsity that exploits both principles and leads to a general framework for image restoration which is (1) trainable end to end, (2) fully interpretable, and (3) much more compact than competing deep learning architectures. We apply this approach to denoising, jpeg deblocking, and demosaicking, and show that, with as few as 100K parameters, its performance on several standard benchmarks is on par or better than state-of-the-art methods that may have an order of magnitude or more parameters.Comment: ECCV 202

LOCAL SPECTRAL COMPONENT DECOMPOSITION FOR WAVELET TRANSFORM ANALYSIS IN MULTI-CHANNEL IMAGE DENOISING

Our point is to join upheaval on multi-channel pictures by abuse the prompt relationship in the creepy field of a nearby area. To fundamental addition a straight segment more than the powerful bits of a M-channel plot, which we depict the strange line, and a brief timeframe later, utilizing the line, we ruin the picture into three portions: a particular M-channel picture and two dull scale pictures. By amazing part of the disintegrating, the bang is chosen the two pictures, and right now calculation necessities to denoise basically the two grayscale pictures, paying little cerebrum to the measure of the channels. There are different assessments for the confirmation of the boundaries, at any rate standard relationship better execution and improving picture quality. Since the wavelet change has remarkable execution, subsequently, it has been broadly applied as a sort of sign and picture preparing contraptions. Right now change is utilized in the picture de-noising and we suggest a standard relationship calculation which gives refreshed execution and Experimental outcomes show the authority of the new assessment