Convolutional Neural Networks based Intra Prediction for HEVC

Traditional intra prediction methods for HEVC rely on using the nearest reference lines for predicting a block, which ignore much richer context between the current block and its neighboring blocks and therefore cause inaccurate prediction especially when weak spatial correlation exists between the current block and the reference lines. To overcome this problem, in this paper, an intra prediction convolutional neural network (IPCNN) is proposed for intra prediction, which exploits the rich context of the current block and therefore is capable of improving the accuracy of predicting the current block. Meanwhile, the predictions of the three nearest blocks can also be refined. To the best of our knowledge, this is the first paper that directly applies CNNs to intra prediction for HEVC. Experimental results validate the effectiveness of applying CNNs to intra prediction and achieved significant performance improvement compared to traditional intra prediction methods.Comment: 10 pages, This is the extended edition of poster paper accepted by DCC 201

Efficient Multiple Line-Based Intra Prediction for HEVC

Traditional intra prediction usually utilizes the nearest reference line to generate the predicted block when considering strong spatial correlation. However, this kind of single line-based method does not always work well due to at least two issues. One is the incoherence caused by the signal noise or the texture of other object, where this texture deviates from the inherent texture of the current block. The other reason is that the nearest reference line usually has worse reconstruction quality in block-based video coding. Due to these two issues, this paper proposes an efficient multiple line-based intra prediction scheme to improve coding efficiency. Besides the nearest reference line, further reference lines are also utilized. The further reference lines with relatively higher quality can provide potential better prediction. At the same time, the residue compensation is introduced to calibrate the prediction of boundary regions in a block when we utilize further reference lines. To speed up the encoding process, this paper designs several fast algorithms. Experimental results show that, compared with HM-16.9, the proposed fast search method achieves 2.0% bit saving on average and up to 3.7%, with increasing the encoding time by 112%.Comment: Accepted for publication in IEEE Transactions on Circuits and Systems for Video Technolog

Inpainting-based Video Compression in FullHD

Compression methods based on inpainting are an evolving alternative to classical transform-based codecs for still images. Attempts to apply these ideas to video compression are rare, since reaching real-time performance is very challenging. Therefore, current approaches focus on simplified frame-by-frame reconstructions that ignore temporal redundancies. As a remedy, we propose a highly efficient, real-time capable prediction and correction approach that fully relies on partial differential equations (PDEs) in all steps of the codec: Dense variational optic flow fields yield accurate motion-compensated predictions, while homogeneous diffusion inpainting is applied for intra prediction. To compress residuals, we introduce a new highly efficient block-based variant of pseudodifferential inpainting. Our novel architecture outperforms other inpainting-based video codecs in terms of both quality and speed. For the first time in inpainting-based video compression, we can decompress FullHD (1080p) videos in real-time with a fully CPU-based implementation, outperforming previous approaches by roughly one order of magnitude

Enhanced Intra Prediction for Video Coding by Using Multiple Neural Networks

This paper enhances the intra prediction by using multiple neural network modes (NM). Each NM serves as an end-to-end mapping from the neighboring reference blocks to the current coding block. For the provided NMs, we present two schemes (appending and substitution) to integrate the NMs with the traditional modes (TM) defined in high efficiency video coding (HEVC). For the appending scheme, each NM is corresponding to a certain range of TMs. The categorization of TMs is based on the expected prediction errors. After determining the relevant TMs for each NM, we present a probability-aware mode signaling scheme. The NMs with higher probabilities to be the best mode are signaled with fewer bits. For the substitution scheme, we propose to replace the highest and lowest probable TMs. New most probable mode (MPM) generation method is also employed when substituting the lowest probable TMs. Experimental results demonstrate that using multiple NMs will improve the coding efficiency apparently compared with the single NM. Specifically, proposed appending scheme with seven NMs can save 2.6%, 3.8%, 3.1% BD-rate for Y, U, V components compared with using single NM in the state-of-the-art works.Comment: Accepted to IEEE Transactions on Multimedi