Mode dependent vector quantization with a rate-distortion optimized codebook for residue coding in video compression

International audienceThe High Efficiency Video Coding standard (HEVC) supportsa total of 35 intra prediction modes which aim at reducingspatial redundancy by exploiting pixel correlation within a localneighborhood. In this paper, we show that spatial correlationremains after intra prediction, leading to high energyprediction residues. We propose a novel scheme for encodingthe prediction residues using a Mode Dependent Vector Quantization(MDVQ) which aims at reducing the redundancy inresidual domain. The MDVQ codebook is optimized in arate-distortion (RD) sense. Experimental results show thatthe codebook can be independent of the quantization parameter(QP) with no loss in terms of coding efficiency. A bitratereduction of 1:1% on average compared to HEVC canbe achieved, while further tests indicate that codebook adaptivitycould substantially improve the performanc