An Estimation-Theoretic Framework for Spatially Scalable Video Coding with Delayed Prediction

Abstract—A novel estimation-theoretic (ET) approach is developed for optimal enhancement layer prediction, in spatially scalable video coding (SVC), which incorporates motion compensation at the enhancement layer, with both current and future information from the base layer. It is inspired by the early ET framework (originated in our group) for quality (SNR) scalability, which achieved optimal enhancement layer prediction by fully accounting for information from the current base layer (e.g., the quantization intervals) and the enhancement layer, to efficiently calculate the conditional expectation that forms the optimal predictor. Central to that approach was the fact that all layers reconstruct approximations to the same original transform coefficient. This, however, is not the case in spatial scalability, where the layers encode different resolution versions of the signal. To approach optimal enhancement layer prediction, the current work departs from existing spatial SVC schemes that employ pixel-domain resampling and causal prediction. Instead, it integrates a transform domain resampling technique that makes the base layer quantization intervals and reconstructions accessible to and usable at the enhancement layer. The approach is extended for an SVC framework that allows delay in enhancement layer coding relative to the base layer, and achieves optimal delayed prediction, in conjunction with spatial SVC. Simulations provide experimental evidence that the overall proposed approach substantially outperforms existing spatially scalable coders. I