De Cock, Jan

Deligiannis, Nikos

Lambert, Peter

Munteanu, Adrian

Mys, Stefaan

Skorupa, Jozef

Slowack, Jürgen

Van de Walle, Rik

English

Ghent University Academic Bibliography

  biblio.ugent.be  The UGent Institutional Repository is the electronic archiving and dissemination platform for all UGent research publications. Ghent University has implemented a mandate stipulating that all academic publications of UGent researchers should be deposited and archived in this repository. Except for items where current copyright restrictions apply, these papers are available in Open Access.  This item is the archived peer-reviewed author-version of: Title: Correlation modeling with decoder-side quantization distortion estimation for distributed video coding Authors: Jozef Škorupa, Jan De Cock, Jürgen Slowack, Stefaan Mys, Nikos Deligiannis, Peter Lambert, Adrian Munteanu and Rik Van de Walle In: Proceedings of Picture Coding Symposium, December 2010.  To refer to or to cite this work, please use the citation to the published version: Jozef Škorupa, Jan De Cock, Jürgen Slowack, Stefaan Mys, Nikos Deligiannis, Peter Lambert, Adrian Munteanu and Rik Van de Walle (2010). Correlation modeling with decoder-side quantization distortion estimation for distributed video coding. Proceedings of Picture Coding Symposium, 2010. CORRELATION MODELINGWITH DECODER-SIDE QUANTIZATION DISTORTIONESTIMATION FOR DISTRIBUTED VIDEO CODINGJozef Sˇkorupa, Jan De Cock, Ju¨rgen Slowack,Stefaan Mys, Peter Lambert, Rik Van de WalleGhent University – IBBT,ELIS – Multimedia Lab,Gaston Crommenlaan 8 bus 201,B-9050 Ledeberg-Ghent, BelgiumNikos Deligiannis and Adrian MunteanuVrije Universiteit Brussel – IBBTElectronics and Informatics DepartmentPleinlaan 2, B-1050 Brussels, BelgiumABSTRACTAiming for low-complexity encoding, distributed videocoders still fail to achieve the performance of current in-dustrial standards for video coding. One of most importantproblems in this area is the accurate modeling of the corre-lation between the predicted signal and the original video.In our previous work we showed that exploiting the quan-tization distortion can significantly improve the accuracy ofa correlation estimator. In this paper we describe how thequantization distortion can be exploited purely at the decoderside without any performance penalty when compared to anencoder-aided system. As a result, the proposed correlationestimator delivers state-of-the-art modeling accuracy whileneatly fitting the low-encoder-complexity characteristic ofdistributed video coding.Index Terms— distributed video coding, correlation mod-eling, quantization distortion1. INTRODUCTIONBased on Wyner-Ziv theory, distributed video coders repre-sent an alternative to the fixed complexity distribution in pre-dictive coders, specified by the current industrial standards.Offering low-complexity encoding, distributed coders are ide-ally suited for application scenarios involving deployment ofa large number of power- or hardware-constrained devices,e.g., (wireless) video surveillance.Unfortunately, Distributed Video Coding (DVC) stillfails to match the coding performance of predictive codingschemes. One of the most important factors [1] influenc-ing the coding performance is the accurate modeling of thecorrelation between the original video signal available at theencoder and the predicted signal (or so-called side informa-tion), which is generated at the decoder in DVC systems.Typical improvements in correlation modeling result ingains which are limited to high rates [2, 3]. Recently, we haveproposed a correlation estimator which exploits the distortioninduced by quantization in order to achieve systematic codinggains throughout the entire rate region [4, 5]. The proposedmethod employed the encoder to estimate the distortion andsend necessary information through the video stream. This in-troduces an unnecessary, albeit small, overhead both in termsof bit-rate and encoder complexity. To alleviate this problem,we have designed a pure decoder-based method to estimatethe quantization distortion which is employed in our correla-tion estimator.The main contribution of this paper is the description of adecoder-based correlation estimator which exploits the quan-tization distortion in order to achieve an accurate estimationof the correlation throughout the entire rate region. We de-scribe our correlation estimator itself in Sect. 3.1, while inSect. 3.2 we focus on the decoder-based estimation of quanti-zation distortion.In order to assess the impact of pure decoder-based esti-mation as opposed to encoder-aided estimation, and to posi-tion our estimator relative to state-of-the-art correlation esti-mators, we have conducted experiments using the DVC ar-chitecture described in Sect. 2. The results reported in Sect. 4indicate that exploiting the quantization distortion in corre-lation estimation yields significant coding gains, while doingthat purely at the decoder brings no performance penalty com-pared to encoder-aided estimation.2. CODEC ARCHITECTUREThe general architecture of our codec (Fig. 1) is based on thecommon DVC schema well described in the literature, e.g.[6]. The frames are divided into key frames In and Wyner-Ziv frames Xn based on a fixed GOP structure. Key framesare intra coded using an H.264/AVC intra coder. For eachWyner-Ziv frame Xn, the decoder generates a prediction Ynas a motion-compensated interpolation of previously decodedframes [6]. The prediction can be seen as a noisy version ofthe original frame corrupted by virtual noise1 Nn = Xn−Yn.By applying an error-correcting code at the encoder and send-ing parity bits to the decoder, the errors in the prediction are1Due to the description of the problem using the virtual noise abstraction,correlation estimation is often referred to as virtual-noise estimation.corrected and the decoded Wyner-Ziv frame Xˆn is obtained2.DecoderEncoderTurbo encoder BufferTurbo decoderFrame interpolationQ-1XI H.264/AVCIntra encoder IFeedback channelT-1T&Q ^CorrelationestimatorH.264/AVCIntra decoderX^Fig. 1. The codec extracts bitplanes from transformed (T) andquantized (Q) frames, and the parity bits calculated for eachbitplane by a turbo coder are used to correct the prediction atthe decoder.3. CORRELATION ESTIMATORIn this section we describe our correlation estimator in twosteps. In Sect. 3.1, we describe correlation estimation as pro-posed in our previous work [5]. Then we show how this esti-mation can be improved by a no-reference quality assessmentat the decoder (Sect. 3.2).3.1. Correlation estimationAs common in the DVC literature, we will assume that thevirtual noise follows a Laplacian distribution, given byfX|Y (x|y) = λk2 e−λk|x−y| , (1)where λk > 0 is the scale parameter, which is to be estimatedby the correlation estimator on a block basis.Most of the state-of-the-art estimators rely on the motion-compensated residual frame R to estimate the correlation.The residual frame is defined asRn(p) = Xˆn−i(p+m−(p))− Xˆn+i(p+m+(p)) , (2)where p denotes the position in the frame and m−(p)),m+(p)) denote the motion vectors pointing to the previousreference frame Xˆn−i and the future reference frame Xˆn+i,respectively.As we have pointed out in our previous work [4, 5], theresidual frame alone is not enough to obtain an accurate corre-lation estimation for the following reasons. Firstly, the resid-ual frame does not properly reflect changes in quantizationsettings. Secondly, the spatial properties of the residual frame2In the following, we drop the temporal index of the frame when it is clearfrom the context.usually differ from those of the virtual-noise frame, which im-pairs the correlation estimation in the frequency domain whenthe transformed residual frame is used. To solve these issues,we have proposed a two-tier estimator [5], as described in thefollowing.In the first step, the virtual noise in the pixel domain isestimated using maximum-likelihood estimationλk =1nk, (3)where nk is the average of the absolute value of the noisein a block Bk. Because the value of nk is not known at thedecoder, it is estimated usingnk =rk2+ q , (4)where rk is the average of the absolute value of the residualframe in the block Bk and q is the average (absolute) distor-tion of the reference frames. Eq. (4) is based on an analyticalmodel for motion-compensated interpolation we described in[5]. The rationale behind the model is that the virtual noiseis a compound distortion consisting of a mismatch caused byblock-based motion estimation (rk/2) and the distortion al-ready present in the reference frames (q).In our previous work, we approximated the second termq with the quantization distortion in the intra coded frames,which was calculated at the encoder side and sent along thestream. The main contribution of this paper is a method forestimation of q at the decoder side, which we describe inSect. 3.2.The second tier of our correlation estimator employs thevirtual noise estimated in the pixel domain to obtain estima-tion of the correlation in the frequency domain. As we al-ready pointed out in [5], this approach has the advantage of in-creased statistical support for the estimation and alleviates theproblem of improper spatial properties of the residual frame.As pointed out by several authors [3, 5], the per-band vari-ance of the noise signal in the frequency domain can be ex-pressed using the variance of the noise in the pixel domainasσ2N(b)k= σ2Nk V(b) , (5)where σ2Nk is variance of the pixel domain noise which is cal-culated as σ2Nk = 2/λ2k using parameter λk obtained in thepixel domain, and V (b) is a scale factor for band b, whichdepends on the autocorrelation of the noise signal. In our pre-vious work, we modeled the autocorrelation of the noise as amixture of autocorrelation of the previously decoded Wyner-Ziv frame, estimated at the decoder, and the autocorrelation ofthe previously decoded key frame. The latter was estimated atthe encoder and communicated through the bitstream. In thispaper we show how V (b) can be accurately estimated purelyat the decoder, alleviating the need for encoder-decoder com-munication (Sect. 3.2).3.2. Decoder-side distortion estimationThe distortion of the key frames and their spatial properties(autocorrelation) exploited in our estimator (Sect. 3.1) arevery important for accurate correlation estimation, especiallyat low bit rates. Albeit very small, the approach from [5]still imposes overhead on the bit rate and encoder complexityby measuring the required values at the encoder and sendingthem along the stream. We now propose a method to estimateq and V (b) required in (4) and (5), respectively, purely at thedecoder side.The method is based on no-reference image quality as-sessment proposed by Branda˜o and Queluz [7]. The distribu-tion in frequency band b of the signal to be coded—in the caseof the H.264/AVC intra coder, the residual after subtractingthe intra prediction—is assumed to be Laplacian with scaleparameter α(b). For each band the scale parameter is obtainedas a weighted average of the maximum-likelihood estimationα(b)ML and prediction from the previously estimated bands α(b)pas followsα(b) = (1− r(b)0 )α(b)ML + r(b)0 α(b)p . (6)In (6), the weighting coefficient r(b)0 = M(b)0 /M is taken asthe ratio between the number of coefficients in band b quan-tized to zeroM (b)0 and the total number of coefficients in theband M . The prediction term α(b)p in (6) is used to compen-sate for overflow in maximum-likelihood estimation when al-most all coefficients are quantized to zeroM (b)0 →M , i.e. atlow bit rates and/or in high frequency bands.The maximum-likelihood estimation operates with thenumber of zero coefficients M (b)0 and the sum of abso-lute values of the inverse quantized coefficients S(b) =∑M−1l=0 |X(b)l |, and the estimated parameter reads asα(b)ML = −2Qlog−M (b)0 Q+√D2MQ+ 4S(b)(7)where Q denotes the quantizer step size and D reads as D =(M (b)0 Q)2 − 4(MQ+ 2S(b))((M −M (b)0 )Q− 2S(b)).For all AC bands, α(b)p is predicted from already estimatedparameters (estimation is performed in zig-zag scan order) asα(b)p = α(b)β(b)T, (8)where α(b) = (1, α(0), . . . α(b−1)) and β(b) is the predictorvector. β(b) is trained offline by minimizing the predictionerror on a given training set. The training procedure is de-scribed in great detail in [7].Knowing the parameter of the distribution, one can calcu-late the squared distortion that a scalar quantizer with knownquantization step and deadzone parameter will induce on suchsignal. Obtaining the distortion includes some heavy, albeitstraightforward, algebraic manipulations [8]. We will denotewith σ2(b) the squared distortion induced by quantization ofthe band b. Recognizing that the DCT is an orthogonal trans-form3, we can clearly see that the squared distortion in thekey frame, i.e. variance of the frame difference I − Iˆ , couldbe expressed (for a 4× 4 transform) asσ2I−Iˆ =11615∑b=0σ2(b) . (9)Assuming that I − Iˆ follows a Laplace distribution, we canexpress the required average absolute value of the distortionin the key frames asq =√σ2I−Iˆ2, (10)which can be directly used in (4).Having estimated the quantization distortion q, we stillneed to obtain the scaling factor V (b) from (5). V (b) can beexpressed asV (b) =∑p1∑p2c(b)p1 c(b)p2RN (p1,p2) (11)where c(b)p1 and c(b)p2 are the corresponding coefficients of thetransformation [3, 5], RN (p1,p2) is the autocorrelation ofN = X − Y , and the pixel positions p1,p2 loop throughthe whole (4 × 4) transform block. We proposed to approx-imate the autocorrelation function as a weighted average oftwo autocorrelations—autocorrelation of frame differencebetween the previously decoded Wyner-Ziv frame and itsprediction Xˆp − Yp, and autocorrelation of frame differencebetween the last decoded key frame and its original I − Iˆ .Together with the weighting coefficient, the autocorrelationreads asRN (.) = wRXˆp−Yp(.) + (1− w)RI−Iˆ(.) (12)w = σ2Xˆp−Yp/(σ2Xˆp−Yp + σ2I−Iˆ) . (13)Analyzing (11) and (12), one can clearly see that the scal-ing factor V (b) can be expressed as the following average:V (b) = wV (b)Xˆp−Yp + (1−w)V(b)I−Iˆ , where both terms V(b)Xˆp−Ypand V (b)I−Iˆ are given as a sum similar to (11) with respective au-tocorrelation functions RXˆp−Yp and RI−Iˆ . Because Xˆp− Ypcan be constructed at the decoder, σ2Xˆp−Yp and RXˆp−Yp canbe calculated directly. σ2I−Iˆ is obtained by no-reference dis-tortion estimation (9) and we propose to estimate the scalingfactor V (b)I−Iˆ with the already obtained information asV(b)I−Iˆ = σ2(b)/σ2I−Iˆ . (14)Using the method described in the previous paragraphs,the proposed estimator alleviates the need for encoder-decoder communication, saving the bit rate and encodercomplexity required for such communication.3In case of the H.264/AVC integer transform, one needs to account forthe scaling first.Table 1. The relative error |q− qe|/qe of no-reference distor-tion assessment according to Eq. (10).QP M&D Foreman Coastguard Mobile20 0.04 0.03 0.12 0.0425 0.09 0.01 0.15 0.0230 0.07 0.04 0.17 0.0135 0.03 0.03 0.23 0.034. EXPERIMENTAL RESULTSWe trained the predictor vector (8) on 12 sequences, differentthan the test sequences. Tests were conducted on four CIFsequences (30 Hz): Mother and daughter (M&D), Foreman,Coastguard and Mobile. GOP sizes of 4 and 8 were used.First, we explored the accuracy of the distortion q esti-mated at the decoder (10) by comparing it to the distortion qeas measured at the encoder. The results (Tab. 1) show that theproposed no-reference quality assessment provides a highlyaccurate estimation for most sequences.To assess the impact of the correlation estimator on theRD performance, we have compared our DVC system em-ploying the proposed estimator, the estimator proposed inour previous work [5] and the state-of-the-art correlationestimator as described by Brites and Pereira in [1]. Our es-timator outperforms the state-of-the-art with average bit rategains of 2.8% (Mobile), 10.8% (Foreman), 13.2% (Coast-guard) and 13.9% (Mother and daughter), as measured by theBjøntegaard-Delta metric. These gains are systematic overthe entire rate region and are brought by the use of the quanti-zation term q in (4); we notice also that having gains over theentire rate range is in steep contrast to typical gains reportedin literature, which tend to diminish towards low rates.Comparing to our previous work [5], where the quantiza-tion distortion is calculated at the encoder, we can see that (i)no-reference assessment at the decoder provides an accurateestimation of the distortion and (ii) the proposed estimator isnot hindered by the lack of exact data from the encoder. Onthe contrary, by eliminating the need for sending the distortionvalues in the stream, small gains—between 0.2% and 0.5% ofBjøntegaard-Delta rate—are achieved. Moreover, the estima-tor seems to be robust against occasional inaccuracies in thedistortion assessment as shown in Fig. 2, where the sequencewith the largest relative error is depicted.5. CONCLUSIONSIt is clear (from results presented here or in our previous work[4, 5]) that exploiting the distortion in the reference framessignificantly improves the accuracy of correlation estima-tion in DVC. The method proposed in this paper estimatesthis distortion purely at the decoder side. We have shownthat, when compared to encoder-aided estimation, using no-reference quality assessment does not decrease the accuracyof the correlation estimator while it reduces the bit rate andencoder complexity.43373941]313335yPSNR [dB]proposed2527290 1000 2000 3000 4000 5000 6000yPSNR [dB]Skorupa et al. [5]Brites and Pereira [1]yPSNR [dB]bit rate [kbps]yPSNR [dB]yPSNR [dB]Fig. 2. Exploiting the quantization distortion leads to signif-icant gains over the state-of-the-art estimators, whether theno-reference decoder-side assessment (proposed) or encoder-aided exact measurement ([5]) is employed. Coastguard,CIF@30Hz, GOP 8.6. REFERENCES[1] C. Brites and F. Pereira, “Correlation noise modelingfor efficient pixel and transform domainWyner-Ziv videocoding,” IEEE Trans. Circuits Syst. for Video Technol.,vol. 18, no. 9, pp. 1177–1190, Sep 2008.[2] X. Huang and S. Forchhammer, “Improved virtual chan-nel noise model for transform domain Wyner-Ziv videocoding,” in Proc. IEEE Inter. Conf. on Acoust., Speech,Signal Processing, Apr 2009.[3] X. Fan, O. C. Au, and N. M. Cheung, “Adaptive correla-tion estimation for general Wyner-Ziv video coding,” inProc. IEEE Inter. Conf. Image Processing, Nov 2009.[4] J. Slowack, S. Mys, J. Sˇkorupa, P. Lambert, and R. Van deWalle, “Accounting for quantization noise in online cor-relation noise estimation for distributed video coding,” inProc. Picture Coding Symposium, May 2009.[5] J. Sˇkorupa, J. Slowack, S. Mys, N. Deligiannis,J. De Cock, P. Lambert, A. Munteanu, and R. Van deWalle, “Exploiting quantization and spatial correlationin virtual-noise modeling for distributed video coding,”Signal Processing: Image Commun., vol. 25, no. 9, pp.674–686, Oct 2010.[6] C. Brites, J. Ascenso, J. Quintas Pedro, and F. Pereira,“Evaluating a feedback channel based transform domainWyner-Ziv video codec,” Signal Processing: ImageCommun., vol. 23, no. 4, pp. 269–297, Apr 2008.[7] T. Branda˜o and M. P. Queluz, “No-reference image qual-ity assessment based on dct domain statistics,” SignalProcess., vol. 88, no. 4, pp. 822–833, Apr 2008.[8] G. J. Sullivan, “Efficient scalar quantization of exponen-tial and Laplacian random variables,” IEEE Trans. In-form. Theory, vol. 42, no. 5, pp. 1365–1374, Sep 1996.

Correlation modeling with decoder-side quantization distortion estimation for distributed video coding

Crossref

https://biblio.ugent.be/publication/1095102/file/1095108.pdf

Correlation modeling with decoder-side quantization distortion estimation for distributed video coding

Abstract

Similar works

Full text

Available Versions

Ghent University Academic Bibliography

Crossref