Centre for Multimedia Signal Processing, Department of Electronic and Information EngineeringRefereed conference paper2004-2005 > Academic research: refereed > Refereed conference paperVersion of RecordPublishe

Chan, YL

Cheung, HK

Siu, WC

PolyU Institutional Repository

ICSP'O4 Proceedings ROBUST AND FAST GLOBAL MOTION ESTIMATION FOR ARBITRARILY SHAPED VIDEO OBJECTS IN MPEG-4 Hoi-Kin Cheung, Yui-Lam Chan and Tun-Chi Siu Centre for Multimedia Signal Processing Department of Electronic and Information Engineering The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong ABSTRACT Global motion estimation is used for esploiting temporal redundancies in arbitrarily shaped video objects, which is coinpulatiorlally Uie most demanding part within the MPEG-4 standard. In this paper, we propose a robust and fast method for the estimation of global motion of video objects used in the MPEG-4. Predictors are widely used in block-based motion estimation and it is well proven that the use of good predictors can speed up the motion estimation process. The proposed algorithm makes use of the shape information (alpha-plane) of video objects to form a new predictor, From the experiniental results, we conclude< that the new predictor provides additional performance gains with reducing computational complesiry of global motion estimation. 1. INTRODUCTION MPEG4 is an inlernational standard which provides a coding scheme for arbitrarily shaped video objects (VOs) [l-31. Each VO is composed of its tempom1 instances, video object planes (VOPs). which is the central concept of MPEG-4 video. A VOP can be fully described by teaure variations and shape representation, as shown in Figure 1. In 'natural scenes, VOPs are obtained by a semiautomatic [4-51 or automatic segmentation [6 ] ,  and the resulting shape informatioii can be represented as a binary alpha-plane. The alpln-plane contaih the information to identlfy the pixels which are inside of an object (value of alpha-plane = I), and the pixels which are outside of the object (value of alpha-plane = 0), as depicted in Figure 1@). Coding of ar6itrarily shaped video objects relies on the reduction of statistical redundancies in the data and on the exploitation of the human visual system limitations. As the lunlinance and color in dynamic images are most correlated in the direction of motion, any redundancy removal is highly dependent on motion information. For tlus purpose, a nonlinear prediction technique, called niotion compensation (MC), has been developed and used with a renuirkable success It consists of two steps: estimating motion behveen successive video frames and then predicting the current frame from previously tmmnitted frames using the motion infonnation [7-XI. Motion compensation is a core technology of all video compression standards developed to date, such as MPEG- 1 [9 ]  andMPEG-2 [10], (b) Figure 1. Representation of the VOP. (a) Image of original "Bream" VOP. (b) Binary alpha-plane of "B" VOP , In general, motion in a VO results from entire motion of the VO and fmm displacements of individual parts cotnposing the VO. The former is often referred to as global motion of the VO and the latter as local motion of the VO. Most motion estimation techniques ignore t l k  aspect and make no distinction between the global and local niotion; global motion is taken into account only implicitly though local estimates. For instance, in terms of motion compensation, MPEG-1 [9] and MPEG-2 [ 101 rely on local motion model of blocks under translation. However. it is usually advantageous to process global and 0-7803-8406-7/04/$20.00 @ 2004 IEEE. 1163 Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on June 29,2010 at 08:20:45 UTC from IEEE Xplore.  Restrictions apply. local motion separately. Not only does it result in more precise estimates, but it also leads to a more compact representation o f  the motion infonnation. Consequently. global motion estimation (GME) for an individual VO is a new coding teclmology for video compression in the recent MPEG-4 standard, as can be found in the MPEG-4 verification model (VM) [ll]. However. techniques for global motion estimation of each foreground VO just directly employ the tools that are used in dynamic sprites generation [12]. In this paper, by considering the shape infomtion of video objects, we propose a novel global motion estimation algorithm that is especially tailored for arbitmily shaped VOs. The paper is organized as follows. Section 2 gives an ovenku. of global motidn estimation. Section 3 describes the proposed algoritlun. Section 4 demonstrates esperimental results. Finally, section 5 provides the conclusions. 2. GLOBAL MOTION ESTIMATION The GME technique is designed to minimize the s u m  of squared daerence between the current VOP 1 and the motion-compensated previous VOP 2'. with e ( i , j )  = l ( i , j )  - I ' ( i ' ,  j ' )  where w is the alpha-plane of the current VOP, ( i j ]  denotes the spatial coordinates of the pixel in the current VOP. (i ; j ' )  denotes the Coordinates of the corresponding pisel in the previous VOP, A4x N is the size of the bounding box that is a box surrounding the current VOP with the minimum number of macroblocks, and the suimnation is carried out over all corresponding pairs of pisels inside the current VOP, that is, the value of w(i,j,J is equal to 1. To describe the global motion field, a 4garameter affine motion model is typically used, which can be written as where m = [ma ml, m2, m3, m4, ms] are the global motion parameters. Basically. m2 and m5 describe the VO under translation while m& mi, m3 and m, describe rotation and deformation of the VO. This affine model can deal with the majority of motion types encountered in video coding. To compute model parameters, implicit in (i ', j I ) ,  we use a gradient descent method to nuniinize E in equation (l), In the MFEG-4 VM, the Levenberg-Marquardt iterative nodinear (LM) algoritlun [13] is employed to perform the object-based minimization in order to get affine parameters [vih m,, mA m4 ni4, ms]. Since the dependence of E on m is nonlinear, LM algorithm updates motion parameters m iteratively [ 131 as follows: (3 )  where m'"" and are the motion parameters at iteration t + I  and t, respectively, H is the Hessian matrix and 6 is the weighted gradient vector. More specifically. the coefficients of matrix Hand vector b are given by and (4) ( 5 )  We are now to describe the flow of the LM algorithm. The motion parameters mina are first initialized with the translational vector. The initialization will be discussed in Section 3 .  Each iteration of the LM algorithm consists of the following steps: For each pixel at location (i,J which is inside the VO, compute its corresponding position in the position (i ',j 3 according to equation (2); Compute the error e(ij) and the partial derivative of e(iJ) with respect to the mk; add the pixel's contribution to matrix EI and vector b as equations ( I )  and (5). Solve the system o f  equations and update the motion parameters defined in equation (3). Continue iterating until the error starts to increase or IV,,,~ iterations are ccarried out. 1. 2. 3 .  4. 3. ROBUST ESTIMATION OF INITIAL MOTION PARAMETERS However, GME is a computation-intensive, time consuming task. In order to reduce the computational complexity, the estimation of initial motion parameters - the predictor is desired. This predictor is also to assure the convergence of the subsequent LM algorithm. Note that this predictor does not need Lo accurate. However, it must be robust enough to make the starting point of the LM algorithm tie within a convergence "basin" of the global minimum of E. For this purpose, an initial estimation of the translational components (ma ms) of the displacement is computed in the MPEG-4 VM [l 11. This predictor is obtained by a matching technique, which minimizes E using a n-step hierarchical starch (n-SHS) [14]. 1 I64 Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on June 29,2010 at 08:20:45 UTC from IEEE Xplore.  Restrictions apply. However, it is well known in block-based motion cstiination that n-SHS can give reasonable translational inotion within a search area of +7. But when the global translational motion of a VO is larger than f7,  n-SBS would typtically give very unreliable values resulting in inany iterations in LM. The worst case performance of GME using n-SHS as initialization is poor in our esperiiiients, And even if the global transfational motion of a VO is within *7, it is well known that the motion parameters from n-SHS can be chaotic. This gives rise to many iterations in the LM algorithm resulting in poor performance. Our improvement over GME using n-SHS is achieved by incorporating the binary alpha-plane to accurately predict the initial motion parameters such that a good initial condition is applied in the LM algorithm to reduce the number of iterations, Since global tmnslational motion is mostly the entire translation between two consecutive VOPs. we can approximate the global translational motion as tlie displacement of centroids between the current and previous alpha-planes. The coordinates of the centroid of the current alpha-plane w, (c., cy), are determined using the moment. as defined in following equations: M-l N-i i x w(i, j )  1-0 i=O and (7) j=o i=o Similarly, tlie coordinates of the centroid of the previous alpha-plane w ’ , (c :- c &, are then computed as = and M-I N-1 c’ = Y (9) Thus, the proposed initial predictor mhit is [ I ,  0, cIx-c, 0, I, cjcyl. 4. SIMULATION RESULTS A series of computer simulations have been conducted to evaluate the p e r f ~ t ~ ~ i ~ ~ e  of the proposed algorithm. Sirnulations have been carried out using arbitrarily shaped video objects of “Bream” and “AkiT sequences. The mean square error WSE) is used to compare the performance of the proposed algorithin with some related techniques. For the stopping criterion of GME, N,,,, was set to 32. Table 1 compares tlie performance of GME without predictor (GME+no-Pred) versus the cases when the predictor is computed using the displacement of centroids (GME+DoC-Pred) or the conventional n-SHS (GME+nSHS_Pred). In Table 1, the performance of GME using both predictors (GME+Both-hd) is also shown. For GME+Both-PEd, E as defined in equation (1) of both predictors are calculated and the predictor with the minimum E is chosen as the starting point of the LM The detailed comparisons of the average MSE and the number of iterations of the LM algorithm between GME+no-Pred, GME+nSHS-Pred md the proposed GME+DoC-Pred are tabulated in Table 1 in which the fraines are temporally dropped by a factor of 0 (no dropping), 1, 2 and 3. Note that the number of iterations can reflect the computational complexity of GME since the initialization process is simple as compared with each iteration step of the LM algorithm. We show that GME+DoC-Pred outperforms GME+no-Pred and GME+nSHS-Pred. The results m more significant for algorithm. the larger skipping factors. In those cases, since the motion between two consecutive VOPs is increasing, the initial predictor becomes critical for G m .  Figure 2 shows that the proposed GhE+DoC-Pred usually gives the minimum initial MSE for all of the frames. Thus, our proposed estimation of predictor can provide a good initial condition for the LM algorithm such that the number of iterations can be reduced and it is also robust enough to make the starting point of the LM algorithm lie within a convergence- “basin” of the global minimum of E. Besides, since GME+Both-Pred has the advantage of selecting the best predictor, it can firther improves the performance. ( 8 )  1165 Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on June 29,2010 at 08:20:45 UTC from IEEE Xplore.  Restrictions apply. Computing. Cambridge, U.K.: Cambridge Univ. Press, 1988. 5. CONCLUSIONS In this paper, a robust predictor for global motion [14] T. Koga, K. Iinuma, A. Hirano, Y. Iijima, and T. Ishiguro, estimation of arbitrarily shaped video objects in MPEG-4 “Motion compensated interframe coding of video has been proposed. The b i w  alpha-plane is used for tlie conferencing,” in Roc. Nat. Telecomniun. Cod.,  New adjustment of tlie starting point of the LM algorithm such Orleaiis,LA,pp.G.5.3.1-G.j.3.5,Dec. 1981  Coding initialization predictor Sequence tl&t a number o f  iterations can be significantly reduced Experimental results show that our proposed algorithm can also improve the reconstmction quality of arbitrarily shaped video objects. Thus. this algorithm has the advantage of being siiiiultaneously robust and fast. 6. ACKNOWLEDGMENTS l l i s  work is supported by the Centre for Multimedia Signal Processing, HKF’olyU (A-PD64). 7. REFERENCES 111 ISOnEC 14496-2, ‘Tiifontiation tehology - Coding of audio-visual objects: visual”, 1998. 121 ISOLlEC, ”MPEG4 video verification model version 11 .0,” lSOllEC JTCI/SC29/WG11 N2172, March 1998. 131 T. Sikora, “The MPEG-4 video standard verification model,” IEEE Tram. Ckcuits Systems Video Teclulol., vol. 7; no, 1, pp. 19-31, Feb. 1997. [4J Roberto Castagno, Toiuadj Ebrahimi, and Murat Kunt, “Video seginentatioii based on niultiple features for interactive inultiniedia applications,” lEEE Trans. Circuits Systems Video Technol., vol. 7, no. 5, pp. 562-571, Sept. 1998. [j] Clitiang Gu and Miiig-Chi& Lee, “Semiautomatic sgnicntation arid tracking of seniantic video objects,” IEEE Trans. Circuits Systems Video Technol., vol. 7, no. 5,  pp. 572-584, Sept. 1998. + [6] Thoinas Meiei- and King N. Ngan, “Automatic segmentation of iiioving objects for video object plane generation,” IEEE Trans. Circuits Systems Video Teclmol., vol. 7, no. 5, pp. 525-538Sept. 1998, [7] Yui-Lam C h i  and Wan-Chi Siu, “An Efficient Search Strategy for Block Motioii Estimation using Image Features”, IEEE Transactions on Image Processing, vol. 10, no. 8, pp. 1223-1238,Ailgt~t 2001. [SI Yui-Lam Chan and Wan-Chi Siu, ‘’New Adaptive Pixel Decimation for Block Motion Vector Estimation,” IEEE Transactions 011 Circuits and Systems for Video Tecluiologv, vol. 6, no. 1, pp.113 -1 18, February 1996. (91 lSO/IEC 11 172-2, “Infomiation technology - Coding of moving pictures and associated audio for digital storage “ l a  at up to about 1.5 Mbi t /s  --Part 2: Video,” 1993. [IO] ISOlIEC 13x18-2, “Infomiation technology - Generic coding of moving pictures and associated audio ixformation: Video,” 1996. [ 1 I] ISOiIEC, “MPEG-4 video Verification model version 18.0,” ISOfiEC JTClISC291wG11 N3908, Januq 2001. [12] F. Dufaus mid J. Konrad, “Efficient, robust, and fast global motion estiiiiation for video coding,” IEEE Trans. on Image Processing, vol. 9, no. 3 .  pp. 497-501, March 2000. [13] W.H. Press, B.P. Flaruiery, S.A. Teukolsky, and W.T. Vetkding, Nunierical Recipes in C: The Art of Scientific GMEt GME+ GME+ GMEt iio-Pred nSHS-Fred Doc-Fred Both-Pred no. of no. o f  no. of no. of iterations iterations iterations iterations (USE) ( M W  ( M W  (USE) Figure 2. Initial MSE for different predictors. Table 1. Comparison of GME without and with predictor($. I (120.7) I ( 151.7J I ( 123.8) I ( 12 1.04) ~ k i 2  I 4.80 1 3.58 I 4.35 I 3.62 66.4) Sk’ in acior=l 243.5 378.0 24k.5 226.9 134.8) 127.9 128.3) Aki2 7.47 1 (481.3) I ( 480.7) I ( 328.7) 1 ( 332.6) Aiiis 1 10.4 \ 10.0 \ 8.9 I 9.42 206.6) 191.6) 192.6 190. 1 741.4 546.7 42 I .2 (242.9) 240.3) 1166 Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on June 29,2010 at 08:20:45 UTC from IEEE Xplore.  Restrictions apply. 

Robust and fast global motion estimation for arbitrarily shaped video objects in MPEG-4

Abstract

Similar works

Full text

Available Versions

PolyU Institutional Repository