In this paper we propose an improved motion estimation algorithm based on evolutionary strategy (ES) for H.264 video codec applied to video. The proposed technique works in a parallel local search for macroblocks. For this purpose (mu+lambda) ES is used with an initial population of heuristically and randomly generated motion vectors. Experimental results show that the proposed scheme can reduce the computational complexity up to 50% of the motion estimation algorithm used in the H.264 reference codec at the same picture quality. Therefore, the proposed algorithm provides a significant improvement in motion estimation in the H.264 video codec

Arachchi, HK

Fernando, WAC

Kalganova, T

Mayuran, S

English

Crossref

Evolutionary Strategy Based Improved Motion Estimation Technique for H.264 Video Coding

Brunel University Research Archive

 EVOLUTIONARY STRATEGY BASED IMPROVED MOTION ESTIMATION TECHNIQUE FOR H.264 VIDEO CODING S. Mayuran   W.A.C. Fernando T. Kalganova H. Kodikara Arachchi Electronic and Computer Engineering, School of Engineering and Design, Brunel University,  Kingston Lane, Uxbridge, UB8 3PH, UK.  {dc04mms, anil.fernando, t.kalganova, eesrhhk}@brunel.ac.lk   Abstract In this paper we propose an improved motion estimation algorithm based on Evolutionary strategy (ES) for H.264 video codec applied to video. The proposed technique works in a parallel local search for macroblocks. For this purpose (μ+λ) ES is used with an initial population of heuristically and randomly generated motion vectors. Experimental results show that the proposed scheme can reduce the computational complexity up to 50% of the motion estimation algorithm used in the H.264 reference codec at the same picture quality. Therefore, the proposed algorithm provides a significant improvement in motion estimation in the H.264 video codec.  Keywords: Motion Estimation, Evolutionary strategy.  1. Introduction   The H.264 video coding standard is jointly developed by the video coding experts group (VCEG) of ITU and the Moving picture experts group (MPEG) of ICE/ISO. The main goals of the H.264/AVC standardization effort have been enhanced compression performance and provision of a “network-friendly” video representation addressing “conversational” (video telephony) and “non-conversational” (Storage, broadcast, or streaming) applications [1]. H.264/AVC has achieved a significant improvement in rate-distortion efficiency relative to existing standards.   The enhanced compression efficiency in H.264 has a great deal of computation complexity penalty. Most expensive factor behind this penalty is the motion model, which uses multiple block sizes and multiple reference frames [1]. Therefore the computational efficiency can be greatly improved if the motion estimation efficiency is improved. In this paper we propose an enhanced motion estimation algorithm based on evolutionary strategy for H.264 video codec. Evolutionary strategy applied to motion estimation is kind of evolutionary algorithm. Previously evolutionary computation has been applied successfully to motion estimation techniques [2] - [5] and specifically promising results were obtained with use of evolutionary strategy [6] - [7]. Furthermore, the evolutionary strategy differs from other evolutionary computation techniques by its simplicity in hardware implementation.  Proposed evolutionary strategy encodes individuals using x, y coordinates and the reference index. Coordinates and reference indices are described by two different genes where one is called object parameter and the other one is called strategy parameter. Object parameter is a potential solution whereas strategy parameter determines the amount of mutation. The rest of the paper is structured as follows. Section 2 describes the motion estimation process and section 3 describes the proposed ES based algorithm for H.264 video codec. Section 4 describes the simulation and results and section 5 concludes the paper.  2. Motion Estimation  In a video encoder, motion estimation (ME) is a vital and the most time consuming component.  The estimation is done by partitioning the entire frame into blocks of pixels known as macro blocks. The frame that is going under motion estimation is known as the current frame. The frame that it is being compared with is called the reference frame. The encoder will use the previously encoded frame to find the matching block for the current frame. The block in the reference frame that has the closest match to the block in the current frame is known as a matching block. A block being compared to the target block is known as possible matching block [9].  The offset between the matching block and the target block is sent to the decoder this is known as a motion vector [9]. If the matching block is in the exact same position as the target block then the motion vector is (0, 0). This is usually the case for static backgrounds and still objects.   A non-zero value will represent a motion vector for foreground images as these usually have high motion .These motion vectors are used by the decoder to reconstruct the frame. At the decoder each motion vector is used to select corresponding block from the reference frame.    In H.264 Motion estimation consumes more resources and time due to its complexity.  H.264 uses different macro block sizes for motion estimation. It has different sizes as 16 ×16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8 and 4 × 4 that are supported in 1-4244-0038-4 2006IEEE CCECE/CCGEI, Ottawa, May 20062037 H.264. The encoder has to select the best block size intelligently which increases the computational complexity. In video compression scheme, motion estimation is one of the computationally intensive tasks taking up to 60% of the time. H.264 does support multiple reference frame which increases complexity further. Instead of having one reference frame, H.264 can take more than one frame as its reference. During the reconstruction, reference frame is used to predict the current frame using the motion vector. This technique is known as motion compensation   3. Proposed Motion Estimation Technique  Evolutionary computation were developed originally observing natural evolution of life. Evolutionary Strategies (ES) were invented to solve technical optimization problems. Thus, the motion and disparity estimation has been carried out using a (1+λ) rudimentary ES for stereoscopic video sequences, which includes calculation of P- and B-frames, weighted prediction, joint motion disparity estimation Error! Reference source not found., [10].   3.1 Proposed Evolutionary strategy  ES uses stochastic solutions where the unfit individuals are omitted from the solution set. Chromosome defines the potential solution to the problem and in the given case is represented by coordinates x and y defining the motion estimation vector. In this work, the (μ+λ)-Evolutionary Strategy shown in Error! Reference source not found. is applied to motion estimation problem, where μ means the total number of parents in previous population, λ stands for the number of offspring generated from uniformly mutated parents, (μ+λ) is the total number of chromosomes in newly generated population. The distinctive feature of proposed algorithm is a way how generated initial population. This process is discussed in details in the following sub-section.   Heuristic-based pre-defined chromosomes Randomly generated chromosomes Initial populationFitness function evaluationSADMutationSelectionSAD – Sum of Absolute DifferenceFitness function evaluationSADis the number of children obtained after mutationis the fixed population size  Fig 1: (μ+λ)-Evolutionary Strategy-based motion estimation algorithm  . 3.2 Generation of Initial Population  The initial population of the existing [7] ES-based algorithm mostly consists of random motion vectors except the predicted motion vector [1] and eight fixed vectors that represent eight neighboring blocks to the collocated block of the reference frame. However, there is a chance to obtain significant information from the previous frame and previously encoded macroblocks. Consider the following huristics:  1. For stationary objects and  for the background when there is no camera movement, (0,0) is the most possible match.  2. If a moving object extends over more than one block, as shown in Error! Reference source not found., which is the most common scenario, motion vector of one or more nearby blocks provide the best match. This is true when there is a camera movement such as panning.  3. There is a strong correlation that a motion vector of the co-located block in the reference frame and/or at least one of its neighbouring block has the same or closer motion vector as the current block.  Based on the above first heuristic, (0, 0) motion vector is used in the initial population. Similarly, based on the second huristic, motion vectors of four previously encoded blocks near the current block have also been added. Finally, motion vector of the co-located block of the reference frame and its eight neighboring blocks are used in the initial population based on the third huristic.    The rest of the population is randomly determined. Diversity of a population is a major requirement of any ES algorithm. Therefore, after adding the heuristic individuals to the initial population, duplicate individuals are eliminated.        Fig 2: Example for similar macroblocks  Three similar macroblocks 2038 3.3 Fitness Function The quality of each chromosome in ES is evaluated by means of fitness function. In this proposed algorithm the accuracy of the motion compensation with the motion vector encoded in the chromosome and number of bits needed to encode this motion vector in H.264 syntax is selected as the criteria for defining the fitness function. The accuracy of prediction is measured with the Sum of Absolute Differences (SAD) between the predicted block and the original block and the cost of representing the motion vector is the bit count needed for the entropy code representing the actual motion vector and its predicted vector. Therefore, we use the weighted sum of SAD and bit count is used as the fitness value. This criterion ensures that the motion compensation consider both the distortion and the compression efficiency in to account  3.4 Termination Criteria  The most simple termination criteria is to run evolutionary strategy over predetermined number of generations and select the best matching block. If the evolution runs over a large number of generations, the computational cost is unacceptably high. On the other hand, if the number of generations is too small, it would hamper the chance of getting the best matching block,. Therefore we have selected threshold based termination criteria based. When the fitness value goes below a set threshold the algorithm is terminated. This improves the performance, especially with the heuristically define initial population. Such initial population gives the chance of having the best matching block to be even within the initial population. Therefore substantial amount of computational time can be saved.    4. Results and Discussion  The proposed Motion estimation was implemented in H.264 reference encoder version JM 9.3 and tested with a number of test sequences. We used existing ES based algorithm [7] to compare the results. Figure 3 compares the motion estimation time for the existing and the proposed algorithms. All simulations were carried out while keeping the PSNR constant.  It is quite clear that both algorithms provide the same compression performance. It is quite clear that the motion estimation time has been reduced without affecting the quality of the picture.   5. Conclusion In this paper we proposed and Evolutionary strategy based motion estimation algorithm for H.264 encoder. Experimental results show that the proposed algorithm can reduce the computational complexity by 50% compared to the existing [7] technique. Therefore the proposed algorithm provides a significant computational complexity improvement to the H.264 video encoder.    Reference  [1] ITU-T Recommendation H.264, “Advanced Video Coding for Generic Audiovisual Services,” May 2005. [2] Man F. So & Angus Wu, “Hardware Implementation of Four-Step Genetic Search Algorithm”, IEEE Signal Processing Society 1999 Workshop on Multimedia Signal Processing, Copenhagen, Denmark, September 1999. [3] Xu Yuelei, Bi Duyan and Mao Baixin, “ A Genetic Search Algorithm For Motion Estimation”, Proc. of 5th Int. Conf. on Signal Processing Proceedings, Beijing, China, 2000. [4] Guanghua Qiu, Chaohuan Hou, “A New Fast Algorithm for the Estimation of Block Motion Vectors”, Proc. of 3rd Int. Conf. on Signal Processing, Beijing, China, 1996. [5] Shen Li, Wei-pu Xu, Hui Wang, Nan-ning Zheng, “A Novel Fast Motion Estimation Method Based on Genetic Algorithm”, Proc. of Int. Conf. on Signal Processing, Kobe, Japan, 1999. [6] Adedoyin S., W.A.C. Fernando, H. Kodikara Arachchi and T. Kalganova, “An Efficient Motion Estimation Algorithm for H.264 Based Conversational Video Codecs,” Proc. of 8th Int. Symposium on Wireless Personal Multimedia Communications, WPMC'05, Aalborg, Denmark, June 2005. [7] Adedoyin S., W.A.C. Fernando, H. Kodikara Arachchi, T. Kalganova and K.K. Loo, “An Evolutionary Strategy Based Motion Estimation Algorithm for H.264 Video Codec,” Proc. of Canadian Conf. on Electrical and Computer Engineering, Saskatchewan, Canada, May 2005. [8] Ingo Rechenberg, Evolutionsstrategie '94. Stuttgart: Frommann-Holzboog 1994. [9] K. Ponudurai, W.A.C. Fernando and K.K. Loo, “Joint Motion and Disparity Estimation in Stereo Video Sequences Using Evolutionary Strategy,” Proceedings of 1st Regional Conference on ICT and E-Paradigms, Colombo, Sri Lanka, 2004 [10] K. Ponudurai, W.A.C. Fernando and K.K. Loo, “Joint Motion and Disparity Estimation in Stereo Video Sequences Using Evolutionary Strategy,” Proceedings of The Seventh International Symposium on Wireless Personal Multimedia Communications, Abano Terme, Italy, September 2004.    2039               (a)  0.005.0010.0015.0020.0025.0015.00 25.00 35.00 45.00 55.00QpME Time (s)Proposed Ref.[7] (b)  0.005.0010.0015.0020.0025.0030.0015.00 25.00 35.00 45.00 55.00QpME Time (s)Proposed Ref.[7]    (c) 0.005.0010.0015.0020.0025.0015.00 25.00 35.00 45.00 55.00QpME Time (s)Proposed Ref.[7]  (d) 0.005.0010.0015.0020.0025.0030.0015.00 25.00 35.00 45.00 55.00QpME Time (s)Proposed Ref.[7]   Fig 3:  Motion Estimation Time vs. Quantization Parameter Comparison for Sequences  (a) Claire (b) Carphone (c) Mother (d) Container  2040

[1] ITU-T Recommendation H.264, “Advanced Video Coding for Generic Audiovisual Services,”

[8] Ingo Rechenberg, Evolutionsstrategie '94.

An Efficient Motion Estimation Algorithm for H.264 Based Conversational Video Codecs,”

An Evolutionary Strategy Based Motion Estimation Algorithm for H.264 Video Codec,”

Bi Duyan and Mao Baixin, “ A Genetic Search Algorithm For Motion Estimation”,

Chaohuan Hou, “A New Fast Algorithm for the Estimation of Block Motion Vectors”,

Hardware Implementation of Four-Step Genetic Search Algorithm”,

Joint Motion and Disparity Estimation in Stereo Video Sequences Using Evolutionary Strategy,”

Nan-ning Zheng, “A Novel Fast Motion Estimation Method Based on Genetic Algorithm”,

Evolutionary strategy based improved motion estimation technique for H.264 video coding

Evolutionary strategy based improved motion estimation technique for H.264 video coding

Abstract

Similar works

Full text

Available Versions

Crossref

Brunel University Research Archive