A parallel H.263 video encoder, which utilises spatial para1 elism,
has been modelled using a multi-threaded program. Spatial
parallelism is a technique where an image is subdivided into equal
parts (as far as physically possible) and each part is proces!;ed by
a separate processor by computing motion and texture mding
with all processors cach acting on a different part of thc ]mag.
This method leads to a performance increase, which is roughly in
proportion to the number of parallel processors used

Cosmas, J

Paker, Y

Pearmain, A

Electronics Letters

English

Brunel University Research Archive

Parallel H.263 video encoder in normal coding mode thread 0 encoder (intra,inter & entropy coding) J.P.  Cosmas, Y. Paker and A.J. Pearmain thread 1 thread 2 encoder encoder (intrajnter & (intra,inter & entropy coding) entropy coding) ~ A parallel H.263 video encoder, which utilises spatial para1 elism, has been modelled using a multi-threaded program. Spatial parallelism is a technique where an image is subdivided into equal parts (as far as physically possible) and each part is proces!;ed by a separate processor by computing motion and texture mding with all processors cach acting on a different part of thc ]mag .  This method leads to a performance increase, which is roughly in proportion to the number of parallel processors used. 4 b a  Introduction: In the coding of images using parallel techniques there are two main approaches: functional parallelism and spatial parallelism. With functional parallelism a separate unit in a .)aral- le1 pipeline computes each function in the coding algorithm [l]. Using this method it is difficult to reassign processing power when there is a large variation in the processing power requiremeni s at a functional level and impossible if dedicated hardware is used. With spatial parallelism, an image is subdivided into equal parts, which are processed by separate processors [2, 31. Spatial parallelism leads to more equal distribution of the procssing power required among the available parallel processors and is also able to redistribute the processing load by reassigning larger/ smaller areas of the image to the processors. This makes the spa- tial parallel technique ideal for increasing the processing power of the H.263 encoder 141 which is required to operate on up to five standard picture formats (e.g. sub-QCIF, QCIF, CIF 4CIF and 16CIF). A b a  b a  the inacroblocks (horizontal and vertical) are coded differentially by using a spatial neighbourhood of three motion vectors already computed [3]. These three motion vectors are candidate predictors for the differential coding. Therefore, to perform motion predic- tion in parallel on each horizontal region of an image, motion esti- mation is performed on each region plus the row of macroblocks inmediately above the region. Motion prediction is then com- puted only for the given region using the motion information of the blocks immediately above for the macroblocks at the bound- aIy. In normal mode, the symbols generated by the intra and inter-coding are Huffman coded. Since Huffman coding of a par- t i d a r  symbol is independent of previous symbols then the mac- roblocks within each region can be coded separately and combined by simply collating the bits, taking care to exclude any padding that may have been added. When coding the new image the fully reconstructed image is passed back from the collator to the separate threads for coding. For k = 1 to 3 Start Thread k Get Images For k = 1 to 3 Image Ready k TRUE Wait for Image Coded 1 to 3 Collate bits For ever loop { Get Images For k = 1 to 3 Image Ready k =TRUE Wait for Image Coded 1 to 3 For k = 1 to 3 Image Coded k = FALSE Collate bits Code Interllntra 1 Release mutex Main() { thread(n) ( Wait for Image Ready n Image Ready n = FALSE Set Mutex Code Intra Image Coded n = TRUE Release Mutex For ever loop ( Wait for Image Ready n Image Ready n = FALSE Set Mutex image Coded n = TRUE 1 1 1 Fig. 2 P.vr~idolocodc~ u j  sojtir~iw nzulti-thread& model of' parallel sj'steni 3 4 Algorithm enhancements: The key issue to parellelising the H.263 encoder is whether the macroblock tcxture and motion coding algorithms require picture element (pel) information from .Nithin the macroblock that is being coded or whether additional pel information from outside is also required. If the former is true then the macroblocks can be shared among the availablc proces- sors whereas if the latter is true then it is much more diffidt  to share the macroblocks among the available processors. A parallel image-coding scheme has been designed so that the image is split as equally as possible into horizontal regions along macroblock boundaries in order to minimise data dependency between blocks. n ~ - - - - ~  intra 0.08 0.17 0.12 1.14 1.29 intra 0.08 0.12 0.12 0.09 2.19 1.88 0 camera thread 3 bits coilator i m, Fig. 1 PuidIeI ryrrenr diagram LI bntropy codcd bits, reconstructed regional im'ige 1~ Fully reconstructed image New imdge When using INTRA mode coding, all the pixels &,at are rcquired for processing the DCT lie within the 16 x 16 inacroblock that is being computed. This poses no problems for the l~arallel coding since the four, luminance and two, chrominance 8 x 8 blocks lie within the macroblock boundary. When using INTER mode coding in noma1 modc, the motion vector components of ELECTRONICS LETTERS 29th October 1998 Vol. 34 Results: The parallel image coding has been implemented using 2, 3 and 4 parallel threads, each with an additional thread for re-col- kiting the output bitstream. The structure of the parallel system is shown in Fig. 1 and the software implementation is shown in Fig. 2. The processing power of the CPU was directed towards each thread in sequence by using inutexes within each thread. Mutexes allow all the processing power to be directed towards one thread while all other threads are put to sleep and consume minimal processing power. Synchronisation of the threads and mutexes are co-ordinated using flags: Image Ready and Image Coded, as shown in Fig. 2. Tests were made on 35 frames of QCIF images for the Newsreader scene and 35 frames of CIF images for the Children scene using a 200MHz Pentium PC with 64 Mbyte of RAM. The results of the speed-up are given in Tables 1 ~ 4. Table 1: Results of intra-coding speed-up (QCIF images of News- reader scene) 'Table 2 Results of inter-coding speed-up (QCIF images of News- reader scene) The time required to collate the bits was negligible and there- Fore was not tabulated. For each thread, the average time to code il horizontal region was measured over a sequence of 35 frames. 'The average speed-up was computed as the time to code the images using a single thread divided by the average time to code No. 22 2109 Number of Coding Thread 1 Thread 2 Thread 3 threads type AV. time AV. time AV time S S S Thread 4 Average Worst-case AV. time speed-up speed-up S Table 4 Results of inter-coding speed-up (CIF images of Children scene) 1 2 3 4 intra 0.78 1.00 1.00 intra 0.46 0.62 1.44 1.13 intra 0.261 0.41 0.461 2.07 1.69 intra 0.26 0.19 0.14 0.36 3.28 2.16 Number of Coding Thread 1 Thread 2 Thread 3 Thread 4 threads type AV. t h e  AV. lime AV. time AV. time S s S S Conclusions; The speed-up for a spatially parallel H.263 video encoder was in proportion to the number of threads used but not as great as expected for CIFIQCIF images for both inter- and intra-modes. This was due to additional overhead time to initialise parameters and to allocate temporary arrays that was repeated for each image region. The worst-case speed-up shows what can be obtained using multi-threads on a single processor but the paral- lelisation algorithms would really show an improvement in per- formance if run on multiple processors particularly if inter- processor comnunication techniques using DMA or shared mem- ory are used. The average speed-up shows what can be obtained when load-sharing algorithms are in place. For the speed-up to be improved, further work is needed to remove unnecessary alloca- tionlreallocation of memory space and to balance the load among the threads. Average Worst-case speed-up speed-up 0 IEE 1998 Electronics Letters Online No.  19981487 J.P. Cosmas, Y .  Paker and A.J. Pearmain (Department of Electrical Engineering, Queen Mary & Westfield College, London E l  4NS, United Kingdom) 16 September 19% 1 2 3 4 Refcrences inter 0.95 1.00 1.00 inter 0.56 0.76 1.44 1.25 inter 0.37 0.69 0.65 1.67 1.37 inter 0.35 0.43 0.35 0.58 2.22 1.63 1 LIN,  W, GOH, K H ,  TYE, B.J., POWELL, G . A ,  OHYA. T., and ADACHI. S.: ‘Real time H.263 video codec using parallel DSP’. IEEE Int. Conf. Image Processing, 1997, Vol. 2, pp. 586-589 2 AKRAMULLAH. A., AHMAD, I ,  and LIOU, M L : ‘A data parallel approach for real time MPEG-2 video encoding’, J.  Pnrullel Distrilut. Comput., 1995, 30, pp. 129-146 and KLEFSTAD, M : ‘A flexible MPEG real time video codec’. ICIP- 96, Lausanne, Switzerland, September 1996, Vol. 3, pp. 829-832 Draft ITU-T Recommendation H.263. ‘Video coding for low bit- rate communication’. March 1996 3 BOUVILLE, C ,  HOULIkR, P I  DUBOIS, J L., MARCHAL, I ,  THERAULT. B , 4 Reduction of ringing noise in transform image coding using simple adaptive filter A. Kaup A numerically simple filter for reducing ringing noise in transform coded images is proposed. The filter adapts to the local image characteristics and has been specifically designed to amend the current deblocking filter in H.263. It is shown that by adding the proposed filter both subjective and objective image quality improve. Introduction: A well-known deficiency of current video coding standards is the visibility of image degradations at high compres- sion ratios. These image degradations manifest themselves in blocking artefacts due to the rigid block partitioning of the image and ringing noise preferably around edges due to coarse quantisa- tion. Both effects are visually very annoying and have a substan- tial impact on the subjectively perceived image quality. A widespread principle for overcoming this problem is lowpass filtering of the decoded image in either the spatial [l] or temporal direction 121. To reduce the numerical complexity, these filters are sometimes restricted to block boundaries, thus specifically tackling blocking noise. A very efficient filter of this type has been stdnd- ardised and included as optional Annex J in the recently released Version 2 of H.263 [3]. However, while this filter removes much of the blocking noise it does not tackle ringing and mosquito noise. Since global smoothing for reducing ringing artefacts tends to eliminate important image details, some proposals try to enhance the decoded image by incorporating prior knowledge about typical image data. This leads to maximum a posteriori (MAP) approaches where the Bayesian paradigm can be used to solve an estimation problem involving both a priori knowledge and the decoded iinage data [4, 51. However, since this estimation process usually involves numerical optimisation of non-convex functionals, the principle is computationally very demanding. Apart from image restoration after decoding it is also possible to diminish blocking and ringing noise by image preprocessing at the encoder site. This train of thought has for example, been fol- lowed in [6], where the quantisation noise of DCT coefficients is shifted from the block boundaries to the inner part of the block. Other proposals employ Dolby-like noise suppression techniques for reducing blocking artefacts [7]. Although such noise shaping requires a matched receiver for best performance, a standardised receiver which does not know about the encoder modifications can still decode an image of reasonable quality. In this Letter we will follow a different approach motivated by the recently released deblocking option in H.263. This filter has been placed within the prediction loop such that the decoded and filtered image serves as reference for the next frame to code. While this makes the filter description normative, it has the advantage that only single frame storage is needed for prediction as well as display. Following this reasoning, we will describe a possible amendment to this option which specifically takes care of the remaining ringing and mosquito noise. Deringing Jilter: Consider a motion compensated reconstructed image in the prediction loop of H.263 after the deblocking filtering as described in Annex J of [3] has taken place. While the resulting image typically has only very little blocking noise remaining at block boundaries, it does still show considerable ringing artefacts especially towards the centre of the image blocks. A deringing fil- ter thus should remove this noise without unduly destroying important high frequency iinage details. This can be achieved by an adaptive lowpass filter where the filter inask varies depending on the local image characteristics. Table 1: Local 3 x 3 neighbourhood considered for filtering Consider a local 3 x 3 neighbourhood of decoded image pixels as depicted in Table 1 having the grey levels g, to g,. Grey level here refers to either luminance or chrominance data. The deblock- 21 10 ELECTRONICS LETTERS 29th October 1998 Vol. 34 No. 22 

Parallel H.263 Encoder in Normal Coding Mode

Abstract

Similar works

Full text

Available Versions

Brunel University Research Archive