CHRISTIAENS, M

Devos, Harald

EECKHAUT, H

FAES, P

Stroobandt, Dirk

English

Ghent University Academic Bibliography

RESUME’s wavelet-based scalable video decoderHendrik Eeckhaut Mark Christiaens Harald Devos Philippe FaesDirk StroobandtHendrik.Eeckhaut@elis.UGent.beParallel information systems - ELIS - Ghent University - Belgiumhttp://www.elis.ugent.be/resumeAbstractIn the RESUME project we developed a real-time FPGAprototype of a fully scalable, wavelet based video decoderwhich overcomes the complexity and bandwidth issues as-sociated with scalable video.1 IntroductionThe finished RESUME-project (Reconfigurable Embed-ded Systems for Use in Scalable Multimedia Environments)explored the benefits of using reconfigurable hardware forthe implementation of scalable multimedia applications bybuilding an FPGA implementation of a real-time, scalable,wavelet-based video decoder. The term scalable videorefers to a coding scheme that can easily accommodatechanges in quality of service (QoS) without the need fortranscoding. A scalable video stream can be decoded atvarying frame rates, resolutions and image qualities bysimply skipping redundant parts in the video stream, onlydecoding those parts that will contribute to the displayedvideo. The algorithmic structure of the RESUME scalablevideo coder and decoder (codec) is shown in Figure 1 andis described in [2].Such a scalable video codec has advantages for both theserver (the provider of the content) and the clients. Onthe one hand the server scales well since it has to produceonly one encoded video stream that can be broadcasted toall clients, irrespective of their QoS requirements. On theother hand the client (or the network) can easily adapt thedecoding parameters to its needs. This way it is possible tooptimize the use of the network, display, the required pro-cessing power, the required memory, ...Scalability has a lot of advantages but comes at a cost.The decoding algorithm is computationally complex and re-ally stresses the system bandwidth as it replaces the block-based DCT-approach with frame-based wavelets. This hasa tremendous impact on the hardware architecture.Motion EstimationWavelet TransformEntropy CodingMotion CompensationInverse Wavelet TransformEntropy DecodingPackUnpackAdapt bitstream to QoSOriginalframesDecodedframesTemporal ScalabilityResolutionScalabilityResolution &QualityScalabilityFigure 1. High-level overview of the codec.2 Hardware implementationWe implemented the complete decoding pipeline of ourcustom wavelet-based scalable video codec on a PCI de-velopment board equipped with a Stratix FPGA S60 and256 MiB of DDR SDRAM memory. The FPGA board isplugged into a standard PC with two monitors, one dedi-cated to displaying the decoded video, the other to interactwith the system (Figure 2). The design goals were real-time,lossless decoding of CIF-sequences (352×288 pixels) at 25frames per second.Implementing a complete video codec is a complex un-dertaking that requires careful planning. We applied the fol-lowing methodology. First the entire software code basewas cleaned-up and we made sure that the algorithms usedwere properly understood. We chose to use a HW/SW-codesign approach leaving as much of the algorithm as pos-sible in SW running on a CPU while implementing thetime-critical parts in reconfigurable HW. The hardware de-sign was implemented using Altera’s SOPC (System-On-a-Programmable-Chip) Builder for component-based systemintegration. For each of the steps in the decoding pipelinewe developed custom components to deliver the requiredhardware acceleration (Figure 3). Most components dif-Figure 2. Photograph of our wavelet-basedscalable video decoder in action.EntropyDecoderVGADDRPCICPU InverseWaveletMotionComp.Direct (master) transfer DMA (slave) transferFigure 3. Simplified hardware overview.fer substantially from others with respect to their maximumclock frequency. To accommodate this, the design is sub-divided into multiple clock domains. Thanks to the use ofSOPC Builder this can be achieved fairly easily by assign-ing different clocks to each of the components.Most components of the design were highly optimized toachieve real-time decoding. The wavelet entropy decoder,very similar to AVC’s CABAC, was exhaustively elabo-rated until it produced one decoded symbol per clock cy-cle by applying multilevel speculation and pipelining [2].We also developed an automatic IDWT generator based onpolyhedral techniques (loop transformations) [1]. This re-sulted in a line-based IDWT especially tailored to the spe-cific access pattern of the external on-board DDR-memory.For all development (hardware and software) we applied awrite-tests-first strategy and used well founded engineeringtechniques as code reuse, refactoring, regression testing andcontinuous integration to continuously guard the quality ofthe entire design.The major bottlenecks in the design were the limitedbandwidth of the PCI-bus and the DDR-memory. Trad-ing in the conventional block-based DCT-transform forthe wavelet-transform results in calculations on larger datablocks: wavelet sub bands instead of macro blocks. FPGAsno longer house sufficient internal memory to buffer the in-termediate results of the decoding processes, forcing off-chip buffering. Of course, this results in large bandwidthrequirements between the external memory and the FPGA.To alleviate this problem, much effort has been put in opti-mizing this communication through the use of DMA trans-fers in efficient burst mode.Now that we have a design capable of lossless decod-ing, we can investigate the options of reconfiguration withsmaller or slower designs to adapt the hardware resource us-age to lower quality requirements. This way we can effec-tively link the video scalability to real hardware scalability.3 ConclusionsSeveral properties of this design lead us to believe whyyou should certainly visit our demo at the University Booth.Firstly, to our knowledge, we present the first completehardware design of a real-time, fully scalable wavelet-basedvideo decoder. Secondly, the design was performed usingstate-of-the-art tools such as Altera’s SOPC Builder (for acomponent-based design), the WRaP-IT/URUK tool-suite(for automatic loop transformations) and CLooGVHDL (forautomatically generating VHDL from loop nests). Thirdly,some components in the design, such as the inverse discretewavelet transform and the arithmetic decoder, are highlyoptimized. Compared to a software implementation at 2GHz (AMD64 3500+) the FPGA-implementation at ap-proximately 55 MHz is 3 times faster. Finally, the quality ofthe entire design was carefully guarded using a regressionframework based on the best practices from the softwareworld (using Apache Maven and Continuum). At the Uni-versity Booth we will demonstrate real-time, scalable videodecoding on our FPGA prototype.AcknowledgmentThis research is supported by the I.W.T. Vlaanderen,grant 020174, the F.W.O., grant G.0021.03, GOA project12.51B.02 of Ghent University and the Altera universityprogram. P. Faes is supported by a PhD grant from theI.W.T. Vlaanderen.References[1] H. Devos, K. Beyls, M. Christiaens, J. Van Campenhout, E. H.D’Hollander, and D. Stroobandt. Finding and applying looptransformations for generating optimized FPGA implementa-tions. Transactions on High Performance Embedded Archi-tectures and Compilers, LNCS 4050, 1(1):151–170, 2007.[2] H. Eeckhaut, M. Christiaens, H. Devos, and D. Stroobandt.Implementing a hardware-friendly wavelet entropy codec forscalable video. In Proceedings of SPIE: Wavelet Applicationsin Industrial Processing III, volume 6001, pages 169–179,October 2005.2

RESUMEs wavelet-based scalable video decoder

https://biblio.ugent.be/publication/416628/file/447972.pdf

RESUMEs wavelet-based scalable video decoder

Abstract

Similar works

Full text

Available Versions

Ghent University Academic Bibliography