Research and developments of Dirac video codec

This thesis was submitted for the degree of Doctor of Philosophy and was awarded by Brunel University.In digital video compression, apart from storage, successful transmission of the compressed video data over the bandwidth limited erroneous channels is another important issue. To enable a video codec for broadcasting application, it is required to implement the corresponding coding tools (e.g. error-resilient coding, rate control etc.). They are normally non-normative parts of a video codec and hence their specifications are not defined in the standard. In Dirac as well, the original codec is optimized for storage purpose only and so, several non-normative part of the encoding tools are still required in order to be able to use in other types of application. Being the "Research and Developments of the Dirac Video Codec" as the research title, phase I of the project is mainly focused on the error-resilient transmission over a noisy channel. The error-resilient coding method used here is a simple and low complex coding scheme which provides the error-resilient transmission of the compressed video bitstream of Dirac video encoder over the packet erasure wired network. The scheme combines source and channel coding approach where error-resilient source coding is achieved by data partitioning in the wavelet transformed domain and channel coding is achieved through the application of either Rate-Compatible Punctured Convolutional (RCPC) Code or Turbo Code (TC) using un-equal error protection between header plus MV and data. The scheme is designed mainly for the packet-erasure channel, i.e. targeted for the Internet broadcasting application. But, for a bandwidth limited channel, it is still required to limit the amount of bits generated from the encoder depending on the available bandwidth in addition to the error-resilient coding. So, in the 2nd phase of the project, a rate control algorithm is presented. The algorithm is based upon the Quality Factor (QF) optimization method where QF of the encoded video is adaptively changing in order to achieve average bitrate which is constant over each Group of Picture (GOP). A relation between the bitrate, R and the QF, which is called Rate-QF (R-QF) model is derived in order to estimate the optimum QF of the current encoding frame for a given target bitrate, R. In some applications like video conferencing, real-time encoding and decoding with minimum delay is crucial, but, the ability to do real-time encoding/decoding is largely determined by the complexity of the encoder/decoder. As we all know that motion estimation process inside the encoder is the most time consuming stage. So, reducing the complexity of the motion estimation stage will certainly give one step closer to the real-time application. So, as a partial contribution toward realtime application, in the final phase of the research, a fast Motion Estimation (ME) strategy is designed and implemented. It is the combination of modified adaptive search plus semi-hierarchical way of motion estimation. The same strategy was implemented in both Dirac and H.264 in order to investigate its performance on different codecs. Together with this fast ME strategy, a method which is called partial cost function calculation in order to further reduce down the computational load of the cost function calculation was presented. The calculation is based upon the pre-defined set of patterns which were chosen in such a way that they have as much maximum coverage as possible over the whole block. In summary, this research work has contributed to the error-resilient transmission of compressed bitstreams of Dirac video encoder over a bandwidth limited error prone channel. In addition to this, the final phase of the research has partially contributed toward the real-time application of the Dirac video codec by implementing a fast motion estimation strategy together with partial cost function calculation idea.BBC R&D and Brunel University

SVCEval-RA: an evaluation framework for adaptive scalable video streaming

[EN] Multimedia content adaption strategies are becoming increasingly important for effective video streaming over the actual heterogeneous networks. Thus, evaluation frameworks for adaptive video play an important role in the designing and deploying process of adaptive multimedia streaming systems. This paper describes a novel simulation framework for rate-adaptive video transmission using the Scalable Video Coding standard (H.264/SVC). Our approach uses feedback information about the available bandwidth to allow the video source to select the most suitable combination of SVC layers for the transmission of a video sequence. The proposed solution has been integrated into the network simulator NS-2 in order to support realistic network simulations. To demonstrate the usefulness of the proposed solution we perform a simulation study where a video sequence was transmitted over a three network scenarios. The experimental results show that the Adaptive SVC scheme implemented in our framework provides an efficient alternative that helps to avoid an increase in the network congestion in resource-constrained networks. Improvements in video quality, in terms of PSNR (Peak Signal to Noise Ratio) and SSIM (Structural Similarity Index) are also obtained.Castellanos Hernández, WE.; Guerri Cebollada, JC.; Arce Vila, P. (2017). SVCEval-RA: an evaluation framework for adaptive scalable video streaming. Multimedia Tools and Applications. 76(1):437-461. doi:10.1007/s11042-015-3046-yS437461761Akhshabi S, Begen AC, Dovrolis C (2011) An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP. In: Proceedings of the second annual ACM conference on Multimedia systems. ACM, pp 157–168Alabdulkarim MN, Rikli N-E (2012) QoS Provisioning for H.264/SVC Streams over Ad-Hoc ZigBee Networks Using Cross-Layer Design. In: 8th International Conference on Wireless Communications, Networking and Mobile Computing (WiCOM). pp 1–8Birkos K, Tselios C, Dagiuklas T, Kotsopoulos S (2013) Peer selection and scheduling of H. 264 SVC video over wireless networks. In: Wireless Communications and Networking Conference (WCNC), 2013 IEEE. pp 1633–1638Castellanos W (2014) SVCEval-RA - An Evaluation Framework for Adaptive Scalable Video Streaming. In: SourceForge Project. http://sourceforge.net/projects/svceval-ra/ . Accessed 1 May 2015Castellanos W, Guerri JC, Arce P (2015) A QoS-aware routing protocol with adaptive feedback scheme for video streaming for mobile networks. Comput Commun. http://dx.doi.org/10.1016/j.comcom.2015.08.012Castellanos W, Arce P, Acelas P, Guerri JC (2012) Route Recovery Algorithm for QoS-Aware Routing in MANETs. Springer Berlin Heidelberg, Bilbao, pp. 81–93Chikkerur S, Sundaram V, Reisslein M, Karam LJ (2011) Objective video quality assessment methods: A classification, review, and performance comparison. Broadcast, IEEE Trans on 57:165–182Choupani R, Wong S, Tolun M (2014) Multiple description coding for SNR scalable video transmission over unreliable networks. Multimed Tools Appl 69:843–858. doi: 10.1007/s11042-012-1150-9CISCO Corp. (2014) Cisco Visual Networking Index Forecast and Methodology. In: White Paper. http://www.cisco.com/c/en/us/solutions/collateral/service-provider/ip-ngn-ip-next-generation-network/white_paper_c11-481360.pdf.Dai M, Zhang Y, Loguinov D (2009) A unified traffic model for MPEG-4 and H. 264 video traces. IEEE Trans Multimedia 11:1010–1023Detti A, Bianchi G, Pisa C, et al. (2009) SVEF: an open-source experimental evaluation framework for H.264 scalable video streaming. In: IEEE Symposium on Computers and Communications. pp 36–41Espina F, Morato D, Izal M, Magaña E (2014) Analytical model for MPEG video frame loss rates and playback interruptions on packet networks. Multimed Tools Appl 72:361–383. doi: 10.1007/s11042-012-1344-1Fiems D, Steyaert B, Bruneel H (2012) A genetic approach to Markovian characterisation of H.264 scalable video. Multimedia Tools Appl 58:125–146Floyd S, Handley M, Kohler E Datagram Congestion Control Protocol (DCCP). http://tools.ietf.org/html/rfc4340 . Accessed 17 Feb 2014Floyd S, Padhye J, Widmer J TCP Friendly Rate Control (TFRC): Protocol Specification. http://tools.ietf.org/html/rfc5348 . Accessed 17 Feb 2014Fraz M, Malkani YA, Elahi MA (2009) Design and implementation of real time video streaming and ROI transmission system using RTP on an embedded digital signal processing (DSP) platform. In: 2nd International Conference on Computer, Control and Communication, 2009. IC4 2009. pp 1–6ISO/IEC (2014) Information technology - Dynamic adaptive streaming over HTTP (DASH) - Part 1: Media presentation description and segment formats.ITU-T (2013) Rec. H.264 & ISO/IEC 14496-10 AVC. Advanced Video Coding for Generic Audiovisual Services.Ivrlač MT, Choi LU, Steinbach E, Nossek JA (2009) Models and analysis of streaming video transmission over wireless fading channels. Signal Process Image Commun 24:651–665. doi: 10.1016/j.image.2009.04.005Karki R, Seenivasan T, Claypool M, Kinicki R (2010) Performance Analysis of Home Streaming Video Using Orb. In: Proceedings of the 20th International Workshop on Network and Operating Systems Support for Digital Audio and Video. ACM, New York, NY, USA, pp 111–116Ke C-H (2012) myEvalSVC-an Integrated Simulation Framework for Evaluation of H. 264/SVC Transmission. KSII Trans Internet Inf Syst (TIIS) 6:377–392. doi: 10.3837/tiis.2012.01.021Ke C-H, Shieh C-K, Hwang W-S, Ziviani A (2008) An Evaluation Framework for More Realistic Simulations of MPEG Video Transmission. J Inf Sci Eng 24:425–440Klaue J, Rathke B, Wolisz A (2003) Evalvid–A framework for video transmission and quality evaluation. In: Computer Performance Evaluation. Modelling Techniques and Tools. Springer, pp 255–272Le TA, Nguyen H (2014) End-to-end transmission of scalable video contents: performance evaluation over EvalSVC—a new open-source evaluation platform. Multimed Tools Appl 72:1239–1256. doi: 10.1007/s11042-013-1444-6Lie A, Klaue J (2008) Evalvid-RA: trace driven simulation of rate adaptive MPEG-4 VBR video. Multimedia Systems 14:33–50. doi: 10.1007/s00530-007-0110-0Moving Pictures Experts Group and ITU-T Video Coding Experts Group (2011) H. 264/SVC reference software (JSVM 9.19.14) and Manual.Nightingale J, Wang Q, Grecos C (2014) Empirical evaluation of H.264/SVC streaming in resource-constrained multihomed mobile networks. Multimed Tools Appl 70:2011–2035. doi: 10.1007/s11042-012-1219-5Parmar H, Thornburgh M (2012) Real-Time Messaging Protocol (RTMP) Specification. AdobePolitis I, Dounis L, Dagiuklas T (2012) H. 264/SVC vs. H. 264/AVC video quality comparison under QoE-driven seamless handoff. Signal Process Image Commun 27:814–826Pozueco L, Pañeda XG, García R, et al. (2013) Adaptable system based on Scalable Video Coding for high-quality video service. Comput Electr Eng 39:775–789. doi: 10.1016/j.compeleceng.2013.01.015Pozueco L, Pañeda XG, García R, et al. (2014) Adaptation engine for a streaming service based on MPEG-DASH. Multimed Tools Appl 1–20. doi: 10.1007/s11042-014-2034-ySchwarz H, Marpe D, Wiegand T (2007) Overview of the Scalable Video Coding Extension of the H.264/AVC Standard. IEEE Trans Circ Syst Video Technol 17:1103–1120. doi: 10.1109/TCSVT.2007.905532Seo H-Y (2013) An Efficient Transmission Scheme of MPEG2-TS over RTP for a Hybrid DMB System. ETRI J 35:655–665. doi: 10.4218/etrij.13.0112.0124Sohn H, Yoo H, De Neve W, et al. (2010) Full-Reference Video Quality Metric for Fully Scalable and Mobile SVC Content. IEEE Trans Broadcast 56:269–280. doi: 10.1109/TBC.2010.2050628Sousa-Vieira M-E (2011) Suitability of the M/G/∞ process for modeling scalable H.264 video traffic. In: Analytical and Stochastic Modeling Techniques and Applications. Springer, pp 149–158Tanwir S, Perros H (2013) A Survey of VBR Video Traffic Models. IEEE Commun Surv Tutor 15:1778–1802. doi: 10.1109/SURV.2013.010413.00071Tanwir S, Perros HG (2014) VBR Video Traffic Models. Wiley, HobokenThe Network Simulator (NS-2). http://www.isi.edu/nsnam/ns . Accessed 6 Feb 2015Unanue I, Urteaga I, Husemann R, et al. (2011) A Tutorial on H. 264/SVC Scalable Video Coding and its Tradeoff between Quality, Coding Efficiency and Performance. Recent Advances on Video Coding 1–24.Van der Auwera G, David PT, Reisslein M, Karam LJ (2008) Traffic and quality characterization of the H. 264/AVC scalable video coding extension. Adv Multimedia 2008:1Wang Y, Claypool M (2005) RealTracer—Tools for Measuring the Performance of RealVideo on the Internet. Multimed Tools Appl 27:411–430. doi: 10.1007/s11042-005-3757-6Wang Z, Lu L, Bovik AC (2004) Video quality assessment based on structural distortion measurement. Signal Process Image Commun 19:121–132. doi: 10.1016/S0923-5965(03)00076–6Wien M, Schwarz H, Oelbaum T (2007) Performance Analysis of SVC. IEEE Trans Circ Syst for Video Technol 17:1194–1203. doi: 10.1109/TCSVT.2007.905530YUV video repository. ftp://ftp.tnt.uni-hannover.de/pub/svc/testsequences/ . Accessed 10 Jan 201

Cross-layer Optimized Wireless Video Surveillance

A wireless video surveillance system contains three major components, the video capture and preprocessing, the video compression and transmission over wireless sensor networks (WSNs), and the video analysis at the receiving end. The coordination of different components is important for improving the end-to-end video quality, especially under the communication resource constraint. Cross-layer control proves to be an efficient measure for optimal system configuration. In this dissertation, we address the problem of implementing cross-layer optimization in the wireless video surveillance system. The thesis work is based on three research projects. In the first project, a single PTU (pan-tilt-unit) camera is used for video object tracking. The problem studied is how to improve the quality of the received video by jointly considering the coding and transmission process. The cross-layer controller determines the optimal coding and transmission parameters, according to the dynamic channel condition and the transmission delay. Multiple error concealment strategies are developed utilizing the special property of the PTU camera motion. In the second project, the binocular PTU camera is adopted for video object tracking. The presented work studied the fast disparity estimation algorithm and the 3D video transcoding over the WSN for real-time applications. The disparity/depth information is estimated in a coarse-to-fine manner using both local and global methods. The transcoding is coordinated by the cross-layer controller based on the channel condition and the data rate constraint, in order to achieve the best view synthesis quality. The third project is applied for multi-camera motion capture in remote healthcare monitoring. The challenge is the resource allocation for multiple video sequences. The presented cross-layer design incorporates the delay sensitive, content-aware video coding and transmission, and the adaptive video coding and transmission to ensure the optimal and balanced quality for the multi-view videos. In these projects, interdisciplinary study is conducted to synergize the surveillance system under the cross-layer optimization framework. Experimental results demonstrate the efficiency of the proposed schemes. The challenges of cross-layer design in existing wireless video surveillance systems are also analyzed to enlighten the future work. Adviser: Song C

Cross-layer Optimized Wireless Video Surveillance

A wireless video surveillance system contains three major components, the video capture and preprocessing, the video compression and transmission over wireless sensor networks (WSNs), and the video analysis at the receiving end. The coordination of different components is important for improving the end-to-end video quality, especially under the communication resource constraint. Cross-layer control proves to be an efficient measure for optimal system configuration. In this dissertation, we address the problem of implementing cross-layer optimization in the wireless video surveillance system. The thesis work is based on three research projects. In the first project, a single PTU (pan-tilt-unit) camera is used for video object tracking. The problem studied is how to improve the quality of the received video by jointly considering the coding and transmission process. The cross-layer controller determines the optimal coding and transmission parameters, according to the dynamic channel condition and the transmission delay. Multiple error concealment strategies are developed utilizing the special property of the PTU camera motion. In the second project, the binocular PTU camera is adopted for video object tracking. The presented work studied the fast disparity estimation algorithm and the 3D video transcoding over the WSN for real-time applications. The disparity/depth information is estimated in a coarse-to-fine manner using both local and global methods. The transcoding is coordinated by the cross-layer controller based on the channel condition and the data rate constraint, in order to achieve the best view synthesis quality. The third project is applied for multi-camera motion capture in remote healthcare monitoring. The challenge is the resource allocation for multiple video sequences. The presented cross-layer design incorporates the delay sensitive, content-aware video coding and transmission, and the adaptive video coding and transmission to ensure the optimal and balanced quality for the multi-view videos. In these projects, interdisciplinary study is conducted to synergize the surveillance system under the cross-layer optimization framework. Experimental results demonstrate the efficiency of the proposed schemes. The challenges of cross-layer design in existing wireless video surveillance systems are also analyzed to enlighten the future work. Adviser: Song C

Complexity management for video encoders.

Software implementation of block-based video coding standards has been used in a wide range of applications. In many cases, such as real-time multimedia systems or power-constrained systems, the coding performance of software-only video encoders and decoders is limited by computational complexity. This thesis presents research work to develop techniques to manage computational complexity of video encoders. These techniques aim to provide significant complexity saving as well as adaptively controlling the computational complexity. This thesis first investigates experimentally the most computationally intensive functions in a video encoder. Based on the results of profile tests, several functions are selected as candidates, on which complexity reduction algorithms will be performed. These functions include discrete cosine transform and related functions as well as motion estimation. Adaptive complexity-reduction algorithms are proposed for computationally expensive functions: discrete cosine transform and motion estimation functions respectively. It is shown that these algorithms can flexibly control the computational complexity of each function with negligible loss of video quality. The inherent characteristics of coded macroblocks are investigated through experimental tests and they are categorized into "skipped" and" unskipped" macroblocks based on two parameters. An innovative algorithm is developed to reduce the computational complexity by predicting "skipped" macroblock prior to encoding and not carrying out the coding process on these macroblocks. The approaches described in this thesis can not only achieve adaptive control of the computational complexity of a video encoder, but also can manage the trade-off between complexity and distortion. These proposed algorithms are evaluated in terms of complexity reduction performance, rate-distortion performance and subjective and objective visual quality by experimental testing.The advantages and disadvantages of each algorithm are discussed

Motion estimation and CABAC VLSI co-processors for real-time high-quality H.264/AVC video coding

Real-time and high-quality video coding is gaining a wide interest in the research and industrial community for different applications. H.264/AVC, a recent standard for high performance video coding, can be successfully exploited in several scenarios including digital video broadcasting, high-definition TV and DVD-based systems, which require to sustain up to tens of Mbits/s. To that purpose this paper proposes optimized architectures for H.264/AVC most critical tasks, Motion estimation and context adaptive binary arithmetic coding. Post synthesis results on sub-micron CMOS standard-cells technologies show that the proposed architectures can actually process in real-time 720 × 480 video sequences at 30 frames/s and grant more than 50 Mbits/s. The achieved circuit complexity and power consumption budgets are suitable for their integration in complex VLSI multimedia systems based either on AHB bus centric on-chip communication system or on novel Network-on-Chip (NoC) infrastructures for MPSoC (Multi-Processor System on Chip

Distributed video coding for wireless video sensor networks: a review of the state-of-the-art architectures

Distributed video coding (DVC) is a relatively new video coding architecture originated from two fundamental theorems namely, Slepian–Wolf and Wyner–Ziv. Recent research developments have made DVC attractive for applications in the emerging domain of wireless video sensor networks (WVSNs). This paper reviews the state-of-the-art DVC architectures with a focus on understanding their opportunities and gaps in addressing the operational requirements and application needs of WVSNs

Optimal packetisation of MPEG-4 using RTP over mobile networks

The introduction of third-generation wireless networks should result in real-time mobile video communications becoming a reality. Delivery of such video is likely to be facilitated by the realtime transport protocol (RTP). Careful packetisation of the video data is necessary to ensure the optimal trade-off between channel utilisation and error robustness. Theoretical analyses for two basic schemes of MPEG-4 data encapsulation within RTP packets are presented. Simulations over a GPRS (general packet radio service) network are used to validate the analysis of the most efficient scheme. Finally, a motion adaptive system for deriving MPEG-4 video packet sizes is presented. Further simulations demonstrate the benefits of the adaptive system