We present Xstream-x264: a real-time cross-layer video streaming technique implemented within a well known open-source H.264 video encoder tool x264. Xstream-x264 uses the transport protocol provided indication of the available data rate for corresponding adjustments in the video encoder.We discuss the design, implementation and the quality evaluation methodology utilised with our tool.We demonstrate via experimental results that the streaming video quality greatly improves with the presented cross-layer approach both in terms of lost frame count and the objective video quality metrics Peak Signal to Noise Ratio (PSNR)

Boreli, Roksana

Lochin, Emmanuel

Sarwar, Golam

English

Open Archive Toulouse Archive Ouverte

Xstream-x264: Real-time H.264 Streamingwith Cross-layer IntegrationGolam Sarwar and Roksana BoreliNational ICT Australia (NICTA); University of NewSouth Wales (UNSW){golam.sarwar,roksana.boreli}@nicta.com.auEmmanuel LochinUniversité de Toulouse; UPS, INSA, INP, ISAE, LAAS;F-31077 Toulouse, Franceemmanuel.lochin@isae.frAbstractWe present Xstream-x264: a real-time cross-layer video streamingtechnique implemented within a well known open-source H.264video encoder tool x264. Xstream-x264 uses the transport protocolprovided indication of the available data rate for corresponding ad-justments in the video encoder. We discuss the design, implementa-tion and the quality evaluation methodology utilised with our tool.We demonstrate via experimental results that the streaming videoquality greatly improves with the presented cross-layer approachboth in terms of lost frame count and the objective video qualitymetrics Peak Signal to Noise Ratio (PSNR).Keywords video encoding, streaming, H.264, DCCP, MPEG4,cross layer1. Introduction and MotivationOnline measurement company ComScore1 estimates that on aver-age, Internet users viewed 30 billion online videos per month in2009, making video streaming one of the most pervasive applica-tions on the Internet. Streaming video quality is highly dependenton the video codec data rate and impairments like frame losses [1],which may be due to congestion or change in the network used dueto mobility.The most commonly used protocol for multimedia communica-tions is Real-time Transport Protocol (RTP) [2], used in conjunc-tion with UDP to provide full transport protocol functionality, al-though it may also be used with other transport protocols like TCP.When used with Real-time Transport Control Protocol (RTCP) [2],RTP will also provide control and quality information. Addition-ally, RTP may have specific media related congestion control usingUDP. Two other not widely used multimedia transport protocolsinclude the Stream Control Transmission Protocol (SCTP) [3] andDatagram Congestion Control Protocol (DCCP) [4]. All of thesetransport protocols, with the exception of UDP, have their respec-tive methods for reacting to losses and congestion on the link. Var-ious link quality metrics may also be available in the lower lay-ers. However, multimedia applications do not have any functionalmeans to take advantage of this information and react by changingthe encoding parameters. As a consequence, the application requir-ing a larger capacity than what is provided by the link and/or trans-port layers will experience packet losses. For the user, the visibleor audible consequence will be a decrease of the perceived quality.There has been a body of research in cross layer approacheswhich have demonstrated the potential of video streaming basedon cross-layer information to improve the quality of end user’sviewing experience [7–10]. Most of these proposals contain sim-ulated results of potential improvements, due to the lack of freely1 http://www.comscore.comavailable tools which can accept cross layer information, encodevideo according to that input and stream the encoded video to a re-ceiver. EvalVid [11] and H.264/AVC JM Reference Software [12]are the most commonly used freely available video encoding eval-uation tools today. MPEG4IP [13] also provides similar function-ality. JM software supports only H.264 reference encoding, whileEvalVid and MPEG4IP support off-line encoding and streaming,requiring the user to first encode the video and then, in the secondstep, stream it. However, none of these tools provides the crucialcapability of encoding and streaming at the same time and moreimportantly, they do not allow the encoder to adjust the target en-coding rate and other encoding quality control parameters basedon external inputs. In this paper, we present a technique to utilizeexternal information and a tool for a cross-layer adaptable H.264video coding and streaming and demonstrate it’s benefits to videoquality on congested links.2. Design of the Cross Layer Video ToolH.264/MPEG-4 is a widely used standard for video compressiontoday [5]. H.264 video, when streamed over congested communi-cation channels, will incur a loss of quality if video frames are lostduring transmission [1]. For real-time multimedia streaming, thiscreates a bigger problem as in-time delivery is crucial and relia-bility achieved using retransmissions may not satisfy the in-timerequirement. To minimize losses and maximize end users’ viewingexperience, we propose a simple and efficient cross layer techniquewhich adjusts the parameters of the ongoing video encoding pro-cess based on external information (eg. throughput, loss informa-tion about the link found in transport protocol).The most common H.264 parameters specified for the encodingprocess are either a user-specified constant quality or a constant bit-rate (CBR) [6]. We note that CBR encoding results in all framesbeing encoded with the same value of the maximum target bitrate, while constant quality, which effectively is VBR encodingwill result in a data rate which is proportional to the degree ofcomplexity and motion in the video. In our proposed technique,the encoder effectively encodes with both a variable quality anda variable rate by following an externally provided rate based oncross layer information. The encoding process changes the ratewhen a change in the available bandwidth is detected, due to loss orcongestion. This allows the application to conform to the availablebandwidth on the link resulting in a lower number of frame lossesand provides an improved viewing experience.We have implemented the proposed technique in a tool, by mod-ifying an open source H.264 encoder x264 [6]. The implementationalso includes a real time encoding and streaming capability and ad-ditional features which aid the evaluation of video quality when thereceived video has frame losses.The functional components of our Xstream-x264 tool are shownin Figure 1.Figure 1: Design of Xstream-x264The analog video source is encoded into H.264 frames based onthroughput or link capacity information provided transport protocol(although other external parameters may also be used from otherlayers in OSI network stack). Meta-data, including frame sequencenumber and fragmentation sequence number is appended to theencoded frame (a total of 4 bytes per frame), which is then streamedwith timing corresponding to the source video. The H.264 video issent over the link of choice and captured by the video sink. Thisvideo is passed through the loss detection module we have alsoimplemented, followed by the computation of video quality.To evaluate the received video quality, we use Peak Signal toNoise Ratio (PSNR), the most widely used objective video qualitymetric standardized by ITU [14]. Although PSNR may not alwaysaccurately estimate the subjective video quality due to non-linearnature of the human visual assessment, studies have shown that itdoes provide a reasonable estimate of the subjective assessment ifit is applied over the same video content under the same environ-ment [15]. We use the Compare component of ImageMagick [16]tool-set for PSNR computation, in conjunction with our frame lossdetection module. To enable PSNR computation using two streamswhich have an unequal number of video frames, for any detectedlost frame, we insert a blank (all white) frame. The existing play-back concealment techniques [17] can still be utilized in additionto our proposal and should further improve the results in terms ofvisual perception. We have however chosen not to use them in ourexperiments, in order to more clearly show the benefits of our pro-posal.Our tool can currently be used with either UDP/RTP or DCCPwith Congestion Control ID 3 (CCID3) [4]. DCCP was targetedas it is a good transport protocol candidate for future multimediaapplications, with CCID3 specifically designed to work with dataand video transmission [4].3. Experiments and AnalysisTo demonstrate the capability of Xstream-x264, in this section wepresent experimental results showing the effects of congestion. Thesimple network topology used in the experiments includes a source(video sender) and a sink (video receiver), connected by a router.Congestion is emulated in the router by using the Linux networkemulator NetEm rather than additional traffic streams. The nominallink bandwidth is 1 Mbit/s and the Round-Trip-Time (RTT) is10ms for all the experiments. All experiments are of 42 secondsduration and the artificial congestion is introduced by reducingthe link bandwidth to 700 Kbit/s after 15 seconds; after the 30thsecond, the link bandwidth is further decreased to 400 Kbit/s. TheMaximum Transmission Unit (MTU) of the network is set to 1400bytes. If an encoded frame is larger than the MTU, the videoframe is fragmented and transmitted in subsequent packets withoutany intermediate frame delay. Otherwise, the encoded frames aretransmitted in real time, with respect to the intra-frame delay of theraw input video. The analog input video used in our experiment isa 41.66 seconds, 1000 frames long scene from a video clip of anaction movie with a very large amount of motion. The input videoplays at 23.976 frames per second (FPS).We compare the calculated PSNR values for our proposed crosslayer mechanism and a CBR encoded video stream, transmittedover the congested link. We use DCCP transport protocol to pro-vide the cross-layer information about the link to the video appli-cation, we also use DCCP for the CBR experiments, with a con-stant bit rate (CBR) of 1 Mbit/s. To present a fair evaluation ofhow our proposed method utilizes the rate information provided byDCCP, we also perform experiments for an ideal scenario wherethe application knows the available rate on the link and changes theencoding parameters according to this presumed knowledge. Theresulting PSNR values are shown in Figure 2, together with the con-gestion pattern used in the experiments. As can be expected, CBR(1 Mbit/s), which is not well suited to the available capacity of thelink, experiences significant losses in the congested period and asa consequence, DCCP-CCID3 (which has a loss and RTT basedcongestion control mechanism [4]) severely restricts the rate avail-able to the application. On the other hand, our proposed cross-layerVBR technique attempts to follow the available rate as advised bythe transport protocol, thereby the result is much lower frame loss.The only loss is due to the frequency of DCCP rate change and thelimitation of the application to only change the encoding rate ona per-frame basis. The ideal case where the application knows theexact rate rather than the DCCP information about the rate showsthe highest PSNR. 0 10 20 30 40 50 60 0  100  200  300  400  500  600  700  800  900  1000 0 100 200 300 400 500 600 700 800 900 1000 1100 1200P SNR  ( d B)L in k Ba nd wi dt h ( kb i t/ s )Frame no.Link bandwidthPSNR − Ideal ScenarioPSNR − Cross Layer DCCPPSNR − Standard DCCPFigure 2: Comparison of PSNR for an ideal scenario, proposedcross-layer DCCP and standard DCCP streaming.The analysis of the PSNR values for the for ideal scenario,proposed cross layer DCCP (averaged over all possible initial ratesfrom 100 Kbit/s to 1 Mbit/s) and standard DCCP is shown in Table1. We present the mean and coefficient of variation (CV, ratio ofstandard deviation to mean) for the full experiment and, separately,for the three congestion regions.To further demonstrate the advantages of the adaptive mecha-nism, we also show the number of lost frames from the same exper-iments in Table 2. Please note that the ideal scenario has no losses,therefore it is omitted. Frame losses are separated into losses for theindex (I), predictive (P) and bi-directional predictive (B) frames. InH.264, the I frames provide reference and necessary decoding in-formation for the P and B frames and therefore lost I frames have agreater impact on quality. As can be seen in the tables, by adaptingto our cross layer technique, not only we have reduced the overallnumber of lost frames, we have also cut down the number of lost Iframes which guarantees improved video playback for the viewer.1 Mbit/s 700 Kbit/s 400 Kbit/sOverallMean(dB)Mean(dB)CV(dB)Mean(dB)CV(dB)Mean(dB)CV(dB)IDEAL 35.20 33.60 0.08 38.68 0.11 32.77 0.11Cross-layerDCCP33.62 30.96 0.27 37.30 0.20 32.28 0.16StandardDCCP19.83 28.95 0.36 22.54 0.72 4.63 0.55Table 1: Detailed analysis of PSNR from Figure 2.EncodingBit rate(Kbit/s)% Totalloss% frameloss (I)% frameloss (P)% frameloss (B)Cross-layerDCCP6.21 0.20 2.30 3.70StandardDCCP47.75 0.60 25.13 21.82Table 2: % Frame Loss for Cross Layer DCCP and Standard DCCP(1 Mbit/s CBR).We further extend our work by emulating the standard YUVvideo sequences provided by Xiph Foundation [18] and ArizonaState University [19] for multimedia research. Videos of differentresolution and motion complexity are chosen for cross layer basedDCCP and standard DCCP streaming. All videos are encoded at23.976 frames per sec (FPS). Standard DCCP streaming is per-formed at 1 Mbit/sec. Emulations are performed under the samecongestion environment shown in Figure 2. Performance of crosslayer DCCP using Xstream-x264 and standard DCCP streaming interms of number of lost frames are presented in Table 3. As can beseen in Table 3, cross-layer based DCCP streaming with Xstream-x264 significantly reduces number of lost frames and shows notableimprovement in all cases.In Table 4, we present detailed analysis of lost I, P and Bframes for the experiments with standard YUV video sequencespresented in Table 3. Cross-layer based streaming significantlyreduces the number dropped I frames which substantially helpsimprove streaming video quality experience in a congested networkenvironment.We illustrate the visual quality difference in Figure 3 by compar-ison of a selected video frame from two sources: as decoded fromthe stream transmitted and received by our proposed cross layer ap-proach and decoded from the 1 Mbit/s CBR received stream. It canbe observed that the image in Figure 3b appears significantly moredistorted than the image from Figure 3a.VideoNameResolutionof theYUVVideo Se-quencesMotionCom-plexity% TotalLostFrames -StandardDCCP%TotalLostFrames-CrossLayerDCCPForeman 352 x 288 High 34.2 9.8Football 352 x 288 High 37.1 9.2Stefan 352 x 264 High 37 9.4Highway 352 x 288 High 33.2 8.6Bus 352 x 288 High 34.3 8.8Coastguard 352 x 288 High 48.4 9.7Carphone 176 x 144 Medium 17.3 5.1Mobile 352 x 288 Medium 31.6 7.4Paris 352 x 288 Medium 39.6 7.6Suzie 176 x 144 Medium 23.1 4.1Akiyo 352 x 288 Low 30 6.4Bridge 352 x 288 Low 30.6 1.1Container 352 x 288 Low 54.9 14News 352 x 288 Low 42.6 11.3Hall 352 x 288 Low 29.1 6.9Table 3: % Frame loss of Cross Layer DCCP and Standard DCCP(1 Mbit/s CBR) for standard YUV video sequences.(a) DCCP Cross-layer (b) Standard DCCPFigure 3: Comparison of visual frame quality.4. Conclusions and Future WorkWe have proposed and implemented a novel cross layer based videoencoding and streaming technique and demonstrated it’s feasibilityby Xstream-x264 based implementation. Our experimental resultsshow that cross-layer based streaming significantly outperformsthe traditional streaming method using DCCP transport protocol.As future work, we plan to integrate RTP transport protocol withour tool and make it available to a broader range of audienceexperimenting with multimedia encoding and streaming.AcknowledgmentsThis research work has been supported by funding from NationalICT Australia (NICTA). NICTA is a research organization fundedby Australian Government research initiatives through AustralianResearch Council (ARC).References[1] A. Huszak, S. Imre, Analysing GOP Structure and Packet Loss Effectson Error Propagation in MPEG–4 Video Streams, ISCCSP 2010.VideoNameEncodingMethod% IFrameLoss% PFrameLoss% BFrameLoss%TotalLossForeman StandardDCCP0.2 17 17 34.2CrossLayerDCCP0.0 4.2 5.6 9.8Football StandardDCCP0.1 20.4 16.6 37.1CrossLayerDCCP0.0 5.2 4.0 9.2Stefan StandardDCCP0.5 15 21.5 37.0CrossLayerDCCP0.0 4.2 5.2 9.4Highway StandardDCCP0.3 9.1 23.8 33.2CrossLayerDCCP0.0 2.5 6.1 8.6Bus StandardDCCP0.4 14 20.9 35.3CrossLayerDCCP0.1 3.4 5.3 8.8Coast-guradStandardDCCP0.4 28.6 19.4 48.4CrossLayerDCCP9.7 0.0 6.1 3.6Carphone StandardDCCP2.4 6.0 8.9 17.3CrossLayerDCCP0.0 1.9 3.2 5.1Mobile StandardDCCP0.3 10.2 21.1 31.6CrossLayerDCCP0.0 2.5 4.9 7.4Paris StandardDCCP0.3 23 16.3 39.6CrossLayerDCCP0.1 4.6 2.9 7.6Suzie StandardDCCP0.0 12.8 10.3 23.1CrossLayerDCCP0.0 2.3 1.8 4.1Akiyo StandardDCCP0.2 13.6 16.2 30CrossLayerDCCP0.0 3.3 3.1 6.4Bridge StandardDCCP0.1 9.0 21.5 30.6CrossLayerDCCP0.0 1.6 5.7 7.3Container StandardDCCP0.5 28 26.4 54.9CrossLayerDCCP0.2 10.5 3.3 14News StandardDCCP0.1 19.1 23.4 42.6CrossLayerDCCP0.1 5.0 6.2 11.3Hall StandardDCCP0.2 5.9 23 29.1CrossLayerDCCP0.0 1.7 5.2 6.9Table 4: Detailed Analysis of % I, P and B Frame Loss for theExperiments Presented in Table 3.[2] H. Schulzrinne, S. Casner, R. Frederick and V. Jacobson, RTP: ATransport Protocol for Real-Time Applications, RFC3550, July 2003.[3] R. Stewart, Ed., Stream Control Transmission Protocol (SCTP), RFC4960, September 2007.[4] S. Floyd, E. Kohler and J. Padhye, Profile for Datagram CongestionControl Protocol (DCCP) Congestion Control ID 3: TCP-FriendlyRate Control (TFRC), RFC 4342, March 2006.[5] T. Wiegand, T., Sullivan, G.J., Bjontegaard, G., Luthra, A., Overviewof the H.264/AVC Video Coding Standard,ITCSVT 2003.[6] x264, http://www.videolan.org/developers/x264.html.[7] S. Benayoune, N. Achir, K. Boussetta, K. Chen, A MAC Centric CrossLayer Approach for H.264 Video Streaming over HSDPA, Journal ofCommunications, Oct 2009.[8] Khan, S. Peng, Y. Steinbach, E. Sgroi, M. Kellerer, W., Application-driven cross-layer optimization for video streaming over wireless net-works, 10.1109/MCOM.2006.1580942, 2006.[9] Soni, R., Chilamkurti, N., Giambene, G., Zeadally, S., A Cross-LayerDesign for H.264 Video Stream Over Wireless Local Area Networks,CSA 2008.[10] Zhengye Liu, Hang Liu, Yao Wang, Cross Layer Adaptation for H.264Video Multicasting Over Wireless Lan, ICME 2006.[11] EvalVid - A Video Quality Evaluation Tool-set, http://www.tkn.tu-berlin.de/research/evalvid/.[12] H.264/AVC JM Reference Software, http://iphome.hhi.de/suehring/tml/[13] MPEG4IP, http://www.mpeg4ip.net/.[14] ITU-T Rec. J.247 (08/08) Objective perceptual multimedia video qual-ity measurement in the presence of a full reference, http://www.itu.int/rec/T-REC-J.247/en[15] Q. Huynh-Thu1 and M. Ghanbari, Scope of validity of PSNR inimage/video quality assessment, IET Electronics Letters, 2008.[16] ImageMagick, http://www.imagemagick.org/[17] Zhenyu Wu, Boyce, J.M., An Error Concealment Scheme for EntireFrame Losses for H.264/AVC, ISCAS 2006.[18] Xiph. Foundation, http://media.xiph.org/[19] Arizona State University, "YUV Video Sequences", http://trace.eas.asu.edu/yuv/

Analysing GOP Structure and Packet Loss Effects on Error Propagation

Xstream-x264: Real-time H.264 streaming with cross-layer integration

http://oatao.univ-toulouse.fr/4892/1/Lochin_4892.pdf

Xstream-x264: Real-time H.264 streaming with cross-layer integration

Abstract

Similar works

Full text

Available Versions

Open Archive Toulouse Archive Ouverte