Understanding user experience of mobile video: Framework, measurement, and optimization

Since users have become the focus of product/service design in last decade, the term User eXperience (UX) has been frequently used in the field of Human-Computer-Interaction (HCI). Research on UX facilitates a better understanding of the various aspects of the user’s interaction with the product or service. Mobile video, as a new and promising service and research field, has attracted great attention. Due to the significance of UX in the success of mobile video (Jordan, 2002), many researchers have centered on this area, examining users’ expectations, motivations, requirements, and usage context. As a result, many influencing factors have been explored (Buchinger, Kriglstein, Brandt & Hlavacs, 2011; Buchinger, Kriglstein & Hlavacs, 2009). However, a general framework for specific mobile video service is lacking for structuring such a great number of factors. To measure user experience of multimedia services such as mobile video, quality of experience (QoE) has recently become a prominent concept. In contrast to the traditionally used concept quality of service (QoS), QoE not only involves objectively measuring the delivered service but also takes into account user’s needs and desires when using the service, emphasizing the user’s overall acceptability on the service. Many QoE metrics are able to estimate the user perceived quality or acceptability of mobile video, but may be not enough accurate for the overall UX prediction due to the complexity of UX. Only a few frameworks of QoE have addressed more aspects of UX for mobile multimedia applications but need be transformed into practical measures. The challenge of optimizing UX remains adaptations to the resource constrains (e.g., network conditions, mobile device capabilities, and heterogeneous usage contexts) as well as meeting complicated user requirements (e.g., usage purposes and personal preferences). In this chapter, we investigate the existing important UX frameworks, compare their similarities and discuss some important features that fit in the mobile video service. Based on the previous research, we propose a simple UX framework for mobile video application by mapping a variety of influencing factors of UX upon a typical mobile video delivery system. Each component and its factors are explored with comprehensive literature reviews. The proposed framework may benefit in user-centred design of mobile video through taking a complete consideration of UX influences and in improvement of mobile videoservice quality by adjusting the values of certain factors to produce a positive user experience. It may also facilitate relative research in the way of locating important issues to study, clarifying research scopes, and setting up proper study procedures. We then review a great deal of research on UX measurement, including QoE metrics and QoE frameworks of mobile multimedia. Finally, we discuss how to achieve an optimal quality of user experience by focusing on the issues of various aspects of UX of mobile video. In the conclusion, we suggest some open issues for future study

A content-aware quantisation mechanism for transform domain distributed video coding

The discrete cosine transform (DCT) is widely applied in modern codecs to remove spatial redundancies, with the resulting DCT coefficients being quantised to achieve compression as well as bit-rate control. In distributed video coding (DVC) architectures like DISCOVER, DCT coefficient quantisation is traditionally performed using predetermined quantisation matrices (QM), which means the compression is heavily dependent on the sequence being coded. This makes bit-rate control challenging, with the situation exacerbated in the coding of high resolution sequences due to QM scarcity and the non-uniform bit-rate gaps between them. This paper introduces a novel content-aware quantisation (CAQ) mechanism to overcome the limitations of existing quantisation methods in transform domain DVC. CAQ creates a frame-specific QM to reduce quantisation errors by analysing the distribution of DCT coefficients. In contrast to the predetermined QM that is applicable to only 4x4 block sizes, CAQ produces QM for larger block sizes to enhance compression at higher resolutions. This provides superior bit-rate control and better output quality by seeking to fully exploit the available bandwidth, which is especially beneficial in bandwidth constrained scenarios. In addition, CAQ generates superior perceptual results by innovatively applying different weightings to the DCT coefficients to reflect the human visual system. Experimental results corroborate that CAQ both quantitatively and qualitatively provides enhanced output quality in bandwidth limited scenarios, by consistently utilising over 90% of available bandwidth

Error resilience and concealment techniques for high-efficiency video coding

This thesis investigates the problem of robust coding and error concealment in High Efficiency Video Coding (HEVC). After a review of the current state of the art, a simulation study about error robustness, revealed that the HEVC has weak protection against network losses with significant impact on video quality degradation. Based on this evidence, the first contribution of this work is a new method to reduce the temporal dependencies between motion vectors, by improving the decoded video quality without compromising the compression efficiency. The second contribution of this thesis is a two-stage approach for reducing the mismatch of temporal predictions in case of video streams received with errors or lost data. At the encoding stage, the reference pictures are dynamically distributed based on a constrained Lagrangian rate-distortion optimization to reduce the number of predictions from a single reference. At the streaming stage, a prioritization algorithm, based on spatial dependencies, selects a reduced set of motion vectors to be transmitted, as side information, to reduce mismatched motion predictions at the decoder. The problem of error concealment-aware video coding is also investigated to enhance the overall error robustness. A new approach based on scalable coding and optimally error concealment selection is proposed, where the optimal error concealment modes are found by simulating transmission losses, followed by a saliency-weighted optimisation. Moreover, recovery residual information is encoded using a rate-controlled enhancement layer. Both are transmitted to the decoder to be used in case of data loss. Finally, an adaptive error resilience scheme is proposed to dynamically predict the video stream that achieves the highest decoded quality for a particular loss case. A neural network selects among the various video streams, encoded with different levels of compression efficiency and error protection, based on information from the video signal, the coded stream and the transmission network. Overall, the new robust video coding methods investigated in this thesis yield consistent quality gains in comparison with other existing methods and also the ones implemented in the HEVC reference software. Furthermore, the trade-off between coding efficiency and error robustness is also better in the proposed methods

Efficient Region-of-Interest Scalable Video Coding with Adaptive Bit-Rate Control

This work relates to the regions-of-interest (ROI) coding that is a desirable feature in future applications based on the scalable video coding, which is an extension of the H.264/MPEG-4 AVC standard. Due to the dramatic technological progress, there is a plurality of heterogeneous devices, which can be used for viewing a variety of video content. Devices such as smartphones and tablets are mostly resource-limited devices, which make it difficult to display high-quality content. Usually, the displayed video content contains one or more ROI(s), which should be adaptively selected from the preencoded scalable video bitstream. Thus, an efficient scalable ROI video coding scheme is proposed in this work, thereby enabling the extraction of the desired regions-of-interest and the adaptive setting of the desirable ROI location, size, and resolution. In addition, an adaptive bit-rate control is provided for the region-of-interest scalable video coding. The performance of the presented techniques is demonstrated and compared with the joint scalable video model reference software (JSVM 9.19), thereby showing significant bit-rate savings as a tradeoff for the relatively low PSNR degradation

SVCEval-RA: an evaluation framework for adaptive scalable video streaming

[EN] Multimedia content adaption strategies are becoming increasingly important for effective video streaming over the actual heterogeneous networks. Thus, evaluation frameworks for adaptive video play an important role in the designing and deploying process of adaptive multimedia streaming systems. This paper describes a novel simulation framework for rate-adaptive video transmission using the Scalable Video Coding standard (H.264/SVC). Our approach uses feedback information about the available bandwidth to allow the video source to select the most suitable combination of SVC layers for the transmission of a video sequence. The proposed solution has been integrated into the network simulator NS-2 in order to support realistic network simulations. To demonstrate the usefulness of the proposed solution we perform a simulation study where a video sequence was transmitted over a three network scenarios. The experimental results show that the Adaptive SVC scheme implemented in our framework provides an efficient alternative that helps to avoid an increase in the network congestion in resource-constrained networks. Improvements in video quality, in terms of PSNR (Peak Signal to Noise Ratio) and SSIM (Structural Similarity Index) are also obtained.Castellanos Hernández, WE.; Guerri Cebollada, JC.; Arce Vila, P. (2017). SVCEval-RA: an evaluation framework for adaptive scalable video streaming. Multimedia Tools and Applications. 76(1):437-461. doi:10.1007/s11042-015-3046-yS437461761Akhshabi S, Begen AC, Dovrolis C (2011) An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP. In: Proceedings of the second annual ACM conference on Multimedia systems. ACM, pp 157–168Alabdulkarim MN, Rikli N-E (2012) QoS Provisioning for H.264/SVC Streams over Ad-Hoc ZigBee Networks Using Cross-Layer Design. In: 8th International Conference on Wireless Communications, Networking and Mobile Computing (WiCOM). pp 1–8Birkos K, Tselios C, Dagiuklas T, Kotsopoulos S (2013) Peer selection and scheduling of H. 264 SVC video over wireless networks. In: Wireless Communications and Networking Conference (WCNC), 2013 IEEE. pp 1633–1638Castellanos W (2014) SVCEval-RA - An Evaluation Framework for Adaptive Scalable Video Streaming. In: SourceForge Project. http://sourceforge.net/projects/svceval-ra/ . Accessed 1 May 2015Castellanos W, Guerri JC, Arce P (2015) A QoS-aware routing protocol with adaptive feedback scheme for video streaming for mobile networks. Comput Commun. http://dx.doi.org/10.1016/j.comcom.2015.08.012Castellanos W, Arce P, Acelas P, Guerri JC (2012) Route Recovery Algorithm for QoS-Aware Routing in MANETs. Springer Berlin Heidelberg, Bilbao, pp. 81–93Chikkerur S, Sundaram V, Reisslein M, Karam LJ (2011) Objective video quality assessment methods: A classification, review, and performance comparison. Broadcast, IEEE Trans on 57:165–182Choupani R, Wong S, Tolun M (2014) Multiple description coding for SNR scalable video transmission over unreliable networks. Multimed Tools Appl 69:843–858. doi: 10.1007/s11042-012-1150-9CISCO Corp. (2014) Cisco Visual Networking Index Forecast and Methodology. In: White Paper. http://www.cisco.com/c/en/us/solutions/collateral/service-provider/ip-ngn-ip-next-generation-network/white_paper_c11-481360.pdf.Dai M, Zhang Y, Loguinov D (2009) A unified traffic model for MPEG-4 and H. 264 video traces. IEEE Trans Multimedia 11:1010–1023Detti A, Bianchi G, Pisa C, et al. (2009) SVEF: an open-source experimental evaluation framework for H.264 scalable video streaming. In: IEEE Symposium on Computers and Communications. pp 36–41Espina F, Morato D, Izal M, Magaña E (2014) Analytical model for MPEG video frame loss rates and playback interruptions on packet networks. Multimed Tools Appl 72:361–383. doi: 10.1007/s11042-012-1344-1Fiems D, Steyaert B, Bruneel H (2012) A genetic approach to Markovian characterisation of H.264 scalable video. Multimedia Tools Appl 58:125–146Floyd S, Handley M, Kohler E Datagram Congestion Control Protocol (DCCP). http://tools.ietf.org/html/rfc4340 . Accessed 17 Feb 2014Floyd S, Padhye J, Widmer J TCP Friendly Rate Control (TFRC): Protocol Specification. http://tools.ietf.org/html/rfc5348 . Accessed 17 Feb 2014Fraz M, Malkani YA, Elahi MA (2009) Design and implementation of real time video streaming and ROI transmission system using RTP on an embedded digital signal processing (DSP) platform. In: 2nd International Conference on Computer, Control and Communication, 2009. IC4 2009. pp 1–6ISO/IEC (2014) Information technology - Dynamic adaptive streaming over HTTP (DASH) - Part 1: Media presentation description and segment formats.ITU-T (2013) Rec. H.264 & ISO/IEC 14496-10 AVC. Advanced Video Coding for Generic Audiovisual Services.Ivrlač MT, Choi LU, Steinbach E, Nossek JA (2009) Models and analysis of streaming video transmission over wireless fading channels. Signal Process Image Commun 24:651–665. doi: 10.1016/j.image.2009.04.005Karki R, Seenivasan T, Claypool M, Kinicki R (2010) Performance Analysis of Home Streaming Video Using Orb. In: Proceedings of the 20th International Workshop on Network and Operating Systems Support for Digital Audio and Video. ACM, New York, NY, USA, pp 111–116Ke C-H (2012) myEvalSVC-an Integrated Simulation Framework for Evaluation of H. 264/SVC Transmission. KSII Trans Internet Inf Syst (TIIS) 6:377–392. doi: 10.3837/tiis.2012.01.021Ke C-H, Shieh C-K, Hwang W-S, Ziviani A (2008) An Evaluation Framework for More Realistic Simulations of MPEG Video Transmission. J Inf Sci Eng 24:425–440Klaue J, Rathke B, Wolisz A (2003) Evalvid–A framework for video transmission and quality evaluation. In: Computer Performance Evaluation. Modelling Techniques and Tools. Springer, pp 255–272Le TA, Nguyen H (2014) End-to-end transmission of scalable video contents: performance evaluation over EvalSVC—a new open-source evaluation platform. Multimed Tools Appl 72:1239–1256. doi: 10.1007/s11042-013-1444-6Lie A, Klaue J (2008) Evalvid-RA: trace driven simulation of rate adaptive MPEG-4 VBR video. Multimedia Systems 14:33–50. doi: 10.1007/s00530-007-0110-0Moving Pictures Experts Group and ITU-T Video Coding Experts Group (2011) H. 264/SVC reference software (JSVM 9.19.14) and Manual.Nightingale J, Wang Q, Grecos C (2014) Empirical evaluation of H.264/SVC streaming in resource-constrained multihomed mobile networks. Multimed Tools Appl 70:2011–2035. doi: 10.1007/s11042-012-1219-5Parmar H, Thornburgh M (2012) Real-Time Messaging Protocol (RTMP) Specification. AdobePolitis I, Dounis L, Dagiuklas T (2012) H. 264/SVC vs. H. 264/AVC video quality comparison under QoE-driven seamless handoff. Signal Process Image Commun 27:814–826Pozueco L, Pañeda XG, García R, et al. (2013) Adaptable system based on Scalable Video Coding for high-quality video service. Comput Electr Eng 39:775–789. doi: 10.1016/j.compeleceng.2013.01.015Pozueco L, Pañeda XG, García R, et al. (2014) Adaptation engine for a streaming service based on MPEG-DASH. Multimed Tools Appl 1–20. doi: 10.1007/s11042-014-2034-ySchwarz H, Marpe D, Wiegand T (2007) Overview of the Scalable Video Coding Extension of the H.264/AVC Standard. IEEE Trans Circ Syst Video Technol 17:1103–1120. doi: 10.1109/TCSVT.2007.905532Seo H-Y (2013) An Efficient Transmission Scheme of MPEG2-TS over RTP for a Hybrid DMB System. ETRI J 35:655–665. doi: 10.4218/etrij.13.0112.0124Sohn H, Yoo H, De Neve W, et al. (2010) Full-Reference Video Quality Metric for Fully Scalable and Mobile SVC Content. IEEE Trans Broadcast 56:269–280. doi: 10.1109/TBC.2010.2050628Sousa-Vieira M-E (2011) Suitability of the M/G/∞ process for modeling scalable H.264 video traffic. In: Analytical and Stochastic Modeling Techniques and Applications. Springer, pp 149–158Tanwir S, Perros H (2013) A Survey of VBR Video Traffic Models. IEEE Commun Surv Tutor 15:1778–1802. doi: 10.1109/SURV.2013.010413.00071Tanwir S, Perros HG (2014) VBR Video Traffic Models. Wiley, HobokenThe Network Simulator (NS-2). http://www.isi.edu/nsnam/ns . Accessed 6 Feb 2015Unanue I, Urteaga I, Husemann R, et al. (2011) A Tutorial on H. 264/SVC Scalable Video Coding and its Tradeoff between Quality, Coding Efficiency and Performance. Recent Advances on Video Coding 1–24.Van der Auwera G, David PT, Reisslein M, Karam LJ (2008) Traffic and quality characterization of the H. 264/AVC scalable video coding extension. Adv Multimedia 2008:1Wang Y, Claypool M (2005) RealTracer—Tools for Measuring the Performance of RealVideo on the Internet. Multimed Tools Appl 27:411–430. doi: 10.1007/s11042-005-3757-6Wang Z, Lu L, Bovik AC (2004) Video quality assessment based on structural distortion measurement. Signal Process Image Commun 19:121–132. doi: 10.1016/S0923-5965(03)00076–6Wien M, Schwarz H, Oelbaum T (2007) Performance Analysis of SVC. IEEE Trans Circ Syst for Video Technol 17:1194–1203. doi: 10.1109/TCSVT.2007.905530YUV video repository. ftp://ftp.tnt.uni-hannover.de/pub/svc/testsequences/ . Accessed 10 Jan 201

Cross-layer Optimized Wireless Video Surveillance

A wireless video surveillance system contains three major components, the video capture and preprocessing, the video compression and transmission over wireless sensor networks (WSNs), and the video analysis at the receiving end. The coordination of different components is important for improving the end-to-end video quality, especially under the communication resource constraint. Cross-layer control proves to be an efficient measure for optimal system configuration. In this dissertation, we address the problem of implementing cross-layer optimization in the wireless video surveillance system. The thesis work is based on three research projects. In the first project, a single PTU (pan-tilt-unit) camera is used for video object tracking. The problem studied is how to improve the quality of the received video by jointly considering the coding and transmission process. The cross-layer controller determines the optimal coding and transmission parameters, according to the dynamic channel condition and the transmission delay. Multiple error concealment strategies are developed utilizing the special property of the PTU camera motion. In the second project, the binocular PTU camera is adopted for video object tracking. The presented work studied the fast disparity estimation algorithm and the 3D video transcoding over the WSN for real-time applications. The disparity/depth information is estimated in a coarse-to-fine manner using both local and global methods. The transcoding is coordinated by the cross-layer controller based on the channel condition and the data rate constraint, in order to achieve the best view synthesis quality. The third project is applied for multi-camera motion capture in remote healthcare monitoring. The challenge is the resource allocation for multiple video sequences. The presented cross-layer design incorporates the delay sensitive, content-aware video coding and transmission, and the adaptive video coding and transmission to ensure the optimal and balanced quality for the multi-view videos. In these projects, interdisciplinary study is conducted to synergize the surveillance system under the cross-layer optimization framework. Experimental results demonstrate the efficiency of the proposed schemes. The challenges of cross-layer design in existing wireless video surveillance systems are also analyzed to enlighten the future work. Adviser: Song C