Video Quality Prediction for Video over Wireless Access Networks (UMTS and WLAN)

Transmission of video content over wireless access networks (in particular, Wireless Local Area Networks (WLAN) and Third Generation Universal Mobile Telecommunication System (3G UMTS)) is growing exponentially and gaining popularity, and is predicted to expose new revenue streams for mobile network operators. However, the success of these video applications over wireless access networks very much depend on meeting the user’s Quality of Service (QoS) requirements. Thus, it is highly desirable to be able to predict and, if appropriate, to control video quality to meet user’s QoS requirements. Video quality is affected by distortions caused by the encoder and the wireless access network. The impact of these distortions is content dependent, but this feature has not been widely used in existing video quality prediction models. The main aim of the project is the development of novel and efficient models for video quality prediction in a non-intrusive way for low bitrate and resolution videos and to demonstrate their application in QoS-driven adaptation schemes for mobile video streaming applications. This led to five main contributions of the thesis as follows:(1) A thorough understanding of the relationships between video quality, wireless access network (UMTS and WLAN) parameters (e.g. packet/block loss, mean burst length and link bandwidth), encoder parameters (e.g. sender bitrate, frame rate) and content type is provided. An understanding of the relationships and interactions between them and their impact on video quality is important as it provides a basis for the development of non-intrusive video quality prediction models.(2) A new content classification method was proposed based on statistical tools as content type was found to be the most important parameter. (3) Efficient regression-based and artificial neural network-based learning models were developed for video quality prediction over WLAN and UMTS access networks. The models are light weight (can be implemented in real time monitoring), provide a measure for user perceived quality, without time consuming subjective tests. The models have potential applications in several other areas, including QoS control and optimization in network planning and content provisioning for network/service providers.(4) The applications of the proposed regression-based models were investigated in (i) optimization of content provisioning and network resource utilization and (ii) A new fuzzy sender bitrate adaptation scheme was presented at the sender side over WLAN and UMTS access networks. (5) Finally, Internet-based subjective tests that captured distortions caused by the encoder and the wireless access network for different types of contents were designed. The database of subjective results has been made available to research community as there is a lack of subjective video quality assessment databases.Partially sponsored by EU FP7 ADAMANTIUM Project (EU Contract 214751

Evaluation of Wirelessly Transmitted Video Quality Using a Modular Fuzzy Logic System

Video transmission over wireless computer networks is increasingly popular as new applications emerge and wireless networks become more widespread and reliable. An ability to quantify the quality of a video transmitted using a wireless computer network is important for determining network performance and its improvement. The process requires analysing the images making up the video from the point of view of noise and associated distortion as well as traffic parameters represented by packet delay, jitter and loss. In this study a modular fuzzy logic based system was developed to quantify the quality of video transmission over a wireless computer network. Peak signal to noise ratio, structural similarity index and image difference were used to represent the user's quality of experience (QoE) while packet delay, jitter and percentage packet loss ratio were used to represent traffic related quality of service (QoS). An overall measure of the video quality was obtained by combining QoE and QoS values. Systematic sampling was used to reduce the number of images processed and a novel scheme was devised whereby the images were partitioned to more sensitively localize distortions. To further validate the developed system, a subjective test involving 25 participants graded the quality of the received video. The image partitioning significantly improved the video quality evaluation. The subjective test results correlated with the developed fuzzy logic approach. The video quality assessment developed in this study was compared against a method that uses spatial efficient entropic differencing and consistent results were observed. The study indicated that the developed fuzzy logic approaches could accurately determine the quality of a wirelessly transmitted video

An automated model for the assessment of QoE of adaptive video streaming over wireless networks

[EN] Nowadays, heterogeneous devices are widely utilizing Hypertext Transfer Protocol (HTTP) to transfer the data. Furthermore, HTTP adaptive video streaming (HAS) technology transmits the video data over wired and wireless networks. In adaptive technology services, a client's application receives a streaming video through the adaptation of its quality to the network condition. However, such a technology has increased the demand for Quality of Experience (QoE) in terms of prediction and assessment. It can also cause a challenging behavior regarding subjective and objective QoE evaluations of HTTP adaptive video over time since each Quality of Service (QoS) parameter affects the QoE of end-users separately. This paper introduces a methodology design for the evaluation of subjective QoE in adaptive video streaming over wireless networks. Besides, some parameters are considered such as video characteristics, segment length, initial delay, switch strategy, stalls, as well as QoS parameters. The experiment's evaluation demonstrated that objective metrics can be mapped to the most significant subjective parameters for user's experience. The automated model could function to demonstrate the importance of correlation for network behaviors' parameters. Consequently, it directly influences the satisfaction of the end-user's perceptual quality. In comparison with other recent related works, the model provided a positive Pearson Correlation value. Simulated results give a better performance between objective Structural Similarity (SSIM) and subjective Mean Opinion Score (MOS) evaluation metrics for all video test samples.This work has been partially supported by the "Ministerio de Economia y Competitividad" in the "Programa Estatal de Fomento de la Investigacion Cientifica y Tecnica de Excelencia, Subprograma Estatal de Generacion de Conocimiento" within the Project under Grant TIN2017-84802-C2-1-P. This study has been partially done in the computer science departments at the (University of Sulaimani and Halabja).Taha, M.; Ali, A.; Lloret, J.; Gondim, PRL.; Canovas, A. (2021). An automated model for the assessment of QoE of adaptive video streaming over wireless networks. Multimedia Tools and Applications. 80(17):26833-26854. https://doi.org/10.1007/s11042-021-10934-92683326854801

Finding perceptually optimal operating points of a real time interactive video-conferencing system

This research aims to address issues faced by real time video-conferencing systems in locating a perceptually optimal operating point under various network and conversational conditions. In order to determine the perceptually optimal operating point of a video-conferencing system, we must first be able to conduct a fair assessment of the quality of the current operating point in the system and compare it with another operating point to determine if one is better than the other in terms of perceptual quality. However at this point in time, there does not exist one objective quality metric that can accurately and fully describe the perceptual quality of a real time video conversation. Hence there is a need for a controlled environment to allow tests to be conducted in and in which we can study different metrics and identify the best trade-offs between them. We begin by studying the components of a typical setup of a real time video-conferencing system and the impacts that various network and conversation conditions can have on the overall perceptual quality. We also look into different metrics available to measure those impacts. We then created a platform to perform black box testing on current video conferencing systems and observe how they handle the changes in operating conditions. The platform is then used to conduct a brief evaluation of the performance of Skype, a popular commercial video-conferencing system. However, we are not able to modify the system parameters of Skype. The main contribution of this thesis is the design of a new testbed that provides a controlled environment to allow tests to be conducted to determine the perceptual optimum operating point of a video conversation under specified network and conversation conditions. This testbed will allow us to modify certain parameters, such as frame rate and frame size, which were not previously possible. The testbed takes as input, two recorded videos of the two speakers of a face-to-face conversation and desired output video parameters, such as frame rate, frame size and delay. A video generation algorithm is designed as part of the testbed to handle modifications to frame rate and frame size of the videos as well as delays inserted into the recorded video conversation to simulate the effects of network delays. The most important issue addressed is the generation of new frames to fill up the gaps created due to a change in frame rate or delay inserted, unlike as in the case of voice, where a period of silence can simply be used to handle these situations. The testbed uses a packetization strategy designed on the basis of an uneven packet transmission rate (UPTR) and that handles the packetization of interleaved video and audio data; it also uses piggybacking to provide redundancy if required. Losses can be injected either randomly or based on packet traces collected via PlanetLab. The processed videos will then be pieced together side-by-side to give the viewpoint of a third-party observing the video conversation from the site of the first speaker. Hence the first speaker will be observed to have a faster reaction time without network delays than that of the second speaker who is simulated to be located at the remote end. The video of the second speaker will also reflect the degradations in perceptual quality induced by the network conditions, whereas the first speaker will be of perfect quality. Hence with the testbed, we are able to generate output videos for different operating points under the same network and conversational conditions and thus able to make comparisons between two operating points. With the testbed in place, we demonstrate how it can be used to evaluate the effects of various parameters on the overall perceptual quality. Lastly, we demonstrate the results of applying an existing efficient search algorithm used for estimating the perceptually optimal mouth-to-ear delay (MED) of a Voice-over-IP(VoIP) conversation to a Video Conversation. This is achieved by using the network simulator designed to conduct a series of subjective and objective tests to identify the perceptual optimum MED under specific network and conversational conditions

Tube streaming: Modelling collaborative media streaming in urban railway networks

We propose a quality assessment framework for crowdsourced media streaming in urban railway networks. We assume that commuters either tune in to some TV/radio channel, or submit requests for content they desire to watch or listen to, which eventually forms a playlist of videos/podcasts/tunes. Given that connectivity is challenged by the movement of trains and the disconnection that this movement causes, users collabo-ratively download (through cellular and WiFi connections) and share content, in order to maintain undisrupted playback. We model collaborative media streaming for the case of the London Underground train network. The proposed quality assessment framework comprises a utility function which characterises the Quality of Experience (QoE) that users (subjectively) perceive and takes into account all the necessary parameters that affect smooth playback. The framework can be used to assess the media streaming quality in any railway network, after adjusting the related parameters. To the best of our knowledge, this is the first study to quantify the perceptual quality of collaborative media streaming in (underground) railway networks from a modelling perspective, as opposed to a systems perspective. Based on real commuter traces from the London Underground network, we evaluate whether audio and video can be streamed to commuters with acceptable QoE. Our results show that even with very high-speed Internet connection, users still experience disruptions, but a carefully designed collaborative mechanism can result in high levels of perceived QoE even in such disruptive scenarios

Resilience of Video Streaming Services to Network Impairments

When dealing with networks, performance management through conventional quality of service (QoS)-based methods becomes difficult and is often ineffective. In fact, quality emerges as an end-to-end factor, for it is particularly sensitive to the end-user perception of the overall service, i.e., the user's quality of experience (QoE). However, the two are not independent from each other and their relationship has to be studied through metrics that go beyond the typical network parameters. To better explore the value of assessing QoE alongside QoS in high-speed, lossy networks, this paper presents an experimental methodology to understand the relation between network QoS onto service QoE, with the aim to perform a combined network-service assessment. Using video streaming services as the test-case (given their extended usage nowadays), in this paper, we provide studies on three network-impaired video-sets with the aim to provide a comprehensive evaluation of the effects of networks on video quality. First, the ReTRIeVED video set provides the means to understand the most impairing effects on networks. Furthermore, it triggered the idea to create our own sets, specialized in the most impairing conditions for 2-D and 3-D: the LIMP Video Quality Database and the 3-D-HEVC-Net Video Quality Database. Our study and methodology are meant to provide service providers with the means to pinpoint the working boundaries of their video-sets in face of different network conditions. At the same time, network operators may use our findings to predict how network control policies affect the user's perception of the service

A robust multimedia traffic SDN-Based management system using patterns and models of QoE estimation with BRNN

[EN] Nowadays, network infrastructures such as Software Defined Networks (SDN) achieve a huge computational power. This allows to add a high processing on the network nodes. In this paper, a multimedia traffic management system is presented. This system is based on estimation models of Quality of Experience (QoE) and also on the traffic patterns classification. In order to achieve this, a QoE estimation method has been modeled. This method allows for classifying the multimedia traffic from multimedia transmission patterns. In order to do this, the SDN controller gathers statistics from the network. The patterns used have been defined from a lineal combination of objective QoE measurements. The model has been defined by Bayesian regularized neural networks (BRNN). From this model, the system is able to classify several kind of traffic according to the quality perceived by the users. Then, a model has been developed to determine which video characteristics need to be changed to provide the user with the best possible quality in the critical moments of the transmission. The choice of these characteristics is based on the quality of service (QoS) parameters, such as delay, jitter, loss rate and bandwidth. Moreover, it is also based on subpatterns defined by clusters from the dataset and which represents network and video characteristics. When a critical network situation is given, the model selects, by using network parameters as entries, the subpattern with the most similar network condition. The minimum Euclidean distance between these entries and the network parameters of the subpatters is calculated to perform this selection. Both models work together to build a reliable multimedia traffic management system perfectly integrated into current network infrastructures, which is able to classify the traffic and solve critical situations changing the video characteristics, by using the SDN architecture.This work has been partially supported by the "Ministerio de Educacion, Cultura y Deporte", through the "Ayudas para contratos predoctorales de Formation del Profesorado Universitario FPU (Convocatoria 2015)", grant number FPU15/06837 and by the "Ministerio de Economia y Competitividad" in the "Programa Estatal de Fomento de la Investigation Cientffica y Tecnica de Excelencia, Subprograma Estatal de Generacion de Conocimiento" within the project under Grant TIN2017-84802-C2-1-P.Canovas Solbes, A.; Rego Mañez, A.; Romero Martínez, JO.; Lloret, J. (2020). A robust multimedia traffic SDN-Based management system using patterns and models of QoE estimation with BRNN. Journal of Network and Computer Applications. 150:1-14. https://doi.org/10.1016/j.jnca.2019.102498S114150Cánovas, A., Taha, M., Lloret, J., & Tomás, J. (2018). Smart resource allocation for improving QoE in IP Multimedia Subsystems. Journal of Network and Computer Applications, 104, 107-116. doi:10.1016/j.jnca.2017.12.020Canovas, A., Jimenez, J. M., Romero, O., & Lloret, J. (2018). Multimedia Data Flow Traffic Classification Using Intelligent Models Based on Traffic Patterns. IEEE Network, 32(6), 100-107. doi:10.1109/mnet.2018.1800121Burden, F., & Winkler, D. (2008). Bayesian Regularization of Neural Networks. Artificial Neural Networks, 23-42. doi:10.1007/978-1-60327-101-1_3Goodman, S. N. (2005). Introduction to Bayesian methods I: measuring the strength of evidence. Clinical Trials, 2(4), 282-290. doi:10.1191/1740774505cn098oaHirschen, K., & Schäfer, M. (2006). Bayesian regularization neural networks for optimizing fluid flow processes. Computer Methods in Applied Mechanics and Engineering, 195(7-8), 481-500. doi:10.1016/j.cma.2005.01.015Huang, X., Yuan, T., Qiao, G., & Ren, Y. (2018). Deep Reinforcement Learning for Multimedia Traffic Control in Software Defined Networking. IEEE Network, 32(6), 35-41. doi:10.1109/mnet.2018.1800097Lin, Y. (2002). Data Mining and Knowledge Discovery, 6(3), 259-275. doi:10.1023/a:1015469627679Lopez-Martin, M., Carro, B., Lloret, J., Egea, S., & Sanchez-Esguevillas, A. (2018). Deep Learning Model for Multimedia Quality of Experience Prediction Based on Network Flow Packets. IEEE Communications Magazine, 56(9), 110-117. doi:10.1109/mcom.2018.1701156Hagan, M. T., & Menhaj, M. B. (1994). Training feedforward networks with the Marquardt algorithm. IEEE Transactions on Neural Networks, 5(6), 989-993. doi:10.1109/72.329697Nguyen, T. T. T., & Armitage, G. (2008). A survey of techniques for internet traffic classification using machine learning. IEEE Communications Surveys & Tutorials, 10(4), 56-76. doi:10.1109/surv.2008.080406Queiroz, W., Capretz, M. A. M., & Dantas, M. (2019). An approach for SDN traffic monitoring based on big data techniques. Journal of Network and Computer Applications, 131, 28-39. doi:10.1016/j.jnca.2019.01.016Rego, A., Canovas, A., Jimenez, J. M., & Lloret, J. (2018). An Intelligent System for Video Surveillance in IoT Environments. IEEE Access, 6, 31580-31598. doi:10.1109/access.2018.2842034Seshadrinathan, K., Soundararajan, R., Bovik, A. C., & Cormack, L. K. (2010). Study of Subjective and Objective Quality Assessment of Video. IEEE Transactions on Image Processing, 19(6), 1427-1441. doi:10.1109/tip.2010.2042111Soysal, M., & Schmidt, E. G. (2010). Machine learning algorithms for accurate flow-based network traffic classification: Evaluation and comparison. Performance Evaluation, 67(6), 451-467. doi:10.1016/j.peva.2010.01.001Tan, X., Xie, Y., Ma, H., Yu, S., & Hu, J. (2019). Recognizing the content types of network traffic based on a hybrid DNN-HMM model. Journal of Network and Computer Applications, 142, 51-62. doi:10.1016/j.jnca.2019.06.004Tongaonkar, A., Torres, R., Iliofotou, M., Keralapura, R., & Nucci, A. (2015). Towards self adaptive network traffic classification. Computer Communications, 56, 35-46. doi:10.1016/j.comcom.2014.03.02

Adaptive Video Streaming Testbed Design for Performance Study and Assessment of QoE

[EN] Hypertext Transfer Protocol adaptive streaming switches between different video qualities, adapting to the network conditions, and avoids stalling streamed frames over high¿oscillation client's throughput improving the users' quality of experience (QoE). Quality of experience has become the most important parameter to lead the service providers to know about the end¿user feedback. Implementing Hypertext Transfer Protocol adaptive streaming applications to find out QoE in real¿life scenarios of vast networks becomes more challenging and complex task regarding to cost, agile, time, and decisions. In this paper, a virtualized network testbed to virtualize various machines to support implementing experiments of adaptive video streaming has been developed. Within the test study, the metrics which demonstrate performance of QoE are investigated, respectively, including initial delay (ie, startup delay at the beginning of playback a video), frequency switches (ie, number of times the quality is changed), accumulative video time (ie, number and length of stalls), CPU usage, and battery energy consumption. Furthermore, the relation between effective parameters of QoS on the aforementioned metrics for different segment length is investigated. Experimental results show that the proposed virtualized system is agile, easy to install and use, and costs less than real testbeds. Moreover, the subjective and objective performance studies of QoE evaluation in the system have proven that the segment lengths of 6 to 8 seconds were faired and more efficient than others according to the investigated parameters.Ministerio de Economia y Competitividad, Grant/Award Number: TIN2014-57991-C3-1-PAbdullah, MTA.; Lloret, J.; Ali, A.; García-García, L. (2018). Adaptive Video Streaming Testbed Design for Performance Study and Assessment of QoE. International Journal of Communication Systems. 1-15. https://doi.org/10.1002/dac.3551S11

Evaluation of objective video quality assessment methods on video sequences with different spatial and temporal activity encoded at different spatial resolutions

With the development of Video on Demand applications due to the availability of high-speed internet access, adaptive streaming algorithms have been developing and improving. The focus is on improving user’s Quality of Experience (QoE) and taking it into account as one of the parameters for the adaptation algorithm. Users often experience changing network conditions, so the goal is to ensure stable video playback with satisfying QoE level. Although subjective Video Quality Assessment (VQA) methods provide more accurate results regarding user’s QoE, objective VQA methods cost less and are less time-consuming. In this article, nine different objective VQA methods are compared on a large set of video sequences with various spatial and temporal activities. VQA methods used in this analysis are: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), MultiScale Structural Similarity Index (MS-SSIM), Video Quality Metric (VQM), Mean Sum of Differences (DELTA), Mean Sum of Absolute Differences (MSAD), Mean Squared Error (MSE), Netflix Video Multimethod Assessment Fusion (Netflix VMAF) and Visual Signal-to-Noise Ratio (VSNR). The video sequences used for testing purposes were encoded according to H.264/AVC with twelve different target coding bitrates, at three different spatial resolutions (resulting in a total of 190 sequences). In addition to objective quality assessment, subjective quality assessment was performed for these sequences. All results acquired by objective VQA methods have been compared with subjective Mean Opinion Score (MOS) results using Pearson Linear Correlation Coefficient (PLCC). Measurement results obtained on a large set of video sequences with different spatial resolutions show that VQA methods like SSIM and VQM correlate better with MOS results compared to PSNR, SSIM, VSNR, DELTA, MSE, VMAF and MSAD. However, the PLCC results for SSIM and VQM are too low (0.7799 and 0.7734, respectively), for the usage of these methods in streaming services instead of subjective testing. These results suggest that more efficient VQA methods should be developed to be used in streaming testing procedures as well as to support the video segmentation process. Furthermore, when comparing results obtained for different spatial resolutions, it can be concluded that the quality of video sequences encoded at lower spatial resolutions in cases of lower target coding bitrate is higher compared to the quality of video sequences encoded at higher spatial resolutions at the same target coding bitrate, particularly when video sequences with higher spatial and temporal information are used

Adaptive reservation of network resources according to video classification scenes

Video quality evaluation needs a combined approach that includes subjective and objective metrics, testing, and monitoring of the network. This paper deals with the novel approach of mapping quality of service (QoS) to quality of experience (QoE) using QoE metrics to determine user satisfaction limits, and applying QoS tools to provide the minimum QoE expected by users. Our aim was to connect objective estimations of video quality with the subjective estimations. A comprehensive tool for the estimation of the subjective evaluation is proposed. This new idea is based on the evaluation and marking of video sequences using the sentinel flag derived from spatial information (SI) and temporal information (TI) in individual video frames. The authors of this paper created a video database for quality evaluation, and derived SI and TI from each video sequence for classifying the scenes. Video scenes from the database were evaluated by objective and subjective assessment. Based on the results, a new model for prediction of subjective quality is defined and presented in this paper. This quality is predicted using an artificial neural network based on the objective evaluation and the type of video sequences defined by qualitative parameters such as resolution, compression standard, and bitstream. Furthermore, the authors created an optimum mapping function to define the threshold for the variable bitrate setting based on the flag in the video, determining the type of scene in the proposed model. This function allows one to allocate a bitrate dynamically for a particular segment of the scene and maintains the desired quality. Our proposed model can help video service providers with the increasing the comfort of the end users. The variable bitstream ensures consistent video quality and customer satisfaction, while network resources are used effectively. The proposed model can also predict the appropriate bitrate based on the required quality of video sequences, defined using either objective or subjective assessment.Web of Science216art. no. 194