Application-Aware Network Design Using Software Defined Networking for Application Performance Optimization for Big Data and Video Streaming

Title from PDF of title page viewed October 30, 2017Dissertation advisor: Deep MedhiVitaIncludes bibliographical references (pages 122-135)Thesis (Ph.D.)--School of Computing and Engineering. University of Missouri--Kansas City, 2017This dissertation investigates improvement in application performance. For applications, we consider two classes: Hadoop MapReduce and video streaming. The Hadoop MapReduce (M/R) framework has become the de facto standard for Big Data analytics. However, the lack of network-awareness of the default MapReduce resource manager in a traditional IP network can cause unbalanced job scheduling and network bottlenecks; such factors can eventually lead to an increase in the Hadoop MapReduce job completion time. Dynamic Video streaming over the HTTP (MPEG-DASH) is becoming the defacto dominating transport for today’s video applications. It has been implemented in today’s major media carriers such as Youtube and Netflix. It enables new video applications to fully utilize the existing physical IP network infrastructure. For new 3D immersive medias such as Virtual Reality and 360-degree videos are drawing great attentions from both consumers and researchers in recent years. One of the biggest challenges in streaming such 3D media is the high band width demands and video quality. A new Tile-based video is introduced in both video codec and streaming layer to reduce the transferred media size. In this dissertation, we propose a Software-Defined Network (SDN) approach in an Application-Aware Network (AAN) platform. We first present an architecture for our approach and then show how this architecture can be applied to two aforementioned application areas. Our approach provides both underlying network functions and application level forwarding logics for Hadoop MapReduce and video streaming. By incorporating a comprehensive view of the network, the SDN controller can optimize MapReduce work loads and DASH flows for videos by application-aware traffic reroute. We quantify the improvement for both Hadoop and MPEG-DASH in terms of job completion time and user’s quality of experience (QoE), respectively. Based on our experiments, we observed that our AAN platform for Hadoop MapReduce job optimization offer a significant improvement compared to a static, traditional IP network environment by reducing job run time by 16% to 300% for various MapReduce benchmark jobs. As for MPEG-DASH based video streaming, we can increase user perceived video bitrate by 100%.Introduction -- Research survey -- Proposed architecture -- AAN-SDN for Hadoop -- Study of User QoE Improvement for Dynamic Adaptive Streaming over HTTP (MPEG-DASH) -- AAN-SDN For MPEG-DASH -- Conclusion -- Appendix A. Mininet Topology Source Code For DASH Setup -- Appendix B. Hadoop Installation Source Code -- Appendix C. Openvswitch Installation Source Code -- Appendix D. HiBench Installation Guid

QoE estimation for different adaptive streaming techniques in mobile networks

Video services are becoming more and more popular for mobile network users and require greater and greater resources and provisions from telecommunications service providers. But operators suffer from problems of interoperability between the different adaptive transmissions techniques they employ in an attempt to satisfy the quality of experience (QoE) of the service provided to users and improve network performance. This article presents a comparison of four such streaming techniques - DASH (dynamic adaptive streaming over HTTP), HDS (HTTP dynamic streaming), HLS (HTTP2 live streaming) and HSS (HTTP smooth streaming) - used in a live video playback by a user in different test scenarios on an emulated long-term evolution (LTE) network. Comparison of performance was carried out using the mean opinion score (MOS) metric calculated based on ITU-T Recommendation P.1203. The streaming techniques that performed best in each of the different test scenarios are revealed.El servicio de video es cada vez más popular por parte de los usuarios de redes móviles, además exige mayores recursos y prestaciones por parte de los proveedores de servicios de telecomunicaciones. Para satisfacer la calidad de la experiencia del servicio suministrado a los usuarios - QoE y mejorar el rendimiento de las redes, los operadores utilizan diferentes técnicas de transmisión adaptativa, las cuales presentan inconvenientes de interoperabilidad entre ellas.  En este artículo se presenta una comparación de las técnicas de streaming DASH (dynamic adaptive streaming over HTTP), HDS (HTTP dynamic streaming), HLS (HTTP2 live streaming) and HSS (HTTP smooth streaming) empleadas en la reproducción de vídeo en vivo por parte de un usuario en diferentes escenarios de prueba, en una red LTE emulada. La comparación de desempeño se realiza mediante la métrica de la MOS calculada a partir de la Recomendación ITU-T P.1203. Se presenta para los diferentes escenarios bajo prueba, la técnica de streaming que mejor desempeño obtiene

QoE estimation for different adaptive streaming techniques in mobile networks

Video services are becoming more and more popular for mobile network users and require greater and greater resources and provisions from telecommunications service providers. But operators suffer from problems of interoperability between the different adaptive transmissions techniques they employ in an attempt to satisfy the quality of experience (QoE) of the service provided to users and improve network performance. This article presents a comparison of four such streaming techniques - DASH (dynamic adaptive streaming over HTTP), HDS (HTTP dynamic streaming), HLS (HTTP2 live streaming) and HSS (HTTP smooth streaming) - used in a live video playback by a user in different test scenarios on an emulated long-term evolution (LTE) network. Comparison of performance was carried out using the mean opinion score (MOS) metric calculated based on ITU-T Recommendation P.1203. The streaming techniques that performed best in each of the different test scenarios are revealed.El servicio de video es cada vez más popular por parte de los usuarios de redes móviles, además exige mayores recursos y prestaciones por parte de los proveedores de servicios de telecomunicaciones. Para satisfacer la calidad de la experiencia del servicio suministrado a los usuarios - QoE y mejorar el rendimiento de las redes, los operadores utilizan diferentes técnicas de transmisión adaptativa, las cuales presentan inconvenientes de interoperabilidad entre ellas.  En este artículo se presenta una comparación de las técnicas de streaming DASH (dynamic adaptive streaming over HTTP), HDS (HTTP dynamic streaming), HLS (HTTP2 live streaming) and HSS (HTTP smooth streaming) empleadas en la reproducción de vídeo en vivo por parte de un usuario en diferentes escenarios de prueba, en una red LTE emulada. La comparación de desempeño se realiza mediante la métrica de la MOS calculada a partir de la Recomendación ITU-T P.1203. Se presenta para los diferentes escenarios bajo prueba, la técnica de streaming que mejor desempeño obtiene

QoE estimation for different adaptive streaming techniques in mobile networks

Video services are becoming more and more popular for mobile network users and require greater and greater resources and provisions from telecommunications service providers. But operators suffer from problems of interoperability between the different adaptive transmissions techniques they employ in an attempt to satisfy the quality of experience (QoE) of the service provided to users and improve network performance. This article presents a comparison of four such streaming techniques - DASH (dynamic adaptive streaming over HTTP), HDS (HTTP dynamic streaming), HLS (HTTP2 live streaming) and HSS (HTTP smooth streaming) - used in a live video playback by a user in different test scenarios on an emulated long-term evolution (LTE) network. Comparison of performance was carried out using the mean opinion score (MOS) metric calculated based on ITU-T Recommendation P.1203. The streaming techniques that performed best in each of the different test scenarios are revealed.El servicio de video es cada vez más popular por parte de los usuarios de redes móviles, además exige mayores recursos y prestaciones por parte de los proveedores de servicios de telecomunicaciones. Para satisfacer la calidad de la experiencia del servicio suministrado a los usuarios - QoE y mejorar el rendimiento de las redes, los operadores utilizan diferentes técnicas de transmisión adaptativa, las cuales presentan inconvenientes de interoperabilidad entre ellas.  En este artículo se presenta una comparación de las técnicas de streaming DASH (dynamic adaptive streaming over HTTP), HDS (HTTP dynamic streaming), HLS (HTTP2 live streaming) and HSS (HTTP smooth streaming) empleadas en la reproducción de vídeo en vivo por parte de un usuario en diferentes escenarios de prueba, en una red LTE emulada. La comparación de desempeño se realiza mediante la métrica de la MOS calculada a partir de la Recomendación ITU-T P.1203. Se presenta para los diferentes escenarios bajo prueba, la técnica de streaming que mejor desempeño obtiene

QoE-Based Low-Delay Live Streaming Using Throughput Predictions

Recently, HTTP-based adaptive streaming has become the de facto standard for video streaming over the Internet. It allows clients to dynamically adapt media characteristics to network conditions in order to ensure a high quality of experience, that is, minimize playback interruptions, while maximizing video quality at a reasonable level of quality changes. In the case of live streaming, this task becomes particularly challenging due to the latency constraints. The challenge further increases if a client uses a wireless network, where the throughput is subject to considerable fluctuations. Consequently, live streams often exhibit latencies of up to 30 seconds. In the present work, we introduce an adaptation algorithm for HTTP-based live streaming called LOLYPOP (Low-Latency Prediction-Based Adaptation) that is designed to operate with a transport latency of few seconds. To reach this goal, LOLYPOP leverages TCP throughput predictions on multiple time scales, from 1 to 10 seconds, along with an estimate of the prediction error distribution. In addition to satisfying the latency constraint, the algorithm heuristically maximizes the quality of experience by maximizing the average video quality as a function of the number of skipped segments and quality transitions. In order to select an efficient prediction method, we studied the performance of several time series prediction methods in IEEE 802.11 wireless access networks. We evaluated LOLYPOP under a large set of experimental conditions limiting the transport latency to 3 seconds, against a state-of-the-art adaptation algorithm from the literature, called FESTIVE. We observed that the average video quality is by up to a factor of 3 higher than with FESTIVE. We also observed that LOLYPOP is able to reach a broader region in the quality of experience space, and thus it is better adjustable to the user profile or service provider requirements.Comment: Technical Report TKN-16-001, Telecommunication Networks Group, Technische Universitaet Berlin. This TR updated TR TKN-15-00