Network-aware video streaming for future media Internet

272 p

Video Adaptation for High-Quality Content Delivery

Modern video players employ complex algorithms to adapt the bitrate of the video that is shown to the user. Bitrate adaptation requires a tradeoff between reducing the probability that the video freezes (rebuffers) and enhancing the quality of the video. A bitrate that is too high leads to frequent rebuffering, while a bitrate that is too low leads to poor video quality. In this dissertation we propose video-adaptation algorithms to deliver content and maximize the viewer\u27s quality of experience (QoE). Video providers partition videos into short segments and encode each segment at multiple bitrates. The video player adaptively chooses the bitrate of each segment to download, possibly choosing different bitrates for successive segments. We formulate bitrate adaptation as a utility-maximization problem, and design algorithms to provide provably near-optimal time-average utility. Real-world systems are generally too complex to be fully represented in a theoretical model and thus present a new set of challenges. We design algorithms that deliver video on production systems, maintaining the strengths of the theoretical algorithms while also tackling challenges faced in production. Our algorithms are now part of the official DASH reference player dash.js and are being used by video providers in production environments. Most online video is streamed via HTTP over TCP. TCP provides reliable delivery at the expense of additional latency incurred when retransmitting lost packets and head-of-line blocking. Using QUIC allows the video player to tolerate some packet loss without incurring the performance penalties. We design and implement algorithms that exploit this added flexibility to provide higher overall QoE by reducing latency and rebuffering while allowing some packet loss. Recently virtual reality content is increasing in popularity, and delivering 360° video comes with new challenges and opportunities. The viewing space is often partitioned in tiles, and a viewer using a head-mounted display only sees a subset of the tiles at any time. We develop an open source simulation environment for fast and reproducible testing of 360° algorithms. We develop adaptation algorithms that provide high QoE by allocating more bandwidth resources to deliver the tiles that the viewer is more likely to see, while ensuring that the video player reacts in a timely manner when the viewer changes their head pose

Llama : Towards Low Latency Live Adaptive Streaming

Multimedia streaming, including on-demand and live delivery of content, has become the largest service, in terms of traffic volume, delivered over the Internet. The ever-increasing demand has led to remarkable advancements in multimedia delivery technology over the past three decades, facilitated by the concurrent pursuit of efficient and quality encoding of digital media. Today, the most prominent technology for online multimedia delivery is HTTP Adaptive Streaming (HAS), which utilises the stateless HTTP architecture - allowing for scalable streaming sessions that can be delivered to millions of viewers around the world using Content Delivery Networks. In HAS, the content is encoded at multiple encoding bitrates, and fragmented into segments of equal duration. The client simply fetches the consecutive segments from the server, at the desired encoding bitrate determined by an ABR algorithm which measures the network conditions and adjusts the bitrate accordingly. This method introduces new challenges to live streaming, where the content is generated in real-time, as it suffers from high end-to-end latency when compared to traditional broadcast methods due to the required buffering at client. This thesis aims to investigate low latency live adaptive streaming, focusing on the reduction of the end-to-end latency. We investigate the impact of latency on the performance of ABR algorithms in low latency scenarios by developing a simulation model and testing prominent on-demand adaptation solutions. Additionally, we conduct extensive subjective testing to further investigate the impact of bitrate changes on the perceived Quality of Experience (QoE) by users. Based on these investigations, we design an ABR algorithm suitable for low latency scenarios which can operate with a small client buffer. We evaluate the proposed low latency adaption solution against on-demand ABR algorithms and the state-of-the-art low latency ABR algorithms, under realistic network conditions using a variety of client and latency settings

Llama - Low Latency Adaptive Media Algorithm

In the recent years, HTTP Adaptive Bit Rate (ABR) streaming including Dynamic Adaptive Streaming over HTTP (DASH) has become the most popular technology for video streaming over the Internet. The client device requests segments of content using HTTP, with an ABR algorithm selecting the quality at which to request each segment to trade-off video quality with the avoidance of stalling. This introduces high latency compared to traditional broadcast methods, mostly in the client buffer which needs to hold enough data to absorb any changes in network conditions. Clients employ an ABR algorithm which monitors network conditions and adjusts the quality at which segments are requested to maximise the user's Quality of Experience. The size of the client buffer depends on the ABR algorithm's capability to respond to changes in network conditions in a timely manner, hence, low latency live streaming requires an ABR algorithm that can perform well with a small client buffer. In this paper, we present Llama - a new ABR algorithm specifically designed to operate in such scenarios. Our new ABR algorithm employs the novel idea of using two independent throughput measurements made over different timescales. We have evaluated Llama by comparing it against four popular ABR algorithms in terms of multiple QoE metrics, across multiple client settings, and in various network scenarios based on CDN logs of a commercial live TV service. Llama outperforms other ABR algorithms, improving the P.1203 Mean Opinion Score (MOS) as well as reducing rebuffering by 33% when using DASH, and 68% with CMAF in the lowest latency scenario

Evaluation of CMAF in live streaming scenarios

HTTP Adaptive Streaming (HAS) technologies such as MPEG DASH are now used extensively to deliver television services to large numbers of viewers. In HAS, the client requests segments of content using HTTP, with an ABR algorithm selecting the quality at which to request each segment to trade-off video quality with the avoidance of stalling. This introduces significant end to end latency compared to traditional broadcast, due to the the client requiring a large enough buffer for the ABR algorithm to react to changes in network conditions in a timely manner. The recently standardised Common Media Application Format (CMAF) has helped address the issue of latency by defining segments as composed of independently transferable chunks. In this paper, we describe a simulation model we have developed to evaluate the performance of four popular ABR algorithms using DASH and CMAF in various low latency live streaming scenarios. Realistic network conditions are used for the evaluation, which are based on throughput data taken from the CDN logs of a commercial live TV service. We quantify the performance of the ABR algorithms using a selection of QoE metrics, and show that CMAF can significantly improve ABR performance in low delay scenarios

Improving quality of experience in adaptive low latency live streaming

HTTP Adaptive Streaming (HAS), the most prominent technology for streaming video over the Internet, suffers from high end-to-end latency when compared to conventional broadcast methods. This latency is caused by the content being delivered as segments rather than as a continuous stream, requiring the client to buffer significant amounts of data to provide resilience to variations in network throughput and enable continuous playout of content without stalling. The client uses an Adaptive Bitrate (ABR) algorithm to select the quality at which to request each segment to trade-off video quality with the avoidance of stalling to improve the Quality of Experience (QoE). The speed at which the ABR algorithm responds to changes in network conditions influences the amount of data that needs to be buffered, and hence to achieve low latency the ABR needs to respond quickly. Llama (Lyko et al. 28) is a new low latency ABR algorithm that we have previously proposed and assessed against four on-demand ABR algorithms. In this article, we report an evaluation of Llama that demonstrates its suitability for low latency streaming and compares its performance against three state-of-the-art low latency ABR algorithms across multiple QoE metrics and in various network scenarios. Additionally, we report an extensive subjective test to assess the impact of variations in video quality on QoE, where the variations are derived from ABR behaviour observed in the evaluation, using short segments and scenarios. We publish our subjective testing results in full and make our throughput traces available to the research community

A Survey of Machine Learning Techniques for Video Quality Prediction from Quality of Delivery Metrics

A growing number of video streaming networks are incorporating machine learning (ML) applications. The growth of video streaming services places enormous pressure on network and video content providers who need to proactively maintain high levels of video quality. ML has been applied to predict the quality of video streams. Quality of delivery (QoD) measurements, which capture the end-to-end performances of network services, have been leveraged in video quality prediction. The drive for end-to-end encryption, for privacy and digital rights management, has brought about a lack of visibility for operators who desire insights from video quality metrics. In response, numerous solutions have been proposed to tackle the challenge of video quality prediction from QoD-derived metrics. This survey provides a review of studies that focus on ML techniques for predicting the QoD metrics in video streaming services. In the context of video quality measurements, we focus on QoD metrics, which are not tied to a particular type of video streaming service. Unlike previous reviews in the area, this contribution considers papers published between 2016 and 2021. Approaches for predicting QoD for video are grouped under the following headings: (1) video quality prediction under QoD impairments, (2) prediction of video quality from encrypted video streaming traffic, (3) predicting the video quality in HAS applications, (4) predicting the video quality in SDN applications, (5) predicting the video quality in wireless settings, and (6) predicting the video quality in WebRTC applications. Throughout the survey, some research challenges and directions in this area are discussed, including (1) machine learning over deep learning; (2) adaptive deep learning for improved video delivery; (3) computational cost and interpretability; (4) self-healing networks and failure recovery. The survey findings reveal that traditional ML algorithms are the most widely adopted models for solving video quality prediction problems. This family of algorithms has a lot of potential because they are well understood, easy to deploy, and have lower computational requirements than deep learning techniques

Semantic Segmentation with Bidirectional Language Models Improves Long-form ASR

We propose a method of segmenting long-form speech by separating semantically complete sentences within the utterance. This prevents the ASR decoder from needlessly processing faraway context while also preventing it from missing relevant context within the current sentence. Semantically complete sentence boundaries are typically demarcated by punctuation in written text; but unfortunately, spoken real-world utterances rarely contain punctuation. We address this limitation by distilling punctuation knowledge from a bidirectional teacher language model (LM) trained on written, punctuated text. We compare our segmenter, which is distilled from the LM teacher, against a segmenter distilled from a acoustic-pause-based teacher used in other works, on a streaming ASR pipeline. The pipeline with our segmenter achieves a 3.2% relative WER gain along with a 60 ms median end-of-segment latency reduction on a YouTube captioning task.Comment: Interspeech 2023. First 3 authors contributed equall