Video Compression and Optimization Technologies - Review

The use of video streaming is constantly increasing. High-resolution video requires resources on both the sender and the receiver side. There are many compression techniques that can be utilized to compress the video and simultaneously maintain quality. The main goal of this paper is to provide an overview of video streaming and QoE. This paper describes the basic concepts and discusses existing methodologies to measure QoE. Subjective, objective, and video compression technologies are discussed. This review paper gathers the codec implementation developed by MPEG, Google, and Apple. This paper outlines the challenges and future research directions that should be considered in the measurement and assessment of quality of experience for video services

User generated HDR gaming video streaming : dataset, codec comparison and challenges

Gaming video streaming services have grown tremendously in the past few years, with higher resolutions, higher frame rates and HDR gaming videos getting increasingly adopted among the gaming community. Since gaming content as such is different from non-gaming content, it is imperative to evaluate the performance of the existing encoders to help understand the bandwidth requirements of such services, as well as further improve the compression efficiency of such encoders. Towards this end, we present in this paper GamingHDRVideoSET, a dataset consisting of eighteen 10-bit UHD-HDR gaming videos and encoded video sequences using four different codecs, together with their objective evaluation results. The dataset is available online at [to be added after paper acceptance]. Additionally, the paper discusses the codec compression efficiency of most widely used practical encoders, i.e., x264 (H.264/AVC), x265 (H.265/HEVC) and libvpx (VP9), as well the recently proposed encoder libaom (AV1), on 10-bit, UHD-HDR content gaming content. Our results show that the latest compression standard AV1 results in the best compression efficiency, followed by HEVC, H.264, and VP9.Comment: 14 pages, 8 figures, submitted to IEEE journa

An objective and subjective quality assessment for passive gaming video streaming

Gaming video streaming has become increasingly popular in recent times. Along with the rise and popularity of cloud gaming services and e-sports, passive gaming video streaming services such as Twitch.tv, YouTubeGaming, etc. where viewers watch the gameplay of other gamers, have seen increasing acceptance. Twitch.tv alone has over 2.2 million monthly streamers and 15 million daily active users with almost a million average concurrent users, making Twitch.tv the 4th biggest internet traffic generator, just after Netflix, YouTube and Apple. Despite the increasing importance and popularity of such live gaming video streaming services, they have until recently not caught the attention of the quality assessment research community. For the continued success of such services, it is imperative to maintain and satisfy the end user Quality of Experience (QoE), which can be measured using various Video Quality Assessment (VQA) methods. Gaming videos are synthetic and artificial in nature and have different streaming requirements as compared to traditional non-gaming content. While there exist a lot of subjective and objective studies in the field of quality assessment of Video-on-demand (VOD) streaming services, such as Netflix and YouTube, along with the design of many VQA metrics, no work has been done previously towards quality assessment of live passive gaming video streaming applications. The research work in this thesis tries to address this gap by using various subjective and objective quality assessment studies. A codec comparison using the three most popular and widely used compression standards is performed to determine their compression efficiency. Furthermore, a subjective and objective comparative study is carried out to find out the difference between gaming and non-gaming videos in terms of the trade-off between quality and data-rate after compression. This is followed by the creation of an open source gaming video dataset, which is then used for a performance evaluation study of the eight most popular VQA metrics. Different temporal pooling strategies and content based classification approaches are evaluated to assess their effect on the VQA metrics. Finally, due to the low performance of existing No-Reference (NR) VQA metrics on gaming video content, two machine learning based NR models are designed using NR features and existing NR metrics, which are shown to outperform existing NR metrics while performing on par with state-of-the-art Full-Reference (FR) VQA metrics

Methodology for Modeling and Comparing Video Codecs: HEVC, EVC, and VVC

Online videos are the major source of internet traffic, and are about to become the largest majority. Increasing effort is aimed to developing more efficient video codecs. In order to compare existing and novel video codecs, this paper presents a simple but effective methodology to model their performance in terms of Rate Distortion (RD). A linear RD model in the dB variables, Peak Signal-to-Noise Ratio (PSNR) and Bitrate (BR), easily allows us to estimate the difference in PSNR or BR between two sets of encoding conditions. Six sequences from the MPEG test set with the same resolution, encoded at different BR and different Quantization Parameters, were used to create the data set to estimate each RD model. Three codecs (HEVC, EVC, and VVC) were compared with this methodology, after estimating their models. Fitting properties of each model and a performance comparison between the models are finally shown and discussed

Methodology for modeling and comparing video codecs: Hevc, evc, and vvc

none3noOnline videos are the major source of internet traffic, and are about to become the largest majority. Increasing effort is aimed to developing more efficient video codecs. In order to compare existing and novel video codecs, this paper presents a simple but effective methodology to model their performance in terms of Rate Distortion (RD). A linear RD model in the dB variables, Peak Signal-to-Noise Ratio (PSNR) and Bitrate (BR), easily allows us to estimate the difference in PSNR or BR between two sets of encoding conditions. Six sequences from the MPEG test set with the same resolution, encoded at different BR and different Quantization Parameters, were used to create the data set to estimate each RD model. Three codecs (HEVC, EVC, and VVC) were compared with this methodology, after estimating their models. Fitting properties of each model and a performance comparison between the models are finally shown and discussed.openBattista S.; Conti M.; Orcioni S.Battista, S.; Conti, M.; Orcioni, S

Cloud media video encoding:review and challenges

In recent years, Internet traffic patterns have been changing. Most of the traffic demand by end users is multimedia, in particular, video streaming accounts for over 53%. This demand has led to improved network infrastructures and computing architectures to meet the challenges of delivering these multimedia services while maintaining an adequate quality of experience. Focusing on the preparation and adequacy of multimedia content for broadcasting, Cloud and Edge Computing infrastructures have been and will be crucial to offer high and ultra-high definition multimedia content in live, real-time, or video-on-demand scenarios. For these reasons, this review paper presents a detailed study of research papers related to encoding and transcoding techniques in cloud computing environments. It begins by discussing the evolution of streaming and the importance of the encoding process, with a focus on the latest streaming methods and codecs. Then, it examines the role of cloud systems in multimedia environments and provides details on the cloud infrastructure for media scenarios. After doing a systematic literature review, we have been able to find 49 valid papers that meet the requirements specified in the research questions. Each paper has been analyzed and classified according to several criteria, besides to inspect their relevance. To conclude this review, we have identified and elaborated on several challenges and open research issues associated with the development of video codecs optimized for diverse factors within both cloud and edge architectures. Additionally, we have discussed emerging challenges in designing new cloud/edge architectures aimed at more efficient delivery of media traffic. This involves investigating ways to improve the overall performance, reliability, and resource utilization of architectures that support the transmission of multimedia content over both cloud and edge computing environments ensuring a good quality of experience for the final user

Організація онлайн трансляції через смартфон

В дипломній роботі представлено огляд найбільш поширених методів проведення онлайн трансляцій через смартфон. Представлено зручні, якісні, бюджетні способи проведення трансляції, їх алгоритми, їх порівняння; розглянуто всю технічну і програмну частину проведення трансляцій. Показано перспективи використання смартфону для проведення онлайн трансляцій, приведені результати проведення трансляцій через платформи та через стороннє програмне забеспечення, показано завантаження на процесор та швидкість розряду батареї. Приведенно найкращі характеристики для транслювання онлайн відео. Показані алгоритми роботи кодеків, OSI моделі та програмного забеспечення.The thesis presents an overview of the most common methods of conducting online broadcasts via smartphone. Convenient, high – quality, budget methods of broadcasting, their algorithms, their comparison are presented; the whole technical and program part of the broadcasts is considered. Showing the mains reason of using a smartphone for conducting online broadcasts, the results of broadcasting through platforms and other developers software, shows the load on the processor and the battery discharge rate. Indicate the best features for streaming online video. Algorithms of operation of codecs, OSI models and software are shown

Bitstream-based video quality modeling and analysis of HTTP-based adaptive streaming

Die Verbreitung erschwinglicher Videoaufnahmetechnologie und verbesserte Internetbandbreiten ermöglichen das Streaming von hochwertigen Videos (Auflösungen > 1080p, Bildwiederholraten ≥ 60fps) online. HTTP-basiertes adaptives Streaming ist die bevorzugte Methode zum Streamen von Videos, bei der Videoparameter an die verfügbare Bandbreite angepasst wird, was sich auf die Videoqualität auswirkt. Adaptives Streaming reduziert Videowiedergabeunterbrechnungen aufgrund geringer Netzwerkbandbreite, wirken sich jedoch auf die wahrgenommene Qualität aus, weswegen eine systematische Bewertung dieser notwendig ist. Diese Bewertung erfolgt üblicherweise für kurze Abschnitte von wenige Sekunden und während einer Sitzung (bis zu mehreren Minuten). Diese Arbeit untersucht beide Aspekte mithilfe perzeptiver und instrumenteller Methoden. Die perzeptive Bewertung der kurzfristigen Videoqualität umfasst eine Reihe von Labortests, die in frei verfügbaren Datensätzen publiziert wurden. Die Qualität von längeren Sitzungen wurde in Labortests mit menschlichen Betrachtern bewertet, die reale Betrachtungsszenarien simulieren. Die Methodik wurde zusätzlich außerhalb des Labors für die Bewertung der kurzfristigen Videoqualität und der Gesamtqualität untersucht, um alternative Ansätze für die perzeptive Qualitätsbewertung zu erforschen. Die instrumentelle Qualitätsevaluierung wurde anhand von bitstrom- und hybriden pixelbasierten Videoqualitätsmodellen durchgeführt, die im Zuge dieser Arbeit entwickelt wurden. Dazu wurde die Modellreihe AVQBits entwickelt, die auf den Labortestergebnissen basieren. Es wurden vier verschiedene Modellvarianten von AVQBits mit verschiedenen Inputinformationen erstellt: Mode 3, Mode 1, Mode 0 und Hybrid Mode 0. Die Modellvarianten wurden untersucht und schneiden besser oder gleichwertig zu anderen aktuellen Modellen ab. Diese Modelle wurden auch auf 360°- und Gaming-Videos, HFR-Inhalte und Bilder angewendet. Darüber hinaus wird ein Langzeitintegrationsmodell (1 - 5 Minuten) auf der Grundlage des ITU-T-P.1203.3-Modells präsentiert, das die verschiedenen Varianten von AVQBits mit sekündigen Qualitätswerten als Videoqualitätskomponente des vorgeschlagenen Langzeitintegrationsmodells verwendet. Alle AVQBits-Varianten, das Langzeitintegrationsmodul und die perzeptiven Testdaten wurden frei zugänglich gemacht, um weitere Forschung zu ermöglichen.The pervasion of affordable capture technology and increased internet bandwidth allows high-quality videos (resolutions > 1080p, framerates ≥ 60fps) to be streamed online. HTTP-based adaptive streaming is the preferred method for streaming videos, adjusting video quality based on available bandwidth. Although adaptive streaming reduces the occurrences of video playout being stopped (called “stalling”) due to narrow network bandwidth, the automatic adaptation has an impact on the quality perceived by the user, which results in the need to systematically assess the perceived quality. Such an evaluation is usually done on a short-term (few seconds) and overall session basis (up to several minutes). In this thesis, both these aspects are assessed using subjective and instrumental methods. The subjective assessment of short-term video quality consists of a series of lab-based video quality tests that have resulted in publicly available datasets. The overall integral quality was subjectively assessed in lab tests with human viewers mimicking a real-life viewing scenario. In addition to the lab tests, the out-of-the-lab test method was investigated for both short-term video quality and overall session quality assessment to explore the possibility of alternative approaches for subjective quality assessment. The instrumental method of quality evaluation was addressed in terms of bitstream- and hybrid pixel-based video quality models developed as part of this thesis. For this, a family of models, namely AVQBits has been conceived using the results of the lab tests as ground truth. Based on the available input information, four different instances of AVQBits, that is, a Mode 3, a Mode 1, a Mode 0, and a Hybrid Mode 0 model are presented. The model instances have been evaluated and they perform better or on par with other state-of-the-art models. These models have further been applied to 360° and gaming videos, HFR content, and images. Also, a long-term integration (1 - 5 mins) model based on the ITU-T P.1203.3 model is presented. In this work, the different instances of AVQBits with the per-1-sec scores output are employed as the video quality component of the proposed long-term integration model. All AVQBits variants as well as the long-term integration module and the subjective test data are made publicly available for further research

Модернизация кроссплатформенной библиотеки для обработки видео на базе FFmpeg

В данной работе проведена модернизация библиотеки FFmpeg путем разработки оберточной кроссплатформенной (ОС Windows и ОС Android) библиотеки кодирования и декодирования видео с целью создания более удобного инструмента для обработки видео с интерфейсом на языках программирования высокого уровня. Проведено тестирование библиотеки на разрабатываемых платформах. Показано, что дополнительные абстракции, которые необходимы для высокоуровнего взаимодействия с библиотекой, не несут потерь в производительности обработки видео.In this work, the FFmpeg library has been upgraded by developing a cross-platform (Windows OS and Android OS) video library encoding and decoding library in order to create a more convenient video processing tool with an interface that defines high-level programming. . The libraries were tested on the developed platforms. It is shown that the additional abstractions required for high-level interaction with the library do not incur any loss in video processing performance