A Bayesian Approach to Block Structure Inference in AV1-based Multi-rate Video Encoding

Due to differences in frame structure, existing multi-rate video encoding algorithms cannot be directly adapted to encoders utilizing special reference frames such as AV1 without introducing substantial rate-distortion loss. To tackle this problem, we propose a novel bayesian block structure inference model inspired by a modification to an HEVC-based algorithm. It estimates the posterior probabilistic distributions of block partitioning, and adapts early terminations in the RDO procedure accordingly. Experimental results show that the proposed method provides flexibility for controlling the tradeoff between speed and coding efficiency, and can achieve an average time saving of 36.1% (up to 50.6%) with negligible bitrate cost.Comment: published in IEEE Data Compression Conference, 201

Avoimen lähdekoodin HEVC-pilvitranskoodausjärjestelmä

Kyky tallentaa valtavia määriä videokuvaa vaatii helppokäyttöisiä ja tehokkaita videonkoodausjärjestelmiä, joiden avulla voidaan mukautua rajallisiin lähetys- ja tallennuskapasiteetteihin. Tässä työssä esitellään avoimen lähdekoodin pilvipalvelu, jonka avulla pystyy transkoodaamaan videoita H.265/HEVC-formaattiin. Vaihtoehtoisia kaupallisia toteutuksia on saatavilla, mutta ne ovat maksumuurien takana. Komentorivikäyttöliittymien käyttäminen taas vaatii syvällistä ymmärtämistä pakkausprosessista parhaan mahdollisen laadun ja nopeuden saavuttamiseksi. Esitelty järjestelmä on helppokäyttöinen, avoimen lähdekoodin toteutus, joka tekee siitä helposti lähestyttävän myös ei-teknisesti valveutuneille käyttäjille. Se on rakennettu käyttämällä FFmpeg-multimediatyökalua, jonka avulla pystyy dekoodaamaan paljon erilaisia sisääntuloja sekä Kvazaar-nimistä HEVC-koodainta videonpakkaukseen

Rinnakkainen toteutus H.265 videokoodaus standardille

The objective of this study was to research the scalability of the parallel features in the new H.265 video compression standard, also know as High Efficiency Video Coding (HEVC). Compared to its predecessor, the H.264 standard, H.265 typically achieves around 50% bitrate reduction for the same subjective video quality. Especially videos with higher resolution (Full HD and beyond) achieve better compression ratios. Also a better utilization of parallel computing resources is provided. H.265 introduces two novel parallelization features: Tiles and Wavefront Parallel Processing (WPP). In Tiles, each video frame is divided into areas that can be decoded without referencing to other areas in the same frame. In WPP, the relations between code blocks in a frame are encoded so that the decoding process can progress through the frame as a front using multiple threads. In this study, the reference implementation for the H.265 decoder was augmented to support both of these parallelization features. The performance of the parallel implementations was measured using three different setups. From the measurement results it could be seen that the introduction of more CPU cores reduced the total decode time of the video frames to a certain point. When using the Tiles feature, it was observed that the encoding geometry, i.e. how each frame was divided into individually decodable areas, had a noticeable effect on the decode times with certain thread counts. When using WPP, it was observed that what was mostly synchronization overhead, sometimes had a negative effect on the decode times when using larger (4-12) amounts of threads.Tämän tutkimuksen aiheena oli tutkia uuden H.265 videonpakkausstandardin (tunnetaan myös nimellä HEVC (engl. High Efficiency Video Coding)) rinnakkaisuusominaisuuksien skaalautuvuutta. Verrattuna edeltäjäänsä, H.264 videonpakkaustandardiin, H.265 tyypillisesti saavuttaa samalla kuvanlaadulla noin 50% pienemmän pakkauskoon. Erityisesti suuren resoluution videoilla (Full HD ja suuremmat) pakkaustehokkuuden paremmuus korostuu. Huomiota on kiinnitetty myös moniydinprosessoreiden hyödyntämiseen videokoodauksessa. H.265 tarjoaa kaksi uutta rinnakkaisuusominaisuutta: niin kutsutut Tiles- ja WPP-menetelmät (engl. \emph{Wavefront Parallel Processing}). Tiles-menetelmässä jokainen videon kuva jaetaan alueisiin, jotka voidaan purkaa viittaamatta saman kuvan muihin alueisiin. WPP-menetelmässä suhteet kuvan lohkoihin pakataan siten että purkamisprosessi pystyy etenemään kuvan läpi rintamana hyödyntäen useampia säikeitä. Tässä tutkimuksessa H.265 videodekooderin referenssitoteutusta laajennettiin tukemaan molempia näistä rinnakkaisuusominaisuuksista. Suorituskykyä mitattiin käyttäen kolmea eri mittausasetelmaa. Mittaustuloksista ilmeni, että prosessoriydinten lukumäärän kasvattaminen nopeutti videoiden purkamista tiettyyn pisteeseen asti. Tiles-menetelmää mitatessa havaittiin, että alueiden geometrialla, eli kuinka kuva jaettiin riippumattomiin alueisiin, on huomattava vaikutus purkamisnopeuteen tietyillä säiemäärillä. WPP-menetelmää mitattaessa havaittiin että korkeampiin säiemääriin (4-12) siirryttäessä purkamisnopeus alkoi hidastua. Tämä johtui pääasiassa säikeiden keskinäiseen synkronointiin kuluvasta ajasta

Image and Video Coding Techniques for Ultra-low Latency

The next generation of wireless networks fosters the adoption of latency-critical applications such as XR, connected industry, or autonomous driving. This survey gathers implementation aspects of different image and video coding schemes and discusses their tradeoffs. Standardized video coding technologies such as HEVC or VVC provide a high compression ratio, but their enormous complexity sets the scene for alternative approaches like still image, mezzanine, or texture compression in scenarios with tight resource or latency constraints. Regardless of the coding scheme, we found inter-device memory transfers and the lack of sub-frame coding as limitations of current full-system and software-programmable implementations.publishedVersionPeer reviewe

Algorithms and methods for video transcoding.

Video transcoding is the process of dynamic video adaptation. Dynamic video adaptation can be defined as the process of converting video from one format to another, changing the bit rate, frame rate or resolution of the encoded video, which is mainly necessitated by the end user requirements. H.264 has been the predominantly used video compression standard for the last 15 years. HEVC (High Efficiency Video Coding) is the latest video compression standard finalised in 2013, which is an improvement over H.264 video compression standard. HEVC performs significantly better than H.264 in terms of the Rate-Distortion performance. As H.264 has been widely used in the last decade, a large amount of video content exists in H.264 format. There is a need to convert H.264 video content to HEVC format to achieve better Rate-Distortion performance and to support legacy video formats on newer devices. However, the computational complexity of HEVC encoder is 2-10 times higher than that of H.264 encoder. This makes it necessary to develop low complexity video transcoding algorithms to transcode from H.264 to HEVC format. This research work proposes low complexity algorithms for H.264 to HEVC video transcoding. The proposed algorithms reduce the computational complexity of H.264 to HEVC video transcoding significantly, with negligible loss in Rate-Distortion performance. This work proposes three different video transcoding algorithms. The MV-based mode merge algorithm uses the block mode and MV variances to estimate the split/non-split decision as part of the HEVC block prediction process. The conditional probability-based mode mapping algorithm models HEVC blocks of sizes 16×16 and lower as a function of H.264 block modes, H.264 and HEVC Quantisation Parameters (QP). The motion-compensated MB residual-based mode mapping algorithm makes the split/non-split decision based on content-adaptive classification models. With a combination of the proposed set of algorithms, the computational complexity of the HEVC encoder is reduced by around 60%, with negligible loss in Rate-Distortion performance, outperforming existing state-of-art algorithms by 20-25% in terms of computational complexity. The proposed algorithms can be used in computation-constrained video transcoding applications, to support video format conversion in smart devices, migration of large-scale H.264 video content from host servers to HEVC, cloud computing-based transcoding applications, and also to support high quality videos over bandwidth-constrained networks

Visual Content Characterization Based on Encoding Rate-Distortion Analysis

Visual content characterization is a fundamentally important but under exploited step in dataset construction, which is essential in solving many image processing and computer vision problems. In the era of machine learning, this has become ever more important, because with the explosion of image and video content nowadays, scrutinizing all potential content is impossible and source content selection has become increasingly difficult. In particular, in the area of image/video coding and quality assessment, it is highly desirable to characterize/select source content and subsequently construct image/video datasets that demonstrate strong representativeness and diversity of the visual world, such that the visual coding and quality assessment methods developed from and validated using such datasets exhibit strong generalizability. Encoding Rate-Distortion (RD) analysis is essential for many multimedia applications. Examples of applications that explicitly use RD analysis include image encoder RD optimization, video quality assessment (VQA), and Quality of Experience (QoE) optimization of streaming videos etc. However, encoding RD analysis has not been well investigated in the context of visual content characterization. This thesis focuses on applying encoding RD analysis as a visual source content characterization method with image/video coding and quality assessment applications in mind. We first conduct a video quality subjective evaluation experiment for state-of-the-art video encoder performance analysis and comparison, where our observations reveal severe problems that motivate the needs of better source content characterization and selection methods. Then the effectiveness of RD analysis in visual source content characterization is demonstrated through a proposed quality control mechanism for video coding by eigen analysis in the space of General Quality Parameter (GQP) functions. Finally, by combining encoding RD analysis with submodular set function optimization, we propose a novel method for automating the process of representative source content selection, which helps boost the RD performance of visual encoders trained with the selected visual contents

Arquitectura híbrida para el estándar de compresión de vídeo H.265/HEVC

En las últimas décadas las aplicaciones multimedia han evolucionado enormemente. Los estándares de compresión de vídeo han contribuido notablemente a este avance convirtiéndose en esenciales para la transmisión de datos multimedia. Desde 1980, los estándares de compresión de vídeo han enfocado sus principales esfuerzos a reducir la tasa de datos generada y proporcionar un alto nivel de calidad visual. Debido a la creciente demanda de contenido digital, hoy en día resulta de vital importancia optimizar el ancho de banda utilizado para la transmisión de vídeo digital, buscando obtener grandes tasas de compresión de datos sin pérdidas apreciables en la calidad del servicio. Veinte años atrás se consideraba a la compresión de vídeo como algo novedoso, actualmente es algo ubicuo. La mayoría de dispositivos que constituyen la electrónica de consumo hacen uso de una manera u otra de la compresión de vídeo. Aunque la mayoría de usuarios asume que estas mejoras son el resultado de los avances generales que se han producido en la electrónica de consumo, lo cierto es que son principalmente debidas a la compresión de vídeo..

Analisi degli standard di codifica per sequenze video 4K e HDR

La fruizione di contenuti multimediali in forma digitale è un aspetto quotidiano della nostra vita. Tra di essi i video svolgono un ruolo molto importante. Tuttavia la crescente richiesta di una maggiore qualità di visualizzazione e una distribuzione dei contenuti sempre più efficiente portano ad un aumento dei requisiti in termini di prestazioni. Pertanto, le tecniche di compressione video risultano in continua evoluzione. Questa tesi ha l’obiettivo di presentare la teoria di base su cui si fonda la codifica video, per poi descrivere gli standard sviluppati fino ad oggi. Infine si concentra sul confronto tra gli standard di codifica video più recenti, H.264/AVC e H.265/HEVC. I risultati sperimentali mostrano come quest’ultimo permetta di ottenere un notevole miglioramento dell’efficienza di codifica, in particolar modo riducendo la dimensione del bit stream codificato fino al 50%

Dynamic simulation and control of optical systems

This thesis deals with the simulation-based investigation and control of optical systems that are mechanically influenced. Here, the focus is on the dynamic-optical modeling of vibration-sensitive mirror systems, which are utilized, e.g., in large astronomy telescopes or high-precision lithography optics. The large-area primary mirrors of telescopes typically consist of many individual hexagonal mirror segments, which are positioned with precise sensors and actuators. Furthermore, an adaptive optical unit usually compensates for the optical aberrations due to atmospheric disturbances. In practice, these aberrations are detected, and corrected, within a few seconds using deformable mirrors. However, to further improve the performance of these optical systems, dynamical disturbances in the mechanics, i.e., small movements and deformations of the optical surfaces, must also be taken into account. Therefore, multidisciplinary simulation methods are developed and presented. Based on this, the dynamical-optical system behavior is modeled using model-order-reduced, flexible multibody systems. Hence, the dynamical analysis of the mechanical-optical system can be performed at low computation costs. Thanks to the optical analysis in the time domain and using Fourier-optical concepts, one can also simulate exposure processes. To actively compensate for aberrations due to mechanical vibrations, model-based control strategies are also designed. They are not only demonstrated by means of simulation examples, but also illustrated through a laboratory experiment. The latter is realized with a low-cost test setup for student training using Arduino microcontrollers, position and force sensors, as well as high-speed cameras