On Versatile Video Coding at UHD with Machine-Learning-Based Super-Resolution

Coding 4K data has become of vital interest in recent years, since the amount of 4K data is significantly increasing. We propose a coding chain with spatial down- and upscaling that combines the next-generation VVC codec with machine learning based single image super-resolution algorithms for 4K. The investigated coding chain, which spatially downscales the 4K data before coding, shows superior quality than the conventional VVC reference software for low bitrate scenarios. Throughout several tests, we find that up to 12 % and 18 % Bjontegaard delta rate gains can be achieved on average when coding 4K sequences with VVC and QP values above 34 and 42, respectively. Additionally, the investigated scenario with up- and downscaling helps to reduce the loss of details and compression artifacts, as it is shown in a visual example.Comment: Originally published as conference paper at QoMEX 202

A Bit Stream Feature-Based Energy Estimator for HEVC Software Encoding

The total energy consumption of today's video coding systems is globally significant and emphasizes the need for sustainable video coder applications. To develop such sustainable video coders, the knowledge of the energy consumption of state-of-the-art video coders is necessary. For that purpose, we need a dedicated setup that measures the energy of the encoding and decoding system. However, such measurements are costly and laborious. To this end, this paper presents an energy estimator that uses a subset of bit stream features to accurately estimate the energy consumption of the HEVC software encoding process. The proposed model reaches a mean estimation error of 4.88% when averaged over presets of the x265 encoder implementation. The results from this work help to identify properties of encoding energy-saving bit streams and, in turn, are useful for developing new energy-efficient video coding algorithms.Comment: arXiv admin note: text overlap with arXiv:2207.0267

Advanced Design Space Exploration for Joint Energy and Quality Optimization for VVC

In recent studies, it could be shown that the energy demand of Versatile Video Coding (VVC) decoders can be twice as high as comparable High Efficiency Video Coding (HEVC) decoders. A significant part of this increase in complexity is attributed to the usage of new coding tools. By using a design space exploration algorithm, it was shown that the energy demand of VVC-coded sequences could be reduced if different coding tool profiles were used for the encoding process. This work extends the algorithm with several optimization strategies, methodological adjustments to optimize perceptual quality, and a new minimization criterion. As a result, we significantly improve the Pareto front, and the rate-distortion and energy efficiency of the state-of-the-art design space exploration. Therefore, we show an energy demand reduction of up to 47% with less than 30% additional bit rate, or a reduction of over 35% with approximately 6% additional bit rate.Comment: accepted as a conference paper for Picture Coding Symposium (PCS) 202

Protein arginine methyltransferase 6 regulates the function of the hematopoietic transcription factor RUNX1

Die Bildung aller Blutzellen ist ein vielschichtiger Prozess aus Proliferation und Differenzierung der pluripotenten Stammzellen. Die Regulation der hämatopoietischen Differenzierung wird unter anderem durch ein Zusammenspiel von zelltypspezifischen Transkriptionsfaktoren gewährleistet (Barreda & Belosevic, 2001). Der Transkriptionsfaktor RUNX1 spielt eine wichtige Rolle bei der Entwicklung von hämatopoietischen Stammzellen und eine funktionelle Veränderung von RUNX1 führt zur Ausbildung von Leukämien. In humanen Leukämien ist RUNX1 ein häufiges Ziel von chromosomalen Translokationen. Außerdem ist die Haploinsuffizienz von RUNX1 beim Menschen ursächlich für familiäre Thrombozytopenien (FPD) mit reduzierten Thrombozytenzahlen (Luddy et al., 1978; Gerrard et al., 1991). Ein knock out von RUNX1 in adulten Mäusen zu einem Defekt in der Megakaryozytendifferenzierung und zu Myelodysplasien (Growney et al., 2005; Ichikawa et al., 2004). Die Funktion von RUNX1 wird unter anderem durch die Interaktion mit anderen Transkriptionsfaktoren und Cofaktoren reguliert. Dabei kann RUNX1 die Expression von Zielgenen aktivieren oder reprimieren, abhängig davon welche Cofaktoren rekrutiert werden. Die Rekrutierung von Chromatin-modifizierenden Cofaktoren durch RUNX1 führt zur Veränderung der Chromatinstruktur und leitet epigenetische Regulationsprozesse ein. Bekannte Histon-modifizierende Interaktionspartner von RUNX1 sind p300 und HDAC1. p300 besitzt Histon-Aceyltransferase-Aktivität und führt zur Ausbildung offener Chromatinstrukturen, welche dann Transkriptionsfaktoren erlaubt an die DNA zu binden und somit zur Aktivierung der Expression von Zielgenen beiträgt (Vogelauer et al., 2000). Der Gegenspieler von Acetyltransferasen sind Histon Deacetylasen (z.B. HDAC1), welche Acetylgruppen entfernen und somit reprimierend auf die Genexpression wirken (Berger, 2007). Bestimmte Zelltypen unterscheiden sich stark voneinander trotz eines gemeinsamen genetischen Codes. Die Expression von zelllinienspezifischen Genen muss daher zelltypspezifisch kontrolliert werden. Dies geschieht durch epigenetische Regulationsvorgänge, welche stammzell-spezifische Gene abschalten, wohingegen zelllinienspezifische Gene angeschaltet werden. Neben der Acetylierung von Histonen ist die Methylierung eine häufige posttranslationale Modifikation, die im Histone Code eine wichtige Rolle spielt. Diese Histonmarkierung wird von Methyltransferasen katalysiert. Die epigenetische Regulation von RUNX1-Zielgenen während der Differenzierung von Vorläufer-/Stammzellen wurde bislang noch nicht näher aufgeklärt. Aus diesem Grund sollten neue Interaktionspartner von RUNX1 mit epigenetischer Funktion identifiziert werden. In dieser Arbeit konnte eine Interaktion von RUNX1 mit der Protein Arginin Methyltransferase 6 (PRMT6) gezeigt werden. Expressionsanalysen wiesen eine Expression von RUNX1 und PRMT6 in Stamm-/Vorläuferzellen nach. Während der Megakaryozytendifferenzierung verhielt sich das Expressionsmuster von PRMT6 gegensätzlich zu der RUNX1-Expression. Zur Überprüfung der Auswirkung von PRMT6 auf die Megakaryozytendifferenzierung, wurden ein knock down sowie eine Überexpression von PRMT6 durchgeführt. Die veränderte CD41-Expression einer Beeinflussung des PRMT6-Levels deutet auf einen reprimierenden Effekt von PRMT6 auf die Megakaryopoiese hin. Das daran anschließende Ziel bestand in dem Nachweis der kooperativen Regulation von differenzierungsspezifischen Genen durch PRMT6 und RUNX1. PRMT6 konnte als Teil eines RUNX1-Corepressorkomplexes mit Sin3a und HDAC1 auf dem Promotorbereich von RUNX1-Zielgenen in hämatopoietischen Vorläuferzellen nachgewiesen werden. Ein in vitro Methyltransferaseassay lieferte Hinweise darauf, dass RUNX1 am Arginin 307 von PRMT6 methyliert wird, was zu einer verstärkten Interaktion beider Faktoren, einer verminderten Aktivierungsfähigkeit und einer erhöhten DNA-Bindungskapazität von RUNX1 führt. Außerdem konnte eine Methylierung des H3R2 am Promotor von megakaryozytären Genen durch PRMT6 gezeigt werden, welche die Bindung des WDR5/MLL-Komplexes an die H3K4me2 Markierung verhindert und somit eine Trimethylierung von H3K4 inhibiert (Hyllus et al., 2007). Auf dem Promotorbereich ist neben den reprimierenden Histonmodifikationen H3R2me2 und H3K27me3 die aktivierende H3K4me2+/me3-Histonmodifikation, sowie die initiierende RNA-Polymerase II vorhanden. Die Gene befinden sich in einem Zwischenzustand (intermediary state) und werden basal transkribiert. Während der Differenzierung in Richtung Megakaryozyten konnte ein Austausch des Corepressorkomplexes gegen einen Coaktivatorkomplex mit p300, PCAF, WDR5, PRMT1 und GATA1/FOG1 beobachtet werden. Ein Verlust von PRMT6 führte zu einer Verringerung der H3R2me2 Markierung. Dadurch konnte WDR5 an den Promotor binden und die H3K4-Trimethylierung wurde durch den WDR5/MLL-Komplex katalysiert. Zusätzlich konnte eine Acetylierung des Promotorbereiches und die Belegung des Promotorbereiches mit der elongierenden RNA-Polymerase II, phosphoryliert am Serin 2, nachgewiesen werden, was zur Induktion der Expression der megakaryozytären Gene führte. Zusammenfassend konnte PRMT6 als neuer Interaktionspartner von RUNX1 identifiziert werden, welcher durch die H3R2-Methylierung differenzierungsspezifische Gene in einem Zwischenzustand hält und reprimierend auf die Megakaryozytendifferenzierung wirkt. Die Expression der intermediary state Gene kann während der Differenzierung schnell aktiviert werden. Zusätzlich konnte eine Methylierung von RUNX1 durch PRMT6 gezeigt werden.The differentiation process from stem cells to mature cells is controlled by a cell-specific transcriptional network, in which transcription factors activate or repress genes according to the lineage differentiation program of a given cell. This program is regulated by external signals, which influence the activity of transcription factors and the status of chromatin (Barreda & Belosevic, 2001). The transcription factor RUNX1 plays a central role in hematopoiesis and angiogenesis and its activity is altered by mutations, deletions and chromosomal translocations in leukemia. RUNX1 is required for the emergence of adult hematopoietic stem cells (HSC) during embryonic development and for proper lineage differentiation from HSC to mature blood cells. Haploinsufficiency of RUNX1 causes familial platelet disorders in humans (FPD/AML) (Luddy et al., 1978; Gerrard et al., 1991). Furthermore, a deletion of RUNX1 leads to an impairment of platelet formation and megakaryocyte differentiation (Growney et al., 2005; Ichikawa et al., 2004). The function of RUNX1 is regulated by interaction with several transcription factors and cofactors. RUNX1 is capable to function as a repressor or as an activator. So far mSin3a and HDACs (histone deacetylase) have been identified as RUNX1 co-repressors. RUNX1 can also recruit co-activator proteins like p300/CBP to regulatory elements of genes and activate their expression (Kitabayashi et al., 1998; Wang et al., 2009). Hematopoietic cell differentiation involves extensive reorganization of chromatin, but chromatin modifications like histone modifications also contribute to the maintenance of a stable expression pattern in the stem cell or terminally differentiated cells. RUNX1 is an important transcription factor for the epigenetic state of target genes during hematopoietic differentiation by recruiting cofactors (Yoshida & Kitabayashi, 2008). Protein arginine methyltransferases (PRMTs) are a new family of enzymes that catalyse the methylation of arginines on non-histone and histone proteins and thereby contribute to the histone code (Bedford, 2007). We identified protein arginine methyltransferase 6 (PRMT6) as a new interaction partner of RUNX1. RUNX1 is methylated by PRMT6 and there is some evidence that this methylation influence the interaction of RUNX1 with PRMT6 and the transactivation capacity of RUNX1. PRMT6 is recruited to RUNX1 target gene promoters in hematopoietic progenitor cells and is part of a histone modification complex. We found that PRMT6 is responsible for histone H3 arginine-2 dimethylation (H3R2me2) that represses H3K4me3 but does not effect H3K4me2. This keeps the promoter in a repressed but poised state for transcriptional activation. During megakaryocytic differentiation PRMT6 is down regulated, the repressive histone marks are removed and transcription of RUNX1 target genes is induced. Our results provide novel mechanistic insight into how RUNX1 activity in hematopoietic progenitor cells maintains differentiation genes in a suppressed state but poised for rapid transcriptional activation

Sweet Streams are Made of This: The System Engineer's View on Energy Efficiency in Video Communications

In recent years, the global use of online video services has increased rapidly. Today, a manifold of applications, such as video streaming, video conferencing, live broadcasting, and social networks, make use of this technology. A recent study found that the development and the success of these services had as a consequence that, nowadays, more than 1% of the global greenhouse-gas emissions are related to online video, with growth rates close to 10% per year. This article reviews the latest findings concerning energy consumption of online video from the system engineer's perspective, where the system engineer is the designer and operator of a typical online video service. We discuss all relevant energy sinks, highlight dependencies with quality-of-service variables as well as video properties, review energy consumption models for different devices from the literature, and aggregate these existing models into a global model for the overall energy consumption of a generic online video service. Analyzing this model and its implications, we find that end-user devices and video encoding have the largest potential for energy savings. Finally, we provide an overview of recent advances in energy efficiency improvement for video streaming and propose future research directions for energy-efficient video streaming services.Comment: 16 pages, 5 figures, accepted for IEEE Circuits and Systems Magazin

Beyond Bj{\o}ntegaard: Limits of Video Compression Performance Comparisons

For 20 years, the gold standard to evaluate the performance of video codecs is to calculate average differences between ratedistortion curves, also called the "Bj{\o}ntegaard Delta". With the help of this tool, the compression performance of codecs can be compared. In the past years, we could observe that the calculus was also deployed for other metrics than bitrate and distortion in terms of peak signal-to-noise ratio, for example other quality metrics such as video multi-method assessment fusion or hardware-dependent metrics such as the decoding energy. However, it is unclear whether the Bj{\o}ntegaard Delta is a valid way to evaluate these metrics. To this end, this paper reviews several interpolation methods and evaluates their accuracy using different performancemetrics. As a result, we propose to use a novel approach based on Akima interpolation, which returns the most accurate results for a large variety of performance metrics. The approximation accuracy of this new method is determined to be below a bound of 1.5%.Comment: Including submission information. This work has been accepted for IEEE International Conference on Image Processing (ICIP) 2022. Copyright may be transferred without notice, after which this version may no longer be accessibl

Learning Frequency-Specific Quantization Scaling in VVC for Standard-Compliant Task-driven Image Coding

Today, visual data is often analyzed by a neural network without any human being involved, which demands for specialized codecs. For standard-compliant codec adaptations towards certain information sinks, HEVC or VVC provide the possibility of frequency-specific quantization with scaling lists. This is a well-known method for the human visual system, where scaling lists are derived from psycho-visual models. In this work, we employ scaling lists when performing VVC intra coding for neural networks as information sink. To this end, we propose a novel data-driven method to obtain optimal scaling lists for arbitrary neural networks. Experiments with Mask R-CNN as information sink reveal that coding the Cityscapes dataset with the proposed scaling lists result in peak bitrate savings of 8.9 % over VVC with constant quantization. By that, our approach also outperforms scaling lists optimized for the human visual system. The generated scaling lists can be found under https://github.com/FAU-LMS/VCM_scaling_lists.Comment: Originally submitted at IEEE ICIP 202

Component-wise Power Estimation of Electrical Devices Using Thermal Imaging

This paper presents a novel method to estimate the power consumption of distinct active components on an electronic carrier board by using thermal imaging. The components and the board can be made of heterogeneous material such as plastic, coated microchips, and metal bonds or wires, where a special coating for high emissivity is not required. The thermal images are recorded when the components on the board are dissipating power. In order to enable reliable estimates, a segmentation of the thermal image must be available that can be obtained by manual labeling, object detection methods, or exploiting layout information. Evaluations show that with low-resolution consumer infrared cameras and dissipated powers larger than 300mW, mean estimation errors of 10% can be achieved.Comment: 10 pages, 8 figure

Power Reduction Opportunities on End-User Devices in Quality-Steady Video Streaming

This paper uses a crowdsourced dataset of online video streaming sessions to investigate opportunities to reduce the power consumption while considering QoE. For this, we base our work on prior studies which model both the end-user's QoE and the end-user device's power consumption with the help of high-level video features such as the bitrate, the frame rate, and the resolution. On top of existing research, which focused on reducing the power consumption at the same QoE optimizing video parameters, we investigate potential power savings by other means such as using a different playback device, a different codec, or a predefined maximum quality level. We find that based on the power consumption of the streaming sessions from the crowdsourcing dataset, devices could save more than 55% of power if all participants adhere to low-power settings.Comment: 4 pages, 3 figure