NN-VVC: Versatile Video Coding boosted by self-supervisedly learned image coding for machines

The recent progress in artificial intelligence has led to an ever-increasing usage of images and videos by machine analysis algorithms, mainly neural networks. Nonetheless, compression, storage and transmission of media have traditionally been designed considering human beings as the viewers of the content. Recent research on image and video coding for machine analysis has progressed mainly in two almost orthogonal directions. The first is represented by end-to-end (E2E) learned codecs which, while offering high performance on image coding, are not yet on par with state-of-the-art conventional video codecs and lack interoperability. The second direction considers using the Versatile Video Coding (VVC) standard or any other conventional video codec (CVC) together with pre- and post-processing operations targeting machine analysis. While the CVC-based methods benefit from interoperability and broad hardware and software support, the machine task performance is often lower than the desired level, particularly in low bitrates. This paper proposes a hybrid codec for machines called NN-VVC, which combines the advantages of an E2E-learned image codec and a CVC to achieve high performance in both image and video coding for machines. Our experiments show that the proposed system achieved up to -43.20% and -26.8% Bj{\o}ntegaard Delta rate reduction over VVC for image and video data, respectively, when evaluated on multiple different datasets and machine vision tasks. To the best of our knowledge, this is the first research paper showing a hybrid video codec that outperforms VVC on multiple datasets and multiple machine vision tasks.Comment: ISM 2023 Best paper award winner versio

NN-VVC : Versatile Video Coding boosted by self-supervisedly learned image coding for machines

The recent progress in artificial intelligence has led to an ever-increasing usage of images and videos by machine analysis algorithms, mainly neural networks. Nonetheless, compression, storage and transmission of media have traditionally been designed considering human beings as the viewers of the content. Recent research on image and video coding for machine analysis has progressed mainly in two almost orthogonal directions. The first is represented by end-to-end (E2E) learned codecs which, while offering high performance on image coding, are not yet on par with state-of-the-art conventional video codecs and lack interoperability. The second direction considers using the Versatile Video Coding (VVC) standard or any other conventional video codec (CVC) together with pre- and post-processing operations targeting machine analysis. While the CVC-based methods benefit from interoperability and broad hardware and software support, the machine task performance is often lower than the desired level, particularly in low bitrates. This paper proposes a hybrid codec for machines called NN-VVC, which combines the advantages of an E2E-learned image codec and a CVC to achieve high performance in both image and video coding for machines. Our experiments show that the proposed system achieved up to -43.20% and -26.8% Bjontegaard Delta rate reduction over VVC for image and video data, respectively, when evaluated on multiple different datasets and machine vision tasks. To the best of our knowledge, this is the first research paper showing a hybrid video codec that outperforms VVC on multiple datasets and multiple machine vision tasks.Peer reviewe

Learned Enhancement Filters for Image Coding for Machines

Machine-To-Machine (M2M) communication applications and use cases, such as object detection and instance segmentation, are becoming mainstream nowadays. As a consequence, majority of multimedia content is likely to be consumed by machines in the coming years. This opens up new challenges on efficient compression of this type of data. Two main directions are being explored in the literature, one being based on existing traditional codecs, such as the Versatile Video Coding (VVC) standard, that are optimized for human-Targeted use cases, and another based on end-To-end trained neural networks. However, traditional codecs have significant benefits in terms of interoperability, real-Time decoding, and availability of hardware implementations over end-To-end learned codecs. Therefore, in this paper, we propose learned post-processing filters that are targeted for enhancing the performance of machine vision tasks for images reconstructed by the VVC codec. The proposed enhancement filters provide significant improvements on the target tasks compared to VVC coded images. The conducted experiments show that the proposed post-processing filters provide about 45% and 49% Bjontegaard Delta Rate gains over VVC in instance segmentation and object detection tasks, respectively.acceptedVersionPeer reviewe

Longitudinal Metabolome-Wide Signals Prior to the Appearance of a First Islet Autoantibody in Children Participating in the TEDDY Study

Children at increased genetic risk for type 1 diabetes (T1D) after environmental exposures may develop pancreatic islet autoantibodies (IA) at a very young age. Metabolic profile changes over time may imply responses to exposures and signal development of the first IA. Our present research in The Environmental Determinants of Diabetes in the Young (TEDDY) study aimed to identify metabolome-wide signals preceding the first IA against GAD (GADA-first) or against insulin (IAA-first). We profiled metabolomes by mass spectrometry from children's plasma at 3-month intervals after birth until appearance of the first IA. A trajectory analysis discovered each first IA preceded by reduced amino acid proline and branched-chain amino acids (BCAAs), respectively. With independent time point analysis following birth, we discovered dehydroascorbic acid (DHAA) contributing to the risk of each first IA, and γ-aminobutyric acid (GABAs) associated with the first autoantibody against insulin (IAA-first). Methionine and alanine, compounds produced in BCAA metabolism and fatty acids, also preceded IA at different time points. Unsaturated triglycerides and phosphatidylethanolamines decreased in abundance before appearance of either autoantibody. Our findings suggest that IAA-first and GADA-first are heralded by different patterns of DHAA, GABA, multiple amino acids, and fatty acids, which may be important to primary prevention of T1D

Heme oxygenase-1 repeat polymorphism in septic acute kidney injury

Abstract Acute kidney injury (AKI) is a syndrome that frequently affects the critically ill. Recently, an increased number of dinucleotide repeats in the HMOX1 gene were reported to associate with development of AKI in cardiac surgery. We aimed to test the replicability of this finding in a Finnish cohort of critically ill septic patients. This multicenter study was part of the national FINNAKI study. We genotyped 300 patients with severe AKI (KDIGO 2 or 3) and 353 controls without AKI (KDIGO 0) for the guanine–thymine (GTn) repeat in the promoter region of the HMOX1 gene. The allele calling was based on the number of repeats, the cut off being 27 repeats in the S–L (short to long) classification, and 27 and 34 repeats for the S–M–L₂ (short to medium to very long) classification. The plasma concentrations of heme oxygenase-1 (HO-1) enzyme were measured on admission. The allele distribution in our patients was similar to that published previously, with peaks at 23 and 30 repeats. The S-allele increases AKI risk. An adjusted OR was 1.30 for each S-allele in an additive genetic model (95% CI 1.01–1.66; p = 0.041). Alleles with a repeat number greater than 34 were significantly associated with lower HO-1 concentration (p<0.001). In septic patients, we report an association between a short repeat in HMOX1 and AKI risk