Multi-Context Dual Hyper-Prior Neural Image Compression

Transform and entropy models are the two core components in deep image compression neural networks. Most existing learning-based image compression methods utilize convolutional-based transform, which lacks the ability to model long-range dependencies, primarily due to the limited receptive field of the convolution operation. To address this limitation, we propose a Transformer-based nonlinear transform. This transform has the remarkable ability to efficiently capture both local and global information from the input image, leading to a more decorrelated latent representation. In addition, we introduce a novel entropy model that incorporates two different hyperpriors to model cross-channel and spatial dependencies of the latent representation. To further improve the entropy model, we add a global context that leverages distant relationships to predict the current latent more accurately. This global context employs a causal attention mechanism to extract long-range information in a content-dependent manner. Our experiments show that our proposed framework performs better than the state-of-the-art methods in terms of rate-distortion performance.Comment: Accepted to IEEE 22$^nd$ International Conference on Machine Learning and Applications 2023 (ICMLA) - Selected for Oral Presentatio

An Improved Upper Bound on the Rate-Distortion Function of Images

Recent work has shown that Variational Autoencoders (VAEs) can be used to upper-bound the information rate-distortion (R-D) function of images, i.e., the fundamental limit of lossy image compression. In this paper, we report an improved upper bound on the R-D function of images implemented by (1) introducing a new VAE model architecture, (2) applying variable-rate compression techniques, and (3) proposing a novel \ourfunction{} to stabilize training. We demonstrate that at least 30\% BD-rate reduction w.r.t. the intra prediction mode in VVC codec is achievable, suggesting that there is still great potential for improving lossy image compression. Code is made publicly available at https://github.com/duanzhiihao/lossy-vae.Comment: Conference paper at ICIP 2023. The first two authors share equal contribution

Generative Fingerprint Augmentation against Membership Inference Attacks

openThis thesis aspires to provide a privacy protection mechanism for neural networks concerning fingerprints. Biometric identifiers, especially fingerprints, have become crucial in the last several years, from banking operations to daily smartphone usage. Using generative adversarial networks (GANs), we train models specialized in compressing and decompressing (Codecs) images in order to augment the data these models used during the learning process to provide additional privacy preservation over the identity of the fingerprints found in such datasets. We test and analyze our framework with custom membership inference attacks (MIA) to assess the quality of our defensive mechanism.This thesis aspires to provide a privacy protection mechanism for neural networks concerning fingerprints. Biometric identifiers, especially fingerprints, have become crucial in the last several years, from banking operations to daily smartphone usage. Using generative adversarial networks (GANs), we train models specialized in compressing and decompressing (Codecs) images in order to augment the data these models used during the learning process to provide additional privacy preservation over the identity of the fingerprints found in such datasets. We test and analyze our framework with custom membership inference attacks (MIA) to assess the quality of our defensive mechanism