Perceptual Image Compression with Cooperative Cross-Modal Side Information

The explosion of data has resulted in more and more associated text being transmitted along with images. Inspired by from distributed source coding, many works utilize image side information to enhance image compression. However, existing methods generally do not consider using text as side information to enhance perceptual compression of images, even though the benefits of multimodal synergy have been widely demonstrated in research. This begs the following question: How can we effectively transfer text-level semantic dependencies to help image compression, which is only available to the decoder? In this work, we propose a novel deep image compression method with text-guided side information to achieve a better rate-perception-distortion tradeoff. Specifically, we employ the CLIP text encoder and an effective Semantic-Spatial Aware block to fuse the text and image features. This is done by predicting a semantic mask to guide the learned text-adaptive affine transformation at the pixel level. Furthermore, we design a text-conditional generative adversarial networks to improve the perceptual quality of reconstructed images. Extensive experiments involving four datasets and ten image quality assessment metrics demonstrate that the proposed approach achieves superior results in terms of rate-perception trade-off and semantic distortion

Progressive Learning with Visual Prompt Tuning for Variable-Rate Image Compression

In this paper, we propose a progressive learning paradigm for transformer-based variable-rate image compression. Our approach covers a wide range of compression rates with the assistance of the Layer-adaptive Prompt Module (LPM). Inspired by visual prompt tuning, we use LPM to extract prompts for input images and hidden features at the encoder side and decoder side, respectively, which are fed as additional information into the Swin Transformer layer of a pre-trained transformer-based image compression model to affect the allocation of attention region and the bits, which in turn changes the target compression ratio of the model. To ensure the network is more lightweight, we involves the integration of prompt networks with less convolutional layers. Exhaustive experiments show that compared to methods based on multiple models, which are optimized separately for different target rates, the proposed method arrives at the same performance with 80% savings in parameter storage and 90% savings in datasets. Meanwhile, our model outperforms all current variable bitrate image methods in terms of rate-distortion performance and approaches the state-of-the-art fixed bitrate image compression methods trained from scratch

How do urban residents use energy for winter heating at home? A large-scale survey in the hot summer and cold winter climate zone in the Yangtze River region

The increasing demand for improving indoor thermal environment in the hot summer and cold winter climate zone (HSCW) in the Yangtze River region in China poses enormous challenges in terms of energy policy and design solutions for this unique region. A comprehensive understanding of people’s habits and behaviors involving winter heating is imperative for decision making for urban heating infrastructure investment strategies that significantly impact the decarbonization of heating. However, there are little studies of a large-scale survey to gain such knowledge acrose the region. The aim of this study is to develop a rigorous survey method in order to obtain reliable data for analysis. Five municipal/capital cities across the upper, middle and downstream Yangtze River were surveyed based on 30 randomly generated locations in each city. A total of 8481 valuable samples were obtained in the survey conducted in the winter from November 2017 to March 2018. It is revealed that air conditioning/air source heat pumps are the predominant systems, accounting for 63% and 58% for bedroom and living room heating respectively. The use patterns of heating are diverse featuring ‘part-time-part-space’ systems in accordance with the occupancy patterns. There is significant evidence of the habit of opening a window to provide a gap for fresh air irrespective of whether the heating is in use. Two-step cluster analysis is employed to subdivide occupants’ heating-related behaviors into three clusters to characterize households. This study fills the knowledge gap of winter-heating-related behaviors. The research outcomes will benefit building energy simulations for energy prediction and help policy makers making decisions on providing strategic guidance in terms of winter heating solutions in this region

Robust estimation of bacterial cell count from optical density

Optical density (OD) is widely used to estimate the density of cells in liquid culture, but cannot be compared between instruments without a standardized calibration protocol and is challenging to relate to actual cell count. We address this with an interlaboratory study comparing three simple, low-cost, and highly accessible OD calibration protocols across 244 laboratories, applied to eight strains of constitutive GFP-expressing E. coli. Based on our results, we recommend calibrating OD to estimated cell count using serial dilution of silica microspheres, which produces highly precise calibration (95.5% of residuals <1.2-fold), is easily assessed for quality control, also assesses instrument effective linear range, and can be combined with fluorescence calibration to obtain units of Molecules of Equivalent Fluorescein (MEFL) per cell, allowing direct comparison and data fusion with flow cytometry measurements: in our study, fluorescence per cell measurements showed only a 1.07-fold mean difference between plate reader and flow cytometry data