Resfusion: Prior Residual Noise embedded Denoising Diffusion Probabilistic Models

Recently, Denoising Diffusion Probabilistic Models have been widely used in image segmentation, by generating segmentation masks conditioned on the input image. However, previous works can not seamlessly integrate existing end-to-end models with denoising diffusion models. Existing research can only select acceleration steps based on experience rather than calculating them specifically. Moreover, most methods are limited to small models and small-scale datasets, unable to generalize to general datasets and a wider range of tasks. Therefore, we propose Resfusion with a novel resnoise-diffusion process, which gradually generates segmentation masks or any type of target image, seamlessly integrating state-of-the-art end-to-end models and denoising diffusion models. Resfusion bridges the discrepancy between the likelihood output and the ground truth output through a Markov process. Through the novel smooth equivalence transformation in resnoise-diffusion process, we determine the optimal acceleration step. Experimental results demonstrate that Resfusion combines the capabilities of existing end-to-end models and denoising diffusion models, further enhancing performance and achieving outstanding results. Moreover, Resfusion is not limited to segmentation tasks, it can easily generalize to any general tasks of image generation and exhibit strong competitiveness

LLM-QAT: Data-Free Quantization Aware Training for Large Language Models

Several post-training quantization methods have been applied to large language models (LLMs), and have been shown to perform well down to 8-bits. We find that these methods break down at lower bit precision, and investigate quantization aware training for LLMs (LLM-QAT) to push quantization levels even further. We propose a data-free distillation method that leverages generations produced by the pre-trained model, which better preserves the original output distribution and allows quantizing any generative model independent of its training data, similar to post-training quantization methods. In addition to quantizing weights and activations, we also quantize the KV cache, which is critical for increasing throughput and support long sequence dependencies at current model sizes. We experiment with LLaMA models of sizes 7B, 13B, and 30B, at quantization levels down to 4-bits. We observe large improvements over training-free methods, especially in the low-bit settings

A proteomic view of Caenorhabditis elegans caused by short-term hypoxic stress

<p>Abstract</p> <p>Background</p> <p>The nematode <it>Caenorhabditis elegans </it>is both sensitive and tolerant to hypoxic stress, particularly when the evolutionarily conserved hypoxia response pathway HIF-1/EGL-9/VHL is involved. Hypoxia-induced changes in the expression of a number of genes have been analyzed using whole genome microarrays in <it>C. elegans</it>, but the changes at the protein level in response to hypoxic stress still remain unclear.</p> <p>Results</p> <p>Here, we utilized a quantitative proteomic approach to evaluate changes in the expression patterns of proteins during the early response to hypoxia in <it>C. elegans</it>. Two-dimensional difference gel electrophoresis (2D-DIGE) was used to compare the proteomic maps of wild type <it>C. elegans </it>strain N2 under a 4-h hypoxia treatment (0.2% oxygen) and under normoxia (control). A subsequent analysis by MALDI-TOF-TOF-MS revealed nineteen protein spots that were differentially expressed. Nine of the protein spots were significantly upregulated, and ten were downregulated upon hypoxic stress. Three of the upregulated proteins were involved in cytoskeletal function (LEV-11, MLC-1, ACT-4), while another three upregulated (ATP-2, ATP-5, VHA-8) were ATP synthases functionally related to energy metabolism. Four ribosomal proteins (RPL-7, RPL-8, RPL-21, RPS-8) were downregulated, indicating a decrease in the level of protein translation upon hypoxic stress. The overexpression of tropomyosin (LEV-11) was further validated by Western blot. In addition, the mutant strain of <it>lev-11(x12</it>) also showed a hypoxia-sensitive phenotype in subsequent analyses, confirming the proteomic findings.</p> <p>Conclusions</p> <p>Taken together, our data suggest that altered protein expression, structural protein remodeling, and the reduction of translation might play important roles in the early response to oxygen deprivation in <it>C. elegans</it>, and this information will help broaden our knowledge on the mechanism of hypoxia response.</p