SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

Large language models (LLMs) show excellent performance but are compute- and memory-intensive. Quantization can reduce memory and accelerate inference. However, for LLMs beyond 100 billion parameters, existing methods cannot maintain accuracy or do not run efficiently on hardware. We propose SmoothQuant, a training-free, accuracy-preserving, and general-purpose post-training quantization (PTQ) solution to enable 8-bit weight, 8-bit activation (W8A8) quantization for LLMs that can be implemented efficiently. We observe that systematic outliers appear at fixed activation channels. Based on the fact that weights are easy to quantize while activations are not, SmoothQuant smooths the activation outliers by offline migrating the quantization difficulty from activations to weights with a mathematically equivalent transformation. SmoothQuant enables an INT8 quantization of both weights and activations for all the GEMMs in LLMs, including OPT-175B, BLOOM-176B, and GLM-130B. SmoothQuant has better hardware efficiency than existing techniques using mixed-precision activation quantization or weight-only quantization. We demonstrate up to 1.56x speedup and 2x memory reduction for LLMs with negligible loss in accuracy. Thanks to the hardware-friendly design, we integrate SmoothQuant into FasterTransformer, a state-of-the-art LLM serving framework, and achieve faster inference speed with half the number of GPUs compared to FP16. Our work offers a turn-key solution that reduces hardware costs and democratizes LLMs. Code is available at: https://github.com/mit-han-lab/smoothquant.Comment: The first two authors contributed equally to this wor

Moderate increase of precipitation stimulates CO2 production by regulating soil organic carbon in a saltmarsh

Saltmarsh is widely recognized as a blue carbon ecosystem with great carbon storage potential. Yet soil respiration with a major contributor of atmospheric CO2 can offset its carbon sink function. Up to date, mechanisms ruling CO2 emissions from saltmarsh soil remain unclear. In particular, the effect of precipitation on soil CO2 emissions is unclear in coastal wetlands, due the lack of outdoor data in real situations. We conducted a 7-year field manipulation experiment in a saltmarsh in the Yellow River Delta, China. Soil respiration in five treatments (−60%, −40%, +0%, +40%, and + 60% of precipitation) was measured in the field. Topsoils from the last 3 years (2019–2021) were analyzed for CO2 production potential by microcosm experiments. Furthermore, quality and quantity of soil organic carbon and microbial function were tested. Results show that only the moderate precipitation rise of +40% induced a 66.2% increase of CO2 production potential for the microcosm experiments, whereas other data showed a weak impact. Consistently, soil respiration was also found to be strongest at +40%. The CO2 production potential is positively correlated with soil organic carbon, including carbon quantity and quality. But microbial diversity did not show any positive response to precipitation sizes. r-/K-strategy seemed to be a plausible explanation for biological factors. Overall, our finding reveal that a moderate precipitation increase, not decrease or a robust increase, in a saltmarsh is likely to improve soil organic carbon quality and quantity, and bacterial oligotroph:copiotroph ratio, ultimately leading to an enhanced CO2 production

Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks

Pre-trained models (PTMs) have been widely used in various downstream tasks. The parameters of PTMs are distributed on the Internet and may suffer backdoor attacks. In this work, we demonstrate the universal vulnerability of PTMs, where fine-tuned PTMs can be easily controlled by backdoor attacks in arbitrary downstream tasks. Specifically, attackers can add a simple pre-training task, which restricts the output representations of trigger instances to pre-defined vectors, namely neuron-level backdoor attack (NeuBA). If the backdoor functionality is not eliminated during fine-tuning, the triggers can make the fine-tuned model predict fixed labels by pre-defined vectors. In the experiments of both natural language processing (NLP) and computer vision (CV), we show that NeuBA absolutely controls the predictions for trigger instances without any knowledge of downstream tasks. Finally, we apply several defense methods to NeuBA and find that model pruning is a promising direction to resist NeuBA by excluding backdoored neurons. Our findings sound a red alarm for the wide use of PTMs. Our source code and models are available at \url{https://github.com/thunlp/NeuBA}

Experimental and Numerical Study on the Permeation Grouting Diffusion Mechanism considering Filtration Effects

Permeation grouting estimation is important for the design of grouting engineering. Filtration effects and rheological behavior play a key role in permeation grouting diffusion of cement-based grouts. To better understand the effect of filtration and grout rheological behavior on the grouting diffusion mechanism, one-dimensional permeation grout injections in sand columns under constant flow rate were performed by a self-developed experimental procedure. Experimental results showed that there were dramatic variations in rheological parameters and porosity along the diffusion distance. However, the rheological parameters changed slightly with time for each position. Based on the experimental results, a numerical model considering the filtration effect and grout rheological behavior was established to describe the mechanism of grout flow in porous media. In addition, numerical solutions from the proposed model are compared with the experimental results. The comparative results showed that the proposed numerical method can match the laboratory tests well. Finally, the effects of the grout flow velocity and the water/cement ratio of the grout on the diffusion mechanism are also discussed

Data_Sheet_1_Moderate increase of precipitation stimulates CO2 production by regulating soil organic carbon in a saltmarsh.docx

Saltmarsh is widely recognized as a blue carbon ecosystem with great carbon storage potential. Yet soil respiration with a major contributor of atmospheric CO2 can offset its carbon sink function. Up to date, mechanisms ruling CO2 emissions from saltmarsh soil remain unclear. In particular, the effect of precipitation on soil CO2 emissions is unclear in coastal wetlands, due the lack of outdoor data in real situations. We conducted a 7-year field manipulation experiment in a saltmarsh in the Yellow River Delta, China. Soil respiration in five treatments (−60%, −40%, +0%, +40%, and + 60% of precipitation) was measured in the field. Topsoils from the last 3 years (2019–2021) were analyzed for CO2 production potential by microcosm experiments. Furthermore, quality and quantity of soil organic carbon and microbial function were tested. Results show that only the moderate precipitation rise of +40% induced a 66.2% increase of CO2 production potential for the microcosm experiments, whereas other data showed a weak impact. Consistently, soil respiration was also found to be strongest at +40%. The CO2 production potential is positively correlated with soil organic carbon, including carbon quantity and quality. But microbial diversity did not show any positive response to precipitation sizes. r-/K-strategy seemed to be a plausible explanation for biological factors. Overall, our finding reveal that a moderate precipitation increase, not decrease or a robust increase, in a saltmarsh is likely to improve soil organic carbon quality and quantity, and bacterial oligotroph:copiotroph ratio, ultimately leading to an enhanced CO2 production.</p

Inundation depth stimulates plant-mediated CH4 emissions by increasing ecosystem carbon uptake and plant height in an estuarine wetland

Plant-mediated CH4 emission is an important part of the ecosystem CH4 emission from vegetated wetlands. Inundation depth may alter the potential magnitude of CH4 releases by changing CH4 production and plant transport, but the relationships between plant-mediated CH4 emissions and inundation depth are still uncertain, especially for estuarine wetlands with changeable hydrological processes. Besides, there are conflicting results regarding the role of inundation depth in plant-mediated CH4 emissions. Here we conducted a novel inundation depth experiment (0, 5, 10, 20, 30 and 40 cm inundation depth) dominated by Phragmites australis in the Yellow River estuary, China. Soil CH4 emissions, ecosystem CH4 emissions, net ecosystem CO2 exchange (NEE), soil organic carbon (SOC) and plant traits were measured during the growing seasons of 2018, 2019 and 2020. Plant-mediated CH4 emissions were the difference between ecosystem CH4 emissions and soil CH4 emissions. The results showed that inundation depth decreased soil CH4 emissions but increased ecosystem CH4 emissions. Plant-mediated CH4 transport from Phragmites australis accounted for 99% of total ecosystem CH4 emissions under different inundation depths. Inundation depth strongly stimulated plant-mediated CH4 emission from 0 to 20 cm during the growing seasons. The increased NEE enhanced plant-mediated CH4 emissions by altering production, suggesting that carbon components derived from photosynthetic carbon input may benefit CH4 production. Additionally, the increased plant height promoted CH4 emission by regulating plant transport, indicating that plant traits may play an important role in transport of CH4. Our findings indicated that NEE and plant height play an important role in plant-mediated CH4 emissions under different inundation depths in estuarine wetland. This study also highlights that hydrological regimes and plant traits are essential for the estimation of CH4 emissions in future projections of global wetland changes. Read the free Plain Language Summary for this article on the Journal blog

Stimulation of long-term ammonium nitrogen deposition on methanogenesis by Methanocellaceae in a coastal wetland

Atmospheric nitrogen deposition caused by human activities has been receiving much attention. Here, after long-term simulated ammonium and nitrate nitrogen deposition (NH4Cl,KNO3, and NH4NO3) in the Yellow River Delta (YRD), a sensitive coastal wetland ecosystem typified by a distinct wet and dry season, methane fluxes were measured, by adopting a closed static chamber technique. The results showed that deposition of ammonium nitrogen accelerated methane emissions all year round. Ammonium nitrogen deposition transformed the YRD from a methane sink into a source during the dry season. Methanocellaceae is the only methanogen with increased abundance after the application of NH4Cl and NH4NO3, which promoted methane emissions, during the wet season. The findings suggested that Methanocellaceae may facilitate methane emissions in response to increased ammonium nitrogen deposition. Other methanogens might have profited from ammonium supplementation, such as Methanosarcinaceae. Deposition of nitrate nitrogen did not affect methane flux significantly. To the best of our knowledge, this study is the first to show that Methanocellaceae may be responsible for methane production in coastal wetland system. This study highlights the significant effect of ammonium nitrogen and slight effect of nitrate nitrogen on methane emission in the YRD and it will be helpful to understand the microbial mechanism responding to increased nitrogen deposition in the sensitive coastal wetland ecosystem. (C) 2017 Elsevier B.V. All rights reserved