Model Compression and Efficient Inference for Large Language Models: A Survey

Transformer based large language models have achieved tremendous success. However, the significant memory and computational costs incurred during the inference process make it challenging to deploy large models on resource-constrained devices. In this paper, we investigate compression and efficient inference methods for large language models from an algorithmic perspective. Regarding taxonomy, similar to smaller models, compression and acceleration algorithms for large language models can still be categorized into quantization, pruning, distillation, compact architecture design, dynamic networks. However, Large language models have two prominent characteristics compared to smaller models: (1) Most of compression algorithms require finetuning or even retraining the model after compression. The most notable aspect of large models is the very high cost associated with model finetuning or training. Therefore, many algorithms for large models, such as quantization and pruning, start to explore tuning-free algorithms. (2) Large models emphasize versatility and generalization rather than performance on a single task. Hence, many algorithms, such as knowledge distillation, focus on how to preserving their versatility and generalization after compression. Since these two characteristics were not very pronounced in early large models, we further distinguish large language models into medium models and ``real'' large models. Additionally, we also provide an introduction to some mature frameworks for efficient inference of large models, which can support basic compression or acceleration algorithms, greatly facilitating model deployment for users.Comment: 47 pages, review 380 papers. The work is ongoin

Vision guided cutting and mechanical handling of lace ribbon

EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Lability of Traditional Chinese Medicine by Intestinal Bacterial Metabolism Results in P450 Cycle-mediated Drug-Drug Interaction

Increasing attention is being paid on how to assess the potential risk for TCM -prescription drug interactions. As many TCMs are capable of biotransformation by intestinal bacteria in the gastrointestinal tract, attention to biotransformation of TCM in the gastrointestinal tract may lead to discovery of novel active components and active mechanisms. In the present study, we proposed that the biotransformation of TCM in human gastrointestinal tract should be integrated into the early evaluation of drug metabolic properties. Ginsenosides were selected as model chemicals since ginseng products are orally administered. There is still no consensus about the in vivo influence of ginseng on cytochrome P450 isoforms (P450s). The majority of naturally occurring ginsenosides (NOGs) is not capable of reaching liver because the absorption of NOGs from the intestines is very poor. However, the intestinal bacteria metabolites, Rh1 and F1, and Compound K (C-K), protopanaxadiol (Ppd), and protopanaxatriol (Ppt) from NOGs may reach the systemic circulation going through liver after oral administration of ginseng extract. We found that in the in vitro experiments, the NOGs exhibited no inhibition or weak inhibition against human P450s activities. Their degradative products in human gastrointestinal tract demonstrate a wide range of direct inhibition of P450-mediated metabolism. The inhibition is reversible and not mechanism-based one. Integrating the in vivo results of their plasma concentrations, we proposed that NOGs might result in ginsenoside-drug interactions via their intestinal bacteria metabolites (C-K, Ppd and Ppt) by influencing CYPs activities. The indirect drug interactions via biotransformation of intestinal bacteria might be a new pathway that needs to be paid additional attention to

Effects of changes in land use structure on nitrogen input in the Pingzhai Reservoir watershed, a karst mountain region

Optimizing land use composition to control nitrogen input into water bodies is one way to address surface source pollution in karst mountain regions. In this study, changes in land use, N sources, and spatial and temporal changes of N migration in the Pingzhai Reservoir watershed were evaluated from 2015 to 2021, and the relationship between land use composition and N input was elucidated. N was the main pollution in the water of the watershed; NO3− was the dominant form of N, and it did not react during migration. N came from soil, livestock manure or domestic sewage, and atmospheric deposition. Isolating the fractionation effects of source nitrogen is crucial to improve the accuracy of nitrogen and oxygen isotope traceability in the Pingzhai Reservoir. From 2015 to 2021, the grassland area in the Pingzhai Reservoir increased by 5.52%, the woodland area increased by 2.01%, the water area increased by 1.44%, the cropland decreased by 5.8%, unused land decreased by 3.18%, and construction land remained unchanged. Policies and reservoir construction were the main drivers of changes in land-use type in the catchment. Changes in land use structure affected nitrogen input patterns, with unused land having a highly significant positive correlation with inputs of NH3–N, NO2−, and TN, and construction land having a significant positive correlation with the input of NO2−. The inhibitory effect of forest and grassland on nitrogen input in the basin was offset by the promoting effect of cropland and construction land on nitrogen input, with unused land becoming a new focus area for nitrogen emissions due to a lack of environmental management. Modifying the area of different land use types in the watershed can effectively control nitrogen input to the watershed

Supplementary document for Mesoporous black TiO₂ hollow shells with controlled cavity size for enhanced visible light photocatalysis - 6623698.pdf

Complementary characterizatio