E-Sparse: Boosting the Large Language Model Inference through Entropy-based N:M Sparsity

Traditional pruning methods are known to be challenging to work in Large Language Models (LLMs) for Generative AI because of their unaffordable training process and large computational demands. For the first time, we introduce the information entropy of hidden state features into a pruning metric design, namely E-Sparse, to improve the accuracy of N:M sparsity on LLM. E-Sparse employs the information richness to leverage the channel importance, and further incorporates several novel techniques to put it into effect: (1) it introduces information entropy to enhance the significance of parameter weights and input feature norms as a novel pruning metric, and performs N:M sparsity without modifying the remaining weights. (2) it designs global naive shuffle and local block shuffle to quickly optimize the information distribution and adequately cope with the impact of N:M sparsity on LLMs' accuracy. E-Sparse is implemented as a Sparse-GEMM on FasterTransformer and runs on NVIDIA Ampere GPUs. Extensive experiments on the LLaMA family and OPT models show that E-Sparse can significantly speed up the model inference over the dense model (up to 1.53X) and obtain significant memory saving (up to 43.52%), with acceptable accuracy loss

Comparative transcripts profiling reveals new insight into molecular processes regulating lycopene accumulation in a sweet orange (Citrus sinensis) red-flesh mutant

<p>Abstract</p> <p>Background</p> <p>Interest in lycopene metabolism and regulation is growing rapidly because accumulative studies have suggested an important role for lycopene in human health promotion. However, little is known about the molecular processes regulating lycopene accumulation in fruits other than tomato so far.</p> <p>Results</p> <p>On a spontaneous sweet orange bud mutant with abnormal lycopene accumulation in fruits and its wild type, comparative transcripts profiling was performed using Massively Parallel Signature Sequencing (MPSS). A total of 6,877,027 and 6,275,309 reliable signatures were obtained for the wild type (WT) and the mutant (MT), respectively. Interpretation of the MPSS signatures revealed that the total number of transcribed gene in MT is 18,106, larger than that in WT 17,670, suggesting that newly initiated transcription occurs in the MT. Further comparison of the transcripts abundance between MT and WT revealed that 3,738 genes show more than two fold expression difference, and 582 genes are up- or down-regulated at 0.05% significance level by more than three fold difference. Functional assignments of the differentially expressed genes indicated that 26 reliable metabolic pathways are altered in the mutant; the most noticeable ones are carotenoid biosynthesis, photosynthesis, and citrate cycle. These data suggest that enhanced photosynthesis and partial impairment of lycopene downstream flux are critical for the formation of lycopene accumulation trait in the mutant.</p> <p>Conclusion</p> <p>This study provided a global picture of the gene expression changes in a sweet orange red-flesh mutant as compared to the wild type. Interpretation of the differentially expressed genes revealed new insight into the molecular processes regulating lycopene accumulation in the sweet orange red-flesh mutant.</p

Karst collapse risk zonation and evaluation in Wuhan, China based on analytic hierarchy process, logistic regression, and insar angular distortion approaches

The current study presents a detailed assessment of risk zones related to karst collapse in Wuhan by analytical hierarchy process (AHP) and logistic regression (LR) models. The results showed that the LR model was more accurate with an area under the receiver operating characteristic (ROC) curve of 0.911 compared to 0.812 derived from the AHP model. Both models performed well in identifying high-risk zones with only a 3% discrepancy in area. However, for the medium-and low-risk classes, although the spatial distribution of risk zoning results were similar between two approaches, the spatial extent of the risk areas varied between final models. The reliability of both methods were reduced significantly by excluding the InSAR-based ground subsidence map from the analysis, with the karst collapse presence falling into the high-risk zone being reduced by approximately 14%, and karst collapse absence falling into the karst area being increased by approximately 6.5% on the training samples. To evaluate the practicality of using only results from ground subsidence maps for the risk zonation, the results of AHP and LR are compared with a weighted angular distortion (WAD) method for karst risk zoning in Wuhan. We find that the areas with relatively large subsidence horizontal gradient values within the karst belts are generally spatially consistent with high-risk class areas identified by the AHP-and LR-based approaches. However, the WAD-based approach cannot be used alone as an ideal karst collapse risk assessment model as it does not include geological and natural factors into the risk zonation. © 2021 by the authors. Licensee MDPI, Basel, Switzerland

Changes in and the association of retinal blood perfusion and retinal nerves in diabetic patients without retinopathy

ObjectiveTo explore intraretinal blood flow perfusion and nerve changes, as well as the correlation between them, in diabetic patients without diabetic retinopathy (NDR).MethodEighty-six NDR patients (86 eyes) who attended the ophthalmology clinic between December 2019 and December 2021 were included. Sixty-four eyes of 64 healthy examined controls in the same period were selected as the control group. The patients underwent routine ophthalmological examination, optical coherence tomography (OCT) and OCT angiography.ResultsThe average thickness, minimum thickness and thickness of each quadrant except for the superior temporal quadrant of the ganglion cell-inner plexiform layer (GCIPL) in the macular area of the affected eyes in the NDR group were lower than that of the tested eyes in the control group (P &lt; 0.05). The average retinal nerve fibre layer (RNFL) thickness of the NDR group and the superior, inferior and nasal quadrants around the optic disc of the affected eyes in the NDR group were lower compared with the tested eyes in the control group (P &lt; 0.001, P = 0.003, P = 0.001, P = 0.009). The mean vessel length density in the parafoveal and perifoveal areas in the NDR group was positively associated with the mean GCIPL thickness in the macular area (ρ = 0.265, ρ = 0.257 and P &lt; 0.001). No blood flow perfusion parameters in the NDR group were correlated with the RNFL thickness of the corresponding quadrant around the optic disc (P &gt; 0.05).ConclusionIn diabetic patients without diabetic retinopathy, the superficial retinal vessel density in the macular area positively correlated with GCIPL thickness, and the superficial retinal vessel density around the optic disc was not correlated with RNFL thickness

Electronic properties of monolayer copper selenide with one-dimensional moir\'e patterns

Strain engineering is a vital way to manipulate the electronic properties of two-dimensional (2D) materials. As a typical representative of transition metal mono-chalcogenides (TMMs), a honeycomb CuSe monolayer features with one-dimensional (1D) moir\'e patterns owing to the uniaxial strain along one of three equivalent orientations of Cu(111) substrates. Here, by combining low-temperature scanning tunneling microscopy/spectroscopy (STM/S) experiments and density functional theory (DFT) calculations, we systematically investigate the electronic properties of the strained CuSe monolayer on the Cu(111) substrate. Our results show the semiconducting feature of CuSe monolayer with a band gap of 1.28 eV and the 1D periodical modulation of electronic properties by the 1D moir\'e patterns. Except for the uniaxially strained CuSe monolayer, we observed domain boundary and line defects in the CuSe monolayer, where the biaxial-strain and strain-free conditions can be investigated respectively. STS measurements for the three different strain regions show that the first peak in conduction band will move downward with the increasing strain. DFT calculations based on the three CuSe atomic models with different strain inside reproduced the peak movement. The present findings not only enrich the fundamental comprehension toward the influence of strain on electronic properties at 2D limit, but also offer the benchmark for the development of 2D semiconductor materials.Comment: 14 pages, 12 figures, 25 referenc

A stochastic dynamic local search method for learning Multiple-Valued Logic networks

金沢大学理工研究域電子情報学系In this paper, we propose a stochastic dynamic local search (SDLS) method for Multiple-Valued Logic (MVL) learning by introducing stochastic dynamics into the traditional local search method. The proposed learning network maintains some trends of quick descent to either global minimum or a local minimum, and at the same time has some chance of escaping from local minima by permitting temporary error increases during learning. Thus the network may eventually reach the global minimum state or its best approximation with very high probability. Simulation results show that the proposed algorithm has the superior abilities to find the global minimum for the MVL network learning within reasonable number of iterations. Copyright © 2007 The Institute of Electronics, Information and Communication Engineers

Beyond the Obvious: Evaluating the Reasoning Ability In Real-life Scenarios of Language Models on Life Scapes Reasoning Benchmark~(LSR-Benchmark)

This paper introduces the Life Scapes Reasoning Benchmark (LSR-Benchmark), a novel dataset targeting real-life scenario reasoning, aiming to close the gap in artificial neural networks' ability to reason in everyday contexts. In contrast to domain knowledge reasoning datasets, LSR-Benchmark comprises free-text formatted questions with rich information on real-life scenarios, human behaviors, and character roles. The dataset consists of 2,162 questions collected from open-source online sources and is manually annotated to improve its quality. Experiments are conducted using state-of-the-art language models, such as gpt3.5-turbo and instruction fine-tuned llama models, to test the performance in LSR-Benchmark. The results reveal that humans outperform these models significantly, indicating a persisting challenge for machine learning models in comprehending daily human life