AWEQ: Post-Training Quantization with Activation-Weight Equalization for Large Language Models

Large language models(LLMs) exhibit excellent performance across a variety of tasks, but they come with significant computational and storage costs. Quantizing these models is an effective way to alleviate this issue. However, existing methods struggle to strike a balance between model accuracy and hardware efficiency. This is where we introduce AWEQ, a post-training method that requires no additional training overhead. AWEQ excels in both ultra-low-bit quantization and 8-bit weight and activation (W8A8) quantization. There is an observation that weight quantization is less challenging than activation quantization. AWEQ transfers the difficulty of activation quantization to weights using channel equalization, achieving a balance between the quantization difficulties of both, and thereby maximizing performance. We have further refined the equalization method to mitigate quantization bias error, ensuring the robustness of the model. Extensive experiments on popular models such as LLaMA and OPT demonstrate that AWEQ outperforms all existing post-training quantization methods for large models

The resilience of interdependent transportation networks under targeted attack

Modern world builds on the resilience of interdependent infrastructures characterized as complex networks. Recently, a framework for analysis of interdependent networks has been developed to explain the mechanism of resilience in interdependent networks. Here we extend this interdependent network model by considering flows in the networks and study the system's resilience under different attack strategies. In our model, nodes may fail due to either overload or loss of interdependency. Under the interaction between these two failure mechanisms, it is shown that interdependent scale-free networks show extreme vulnerability. The resilience of interdependent SF networks is found in our simulation much smaller than single SF network or interdependent SF networks without flows.Comment: 5 pages, 4 figure

Meta-analysis of interleukin 6, 8, and 10 between off-pump and on-pump coronary artery bypass groups

This study aimed to evaluate the role of off-pump coronary artery bypass (CAB) surgery on the decrease of postoperative inflammatory responses in patients. We systematically searched databases of PubMed and Embase to select the related studies. Interleukin (IL) 6, 8, and 10 were used as outcomes and pooled analysis was performed using R 3.12 software. Standardized mean differences (SMDs) and their 95% confidence intervals (95% CIs) were considered as effect estimates. A total of 27 studies, including 1340 participants, were recruited in this meta-analysis. The pooled analyses showed that postoperative concentration of IL-10 at 12 hours was significantly lower in off-pump CAB group compared to on-pump CAB group (SMD = −1.3640, 95% CI = −2.0086-−0.7193). However, no significant differences were found in pre and postoperative concentrations of IL-6 and 8 between off-pump and on-pump CAB groups. These results suggest that there is no advantage of off-pump CAB surgery in the reduction of inflammation compared to on-pump CAB surgery

Seismic Data Interpolation based on Denoising Diffusion Implicit Models with Resampling

The incompleteness of the seismic data caused by missing traces along the spatial extension is a common issue in seismic acquisition due to the existence of obstacles and economic constraints, which severely impairs the imaging quality of subsurface geological structures. Recently, deep learning-based seismic interpolation methods have attained promising progress, while achieving stable training of generative adversarial networks is not easy, and performance degradation is usually notable if the missing patterns in the testing and training do not match. In this paper, we propose a novel seismic denoising diffusion implicit model with resampling. The model training is established on the denoising diffusion probabilistic model, where U-Net is equipped with the multi-head self-attention to match the noise in each step. The cosine noise schedule, serving as the global noise configuration, promotes the high utilization of known trace information by accelerating the passage of the excessive noise stages. The model inference utilizes the denoising diffusion implicit model, conditioning on the known traces, to enable high-quality interpolation with fewer diffusion steps. To enhance the coherency between the known traces and the missing traces within each reverse step, the inference process integrates a resampling strategy to achieve an information recap on the former interpolated traces. Extensive experiments conducted on synthetic and field seismic data validate the superiority of our model and its robustness on various missing patterns. In addition, uncertainty quantification and ablation studies are also investigated.Comment: 14 pages, 13 figure

3D Flower-Like Hierarchitectures Constructed by SnS/SnS 2

Sn chalcogenides, including SnS, Sn2S3, and SnS2, have been extensively studied as anode materials for lithium batteries. In order to obtain one kind of high capacity, long cycle life lithium batteries anode materials, three-dimensional (3D) flower-like hierarchitectures constructed by SnS/SnS2 heterostructure nanosheets with thickness of ~20 nm have been synthesized via a simple one-pot solvothermal method. The obtained samples exhibit excellent electrochemical performance as anode for Li-ion batteries (LIBs), which deliver a first discharge capacity of 1277 mAhg−1 and remain a reversible capacity up to 500 mAhg−1 after 50 cycles at a current of 100 mAg−1

Drude Conductivity of Dirac Fermions in Graphene

Electrons moving in graphene behave as massless Dirac fermions, and they exhibit fascinating low-frequency electrical transport phenomena. Their dynamic response, however, is little known at frequencies above one terahertz (THz). Such knowledge is important not only for a deeper understanding of the Dirac electron quantum transport, but also for graphene applications in ultrahigh speed THz electronics and IR optoelectronics. In this paper, we report the first measurement of high-frequency conductivity of graphene from THz to mid-IR at different carrier concentrations. The conductivity exhibits Drude-like frequency dependence and increases dramatically at THz frequencies, but its absolute strength is substantially lower than theoretical predictions. This anomalous reduction of free electron oscillator strength is corroborated by corresponding changes in graphene interband transitions, as required by the sum rule. Our surprising observation indicates that many-body effects and Dirac fermion-impurity interactions beyond current transport theories are important for Dirac fermion electrical response in graphene

Electrical Control of Plasmon Resonance with Graphene

Surface plasmon, with its unique capability to concentrate light into sub-wavelength volume, has enabled great advances in photon science, ranging from nano-antenna and single-molecule Raman scattering to plasmonic waveguide and metamaterials. In many applications it is desirable to control the surface plasmon resonance in situ with electric field. Graphene, with its unique tunable optical properties, provides an ideal material to integrate with nanometallic structures for realizing such control. Here we demonstrate effective modulation of the plasmon resonance in a model system composed of hybrid graphene-gold nanorod structure. Upon electrical gating the strong optical transitions in graphene can be switched on and off, which leads to significant modulation of both the resonance frequency and quality factor of plasmon resonance in gold nanorods. Hybrid graphene-nanometallic structures, as exemplified by this combination of graphene and gold nanorod, provide a general and powerful way for electrical control of plasmon resonances. It holds promise for novel active optical devices and plasmonic circuits at the deep subwavelength scale

Coxsackievirus A6 Induces Cell Cycle Arrest in G0/G1 Phase for Viral Production

Recent epidemiological data indicate that outbreaks of hand, foot, and mouth disease (HFMD), which can be categorized according to its clinical symptoms as typical or atypical, have markedly increased worldwide. A primary causative agent for typical HFMD outbreaks, enterovirus 71 (EV71), has been shown to manipulate the cell cycle in S phase for own replication; however, it is not clear whether coxsackievirus (CVA6), the main agent for atypical HFMD, also regulates the host cell cycle. In this study, we demonstrate for the first time that CVA6 infection arrests the host cell cycle in G0/G1-phase. Furthermore, synchronization in G0/G1 phase, but not S phase or G2/M phase, promotes viral production. To investigate the mechanism of cell cycle arrest induced by CVA6 infection, we analyzed cell cycle progression after cell cycle synchronization at G0/G1 or G2/M. Our results demonstrate that CVA6 infection promotes G0/G1 phase entry from G2/M phase, and inhibits G0/G1 exit into S phase. In line with its role to arrest cells in G0/G1 phase, the expression of cyclinD1, CDK4, cyclinE1, CDK2, cyclinB1, CDK1, P53, P21, and P16 is regulated by CVA6. Finally, the non-structural proteins of CVA6, RNA-dependent RNA polymerase 3D and protease 3C , are demonstrated to be responsible for the G0/G1-phase arrest. These findings suggest that CVA6 infection arrested cell cycle in G0/G1-phase via non-structural proteins 3D and 3C, which may provide favorable environments for virus production