Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling

Quantization of transformer language models faces significant challenges due to the existence of detrimental outliers in activations. We observe that these outliers are asymmetric and concentrated in specific channels. To address this issue, we propose the Outlier Suppression+ framework. First, we introduce channel-wise shifting and scaling operations to eliminate asymmetric presentation and scale down problematic channels. We demonstrate that these operations can be seamlessly migrated into subsequent modules while maintaining equivalence. Second, we quantitatively analyze the optimal values for shifting and scaling, taking into account both the asymmetric property and quantization errors of weights in the next layer. Our lightweight framework can incur minimal performance degradation under static and standard post-training quantization settings. Comprehensive results across various tasks and models reveal that our approach achieves near-floating-point performance on both small models, such as BERT, and large language models (LLMs) including OPTs, BLOOM, and BLOOMZ at 8-bit and 6-bit settings. Furthermore, we establish a new state of the art for 4-bit BERT

State-of-the-Art in Carbides/Carbon Composites for Electromagnetic Wave Absorption

Electromagnetic wave absorbing materials (EWAMs) have made great progress in the past decades, and are playing an increasingly important role in radiation prevention and antiradar detection due to their essential attenuation toward incident EM wave. With the flourish of nanotechnology, the design of high-performance EWAMs is not just dependent on the intrinsic characteristics of single-component medium, but pays more attention to the synergistic effects from different components to generate rich loss mechanisms. Among various candidates, carbides and carbon materials are usually labeled with the features of chemical stability, low density, tunable dielectric property, and diversified morphology/microstructure, and thus the combination of carbides and carbon materials will be a promising way to acquire new EWAMs with good practical application prospects. In this review, we introduce EM loss mechanisms related to dielectric composites, and then highlight the state-of-the-art progress in carbides/carbon composites as high-performance EWAMs, including silicon carbide/carbon, MXene/carbon, molybdenum carbide/carbon, as well as some uncommon carbides/carbon composites and multicomponent composites. The critical information regarding composition optimization, structural engineering, performance reinforcement, and structure-function relationship are discussed in detail. In addition, some challenges and perspectives for the development of carbides/carbon composites are also proposed after comparing the performance of some representative composites

Exploring the Potential of Flexible 8-bit Format: Design and Algorithm

Neural network quantization is widely used to reduce model inference complexity in real-world deployments. However, traditional integer quantization suffers from accuracy degradation when adapting to various dynamic ranges. Recent research has focused on a new 8-bit format, FP8, with hardware support for both training and inference of neural networks but lacks guidance for hardware design. In this paper, we analyze the benefits of using FP8 quantization and provide a comprehensive comparison of FP8 with INT quantization. Then we propose a flexible mixed-precision quantization framework that supports various number systems, enabling optimal selection of the most appropriate quantization format for different neural network architectures. Experimental results demonstrate that our proposed framework achieves competitive performance compared to full precision on various tasks, including image classification, object detection, segmentation, and natural language understanding. Our work furnishes critical insights into the tangible benefits and feasibility of employing FP8 quantization, paving the way for heightened neural network efficiency in tangible scenarios. Our code is available in the supplementary material

Design and Performance of the Hotrod Melt-Tip Ice-Drilling System

We introduce the design and performance of a melt-tip ice-drilling system designed to insert a temperature sensor cable into ice. The melt tip is relatively simple and low cost, designed for a one-way trip to the ice-bed interface. The drilling system consists of a melt tip, umbilical cable, winch, interface, power supply, and support items. The melt tip and the winch are the most novel elements of the drilling system, and we make the hardware and electrical designs of these components available open access. Tests conducted in a laboratory ice well indicate that the melt tip has an electrical energy to forward melting heat transfer efficiency of ~35 % with a theoretical maximum penetration rate of ~12 m/hr at maximum 6.0 kW power. In contrast, ice-sheet testing suggests the melt tip has an analogous heat transfer efficiency of ~15 % with a theoretical maximum penetration rate of ~6 m/hr. We expect the efficiency gap between laboratory and field performance to decrease with increasing operator experience. Umbilical freeze-in due to borehole refreezing is the primary depth-limiting factor of the drilling system. Enthalpy-based borehole refreezing assessments predict refreezing below critical umbilical diameter in ~4 hours at -20 ˚C ice temperatures and ~20 hours at -2 ˚C. This corresponds to a theoretical depth limit of up to ~200 m, depending on firn thickness, ice temperature and operator experience.</p

Inside-out Ca2+ signalling prompted by STIM1 conformational switch

Store-operated Ca(2+) entry mediated by STIM1 and ORAI1 constitutes one of the major Ca(2+) entry routes in mammalian cells. The molecular choreography of STIM1–ORAI1 coupling is initiated by endoplasmic reticulum (ER) Ca(2+) store depletion with subsequent oligomerization of the STIM1 ER-luminal domain, followed by its redistribution towards the plasma membrane to gate ORAI1 channels. The mechanistic underpinnings of this inside-out Ca(2+) signalling were largely undefined. By taking advantage of a unique gain-of-function mutation within the STIM1 transmembrane domain (STIM1-TM), here we show that local rearrangement, rather than alteration in the oligomeric state of STIM1-TM, prompts conformational changes in the cytosolic juxtamembrane coiled-coil region. Importantly, we further identify critical residues within the cytoplasmic domain of STIM1 (STIM1-CT) that entail autoinhibition. On the basis of these findings, we propose a model in which STIM1-TM reorganization switches STIM1-CT into an extended conformation, thereby projecting the ORAI-activating domain to gate ORAI1 channels

How Does Agricultural Green Transformation Improve Residents’ Health? Empirical Evidence from China

Promoting green and sustainable agriculture is of great significance for ensuring food security and addressing global challenges. Meanwhile, health has increasingly become a global concern. Nutrition and health are the purpose of agricultural production. As two major global issues, how agriculture empowers human health has long been discussed. Based on the provincial panel data of China from 2003 to 2021, this paper studies the impact of agricultural green transformation (AGT) on residents’ health and explores its impact mechanism. The empirical results show that: (1) AGT in China has significantly reduced the average mortality rate and maternal mortality rate and significantly increased the average life expectancy, indicating that AGT in China has significantly improved the health level. The conclusions of robustness testing methods such as replacing AGT indicators and controlling endogeneity are still valid; and (2) The impact of AGT on residents’ health depends on the regional economic level, and there is a threshold effect. Compared with low-income areas, the positive effects of AGT in high-income areas on residents’ health are more pronounced; and (3) Agricultural carbon emissions play an intermediary effect between AGT and residents’ health, and AGT can improve residents’ health by reducing carbon emissions. The level of local education development plays a moderating role in the relationship between AGT and residents’ health. Agricultural policy implications include enhancing the ability to protect and utilize agricultural resources, promoting the green and low-carbon transformation of agriculture, and taking a more scientific and systematic approach towards the complex diversity of health risk factors

The Polarization of the Coupling Strength of Interdependent Networks Stimulates Cooperation

We introduce a mixed network coupling mechanism and study its effects on how cooperation evolves in interdependent networks. This mechanism allows some players (conservative-driven) to establish a fixed-strength coupling, while other players (radical-driven) adjust their coupling strength through the evolution of strategy. By means of numerical simulation, a hump-like relationship between the level of cooperation and conservative participant density is revealed. Interestingly, interspecies interactions stimulate polarization of the coupling strength of radical-driven players, promoting cooperation between two types of players. We thus demonstrate that a simple mixed network coupling mechanism substantially expands the scope of cooperation among structured populations

Review of operations for multi-energy coupled virtual power plants participating in electricity market

The application projects of traditional virtual power plants generally have some significant problems, such as single energy structure, insufficient participation in the market, sparse energy coupling relationship and lack of new loads, which lead to poor operation stability, low economic benefits and difficult risk management of traditional virtual power plants. With the development of distributed energy resources such as electric vehicles, air conditioners, and electric heating loads and the advancement of information and communication technologies, virtual power plants have gradually evolved from aggregating a single type of energy resource to having the ability to meet the aggregation and management of multiple types of energy resources. Virtual power plant technology provides an effective means to aggregate distributed energy and user-side resources to participate in energy market transactions. A virtual power plant with multiple energy sources such as electricity, heat and gas coupling can effectively avoid the lack of system stability and security caused by a single resource structure, and effectively meet the needs of users for multiple types of energy. This paper focuses on multi-energy coupled virtual power plants, summarizes the current status of research on multi-energy coupled virtual power plants participating in power market operation from the aspects of multi-energy coupling mechanism and operation optimization in power market, and finally provides an outlook on the research direction of multi-energy coupled virtual power plants