Exploring the Potential of Flexible 8-bit Format: Design and Algorithm

Neural network quantization is widely used to reduce model inference complexity in real-world deployments. However, traditional integer quantization suffers from accuracy degradation when adapting to various dynamic ranges. Recent research has focused on a new 8-bit format, FP8, with hardware support for both training and inference of neural networks but lacks guidance for hardware design. In this paper, we analyze the benefits of using FP8 quantization and provide a comprehensive comparison of FP8 with INT quantization. Then we propose a flexible mixed-precision quantization framework that supports various number systems, enabling optimal selection of the most appropriate quantization format for different neural network architectures. Experimental results demonstrate that our proposed framework achieves competitive performance compared to full precision on various tasks, including image classification, object detection, segmentation, and natural language understanding. Our work furnishes critical insights into the tangible benefits and feasibility of employing FP8 quantization, paving the way for heightened neural network efficiency in tangible scenarios. Our code is available in the supplementary material

L-shaped association of serum calcium with all-cause and CVD mortality in the US adults: A population-based prospective cohort study

BackgroundCalcium is involved in many biological processes, but the impact of serum calcium levels on long-term mortality in general populations has been rarely investigated.MethodsThis prospective cohort study analyzed data from the National Health and Nutrition Examination Survey (1999–2018). All-cause mortality, cardiovascular disease (CVD) mortality, and cancer mortality were obtained through linkage to the National Death Index. Survey-weighted multivariate Cox regression was performed to compute hazard ratios (HRs) and 95% confidential intervals (CIs) for the associations of calcium levels with risks of mortality. Restricted cubic spline analyses were performed to examine the non-linear association of calcium levels with all-cause and disease-specific mortality.ResultsA total of 51,042 individuals were included in the current study. During an average of 9.7 years of follow-up, 7,592 all-cause deaths were identified, including 2,391 CVD deaths and 1,641 cancer deaths. Compared with participants in the first quartile (Q1) of serum calcium level [≤2.299 mmol/L], the risk of all-cause mortality was lower for participants in the second quartile (Q2) [2.300–2.349 mmol/L], the third quartile (Q3) [2.350–2.424 mmol/L] and the fourth quartile (Q4) [≥2.425 mmol/L] with multivariable-adjusted HRs of 0.81 (95% CI, 0.74–0.88), 0.78 (95% CI, 0.71–0.86), and 0.80 (95% CI, 0.73, 0.88). Similar associations were observed for CVD mortality, with HRs of 0.82 (95% CI, 0.71–0.95), 0.87 (95% CI, 0.74–1.02), and 0.83 (95% CI, 0.72, 0.97) in Q2–Q4 quartile. Furthermore, the L-shaped non-linear associations were detected for serum calcium with the risk of all-cause mortality. Below the median of 2.350 mmol/L, per 0.1 mmol/L higher serum calcium was associated with a 24% lower risk of all-cause mortality (HR: 0.76, 95% CI, 0.70–0.83), however, no significant changes were observed when serum calcium was above the median. Similar L-shaped associations were detected for serum calcium with the risk of CVD mortality with a 25% reduction in the risk of CVD death per 0.1 mmol/L higher serum calcium below the median (HR: 0.75, 95% CI, 0.65–0.86).ConclusionL-shaped associations of serum calcium with all-cause and CVD mortality were observed in US adults, and hypocalcemia was associated with a higher risk of all-cause mortality and CVD mortality

Manipulation of Magnetization Switching by Ultrafast Spin-Polarized Hot-Electron Transport in Synthetic Antiferromagnet

Uncovering the physical mechanisms that govern ultrafast charge and spin dynamics is becoming indispensable both at the fundamental level and to develop future spin-based electronics. Recently it has been shown that femtosecond pulsed-laser excitation of magnetic thin films produces intense and ultrafast spin-polarized hot electrons, thus attracting a lot of attention. While spin-polarized hot electrons are known to play a pivotal role in the ultrafast laser-induced demagnetization, their effect on magnetization switching remains an open issue. This study uncovers the effect of spin-polarized hot electrons generated by laser excitation on magnetization switching in a Co/Pt based perpendicular magnetic anisotropy-based synthetic antiferromagnet (p-SAF) using the time-resolved magneto-optical Kerr effect. It has been found that, at low pump fluence, the equivalent magnetic field generated by the hot-electron spin current plays a dominant role in assisting the magnetization switching of the lower layer in the antiferromagnetic configuration, while the strong thermal stability of the Ruderman Kittel Kasuya Yosida exchange interaction inhibits the further weakening of the switching field at high pump fluence. This study provides a viable way to control the magnetization switching of the antiferromagnetically exchange-coupled systems for spintronic applications with ultrafast control of the information operation

GLM-130B: An Open Bilingual Pre-trained Model

We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters. It is an attempt to open-source a 100B-scale model at least as good as GPT-3 (davinci) and unveil how models of such a scale can be successfully pre-trained. Over the course of this effort, we face numerous unexpected technical and engineering challenges, particularly on loss spikes and divergence. In this paper, we introduce the training process of GLM-130B including its design choices, training strategies for both efficiency and stability, and engineering efforts. The resultant GLM-130B model offers significant outperformance over GPT-3 175B (davinci) on a wide range of popular English benchmarks while the performance advantage is not observed in OPT-175B and BLOOM-176B. It also consistently and significantly outperforms ERNIE TITAN 3.0 260B -- the largest Chinese language model -- across related benchmarks. Finally, we leverage a unique scaling property of GLM-130B to reach INT4 quantization without post training, with almost no performance loss, making it the first among 100B-scale models and more importantly, allowing its effective inference on 4$\times$RTX 3090 (24G) or 8$\times$RTX 2080 Ti (11G) GPUs, the most affordable GPUs required for using 100B-scale models. The GLM-130B model weights are publicly accessible and its code, training logs, related toolkit, and lessons learned are open-sourced at \url{https://github.com/THUDM/GLM-130B/}.Comment: Accepted to ICLR 202

Rare deleterious germline variants and risk of lung cancer

Recent studies suggest that rare variants exhibit stronger effect sizes and might play a crucial role in the etiology of lung cancers (LC). Whole exome plus targeted sequencing of germline DNA was performed on 1045 LC cases and 885 controls in the discovery set. To unveil the inherited causal variants, we focused on rare and predicted deleterious variants and small indels enriched in cases or controls. Promising candidates were further validated in a series of 26,803 LCs and 555,107 controls. During discovery, we identified 25 rare deleterious variants associated with LC susceptibility, including 13 reported in ClinVar. Of the five validated candidates, we discovered two pathogenic variants in known LC susceptibility loci, ATM p.V2716A (Odds Ratio [OR] 19.55, 95%CI 5.04–75.6) and MPZL2 p.I24M frameshift deletion (OR 3.88, 95%CI 1.71–8.8); and three in novel LC susceptibility genes, POMC c.*28delT at 3′ UTR (OR 4.33, 95%CI 2.03–9.24), STAU2 p.N364M frameshift deletion (OR 4.48, 95%CI 1.73–11.55), and MLNR p.Q334V frameshift deletion (OR 2.69, 95%CI 1.33–5.43). The potential cancer-promoting role of selected candidate genes and variants was further supported by endogenous DNA damage assays. Our analyses led to the identification of new rare deleterious variants with LC susceptibility. However, in-depth mechanistic studies are still needed to evaluate the pathogenic effects of these specific alleles

Efficient spin–orbit torque switching in perpendicularly magnetized CoFeB facilitated by Fe2O3 underlayer

Spin–orbit torque (SOT) is recognized as an effective way to manipulate magnetization in spintronic devices. For the low-power consumption and high-endurance requirements of future computer architectures, reducing the critical SOT switching current density and improving SOT efficiency are crucial, especially in the perpendicularly magnetized structures. Here, we have conducted a comprehensive study on improving the SOT efficiency of the Ta/CoFeB structure with a perpendicular magnetic anisotropy by inserting an oxide insulating layer Fe2O3 as the bottom layer. We found that only a 1–5 nm thickness of Fe2O3 significantly reduces the SOT critical switching current by 70% and enhances the spin Hall angle of Ta. The spin Hall angle increases from 0.078 for pure Ta/CoFeB to 0.13 for Fe2O3/Ta/CoFeB, and both types of spin–orbit torques, damping-like and field-like torques, are significantly enhanced. It is suggested that the atomic diffusion of O from the Fe2O3 underlayer leads to the partial oxidization of the Ta layer as well as the Ta/CoFeB interfaces, accounting for the observed enhanced SOT efficiency. Our results provide a reliable method to improve the SOT performance in perpendicularly magnetized structures by inserting the oxide underlayer using magnetron sputtering, in favor of its potential real-world application in spintronic devices

Direct observation of spin polarization in epitaxial Fe3O4(001)/MgO thin films grown by magnetron sputtering

We obtained epitaxial single-crystal Fe3O4(001)/MgO(001) thin films by magnetron sputtering. The high quality of the grown Fe3O4 films was confirmed by reflection high-energy electron diffraction and x-ray photoelectron spectroscopy. Atomic magnetic properties of Fe3O4(001)/MgO(001) were investigated using vibrating sample magnetometry and x-ray magnetic circular dichroism. The values of saturation magnetization and magnetic moment are 407 ± 5 emu/cm3 (3.26 ± 0.04 μ B / (f. u.)) and 3.31 ± 0.15 μ B / (f. u.), respectively, in the Fe3O4 film as thin as 5 nm, which are close to the bulk values. The spin polarization was directly measured using spin-resolved photoemission spectroscopy. The measured spin polarization has a maximum value of -42% ± 3%, which is comparable to the theoretical value for the (2 × 2)R45° reconstructed Fe3O4(001) surface. Furthermore, the film thickness-dependent measurements indicate that the anti-phase boundaries significantly decrease the spin polarization rather than the lattice mismatch. Our results demonstrate that epitaxial Fe3O4(001)/MgO thin films grown by magnetron sputtering have desired magnetic properties, facilitating the potential application of Fe3O4-based spintronic devices