Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs

Large Language Models (LLMs) have proven their exceptional capabilities in performing language-related tasks. However, their deployment poses significant challenges due to their considerable memory and storage requirements. In response to this issue, weight-only quantization, particularly 3 and 4-bit weight-only quantization, has emerged as one of the most viable solutions. As the number of bits decreases, the quantization grid broadens, thus emphasizing the importance of up and down rounding. While previous studies have demonstrated that fine-tuning up and down rounding with the addition of perturbations can enhance accuracy in some scenarios, our study is driven by the precise and limited boundary of these perturbations, where only the threshold for altering the rounding value is of significance. Consequently, we propose a concise and highly effective approach for optimizing the weight rounding task. Our method, named SignRound, involves lightweight block-wise tuning using signed gradient descent, enabling us to achieve outstanding results within 400 steps. SignRound competes impressively against recent methods without introducing additional inference overhead. The source code will be publicly available at \url{https://github.com/intel/neural-compressor} soon

Modulation Design and Optimization for RIS-Assisted Symbiotic Radios

In reconfigurable intelligent surface (RIS)-assisted symbiotic radio (SR), the RIS acts as a secondary transmitter by modulating its information bits over the incident primary signal and simultaneously assists the primary transmission, then a cooperative receiver is used to jointly decode the primary and secondary signals. Most existing works of SR focus on using RIS to enhance the reflecting link while ignoring the ambiguity problem for the joint detection caused by the multiplication relationship of the primary and secondary signals. Particularly, in case of a blocked direct link, joint detection will suffer from severe performance loss due to the ambiguity, when using the conventional on-off keying and binary phase shift keying modulation schemes for RIS. To address this issue, we propose a novel modulation scheme for RIS-assisted SR that divides the phase-shift matrix into two components: the symbol-invariant and symbol-varying components, which are used to assist the primary transmission and carry the secondary signal, respectively. To design these two components, we focus on the detection of the composite signal formed by the primary and secondary signals, through which a problem of minimizing the bit error rate (BER) of the composite signal is formulated to improve both the BER performance of the primary and secondary ones. By solving the problem, we derive the closed-form solution of the optimal symbol-invariant and symbol-varying components, which is related to the channel strength ratio of the direct link to the reflecting link. Moreover, theoretical BER performance is analyzed. Finally, simulation results show the superiority of the proposed modulation scheme over its conventional counterpart.Comment: 16 pages,15 figure

Comparison of Efficiencies of Non-invasive Prenatal Testing, Karyotyping, and Chromosomal Micro-Array for Diagnosing Fetal Chromosomal Anomalies in the Second and Third Trimesters

In this study, we aimed to compare the efficiency of non-invasive prenatal testing (NIPT), karyotyping, and chromosomal micro-array (CMA) for the diagnosis of fetal chromosomal anomalies in the second and third trimesters. Pregnant women, who underwent amniocenteses for prenatal genetic diagnoses during their middle and late trimesters, were recruited at the Prenatal Diagnosis Center of Taizhou City. Maternal blood was separated for NIPT, and amniotic fluid cells were cultured for karyotyping and CMA. The diagnostic efficiency of NIPT for detecting fetal imbalanced anomalies was compared with karyotyping and CMA. A total of 69 fetal chromosomal imbalances were confirmed by CMA, 37 were diagnosed by NIPT and 35 were found by karyotyping. The sensitivities of NIPT and karyotyping for diagnosing aneuploidy were 96.3% and 100% respectively. Only one mosaic sexual chromosome monosomy was misdiagnosed by NIPT, whereas the sensitivity of NIPT and karyotyping was 70% and 30%, respectively, for detecting pathogenic deletions and duplications sized from 5–20 Mb. Taken together, our results suggest that the efficiency of NIPT was similar to the formula karyotyping for detecting chromosome imbalance in the second and third trimesters

Effective Quantization for Diffusion Models on CPUs

Diffusion models have gained popularity for generating images from textual descriptions. Nonetheless, the substantial need for computational resources continues to present a noteworthy challenge, contributing to time-consuming processes. Quantization, a technique employed to compress deep learning models for enhanced efficiency, presents challenges when applied to diffusion models. These models are notably more sensitive to quantization compared to other model types, potentially resulting in a degradation of image quality. In this paper, we introduce a novel approach to quantize the diffusion models by leveraging both quantization-aware training and distillation. Our results show the quantized models can maintain the high image quality while demonstrating the inference efficiency on CPUs. The code is publicly available at: https://github.com/intel/intel-extension-for-transformers

Coarse-to-fine Knowledge Graph Domain Adaptation based on Distantly-supervised Iterative Training

Modern supervised learning neural network models require a large amount of manually labeled data, which makes the construction of domain-specific knowledge graphs time-consuming and labor-intensive. In parallel, although there has been much research on named entity recognition and relation extraction based on distantly supervised learning, constructing a domain-specific knowledge graph from large collections of textual data without manual annotations is still an urgent problem to be solved. In response, we propose an integrated framework for adapting and re-learning knowledge graphs from one coarse domain (biomedical) to a finer-define domain (oncology). In this framework, we apply distant-supervision on cross-domain knowledge graph adaptation. Consequently, no manual data annotation is required to train the model. We introduce a novel iterative training strategy to facilitate the discovery of domain-specific named entities and triples. Experimental results indicate that the proposed framework can perform domain adaptation and construction of knowledge graph efficiently

Trace Metal Distribution in Sulfide Minerals from Ultramafic-Hosted Hydrothermal Systems: Examples from the Kairei Vent Field, Central Indian Ridge

The ultramafic-hosted Kairei vent field is located at 25°19′ S, 70°02′ E, towards the Northern end of segment 1 of the Central Indian Ridge (CIR-S1) at a water depth of ~2450 m. This study aims to investigate the distribution of trace elements among sulfide minerals of differing textures and to examine the possible factors controlling the trace element distribution in those minerals using LA-ICP-MS spot and line scan analyses. Our results show that there are distinct systematic differences in trace element distributions throughout the different minerals, as follows: (1) pyrite is divided into three types at Kairei, including early-stage euhedral pyrite (py-I), sub-euhedral pyrite (py-II), and colloform pyrite (py-III). Pyrite is generally enriched with Mo, Au, As, Tl, Mn, and U. Pyrite-I has high contents of Se, Te, Bi, and Ni when compared to the other types; py-II is enriched in Au relative to py-I and py-III, but poor in Ni; py-III is enriched in Mo, Pb, and U but is poor in Se, Te, Bi, and Au relative to py-I and py-II. Variations in the concentrations of Se, Te, and Bi in pyrite are most likely governed by the strong temperature gradient. There is generally a lower concentration of nickel than Co in pyrite, indicating that our samples precipitated at high temperatures, whereas the extreme Co enrichment is likely from a magmatic heat source combined with an influence of serpentinization reactions. (2) Chalcopyrite is characterized by high concentrations of Co, Se, and Te. The abundance of Se and Te in chalcopyrite over the other minerals is interpreted to have been caused by the high solubilities of Se and Te in the chalcopyrite lattice at high temperatures. The concentrations of Sb, As, and Au are relatively low in chalcopyrite from the Kairei vent field. (3) Sphalerite from Zn-rich chimneys is characterized by high concentrations of Sn, Co, Ga, Ge, Ag, Pb, Sb, As, and Cd, but is depleted in Se, Te, Bi, Mo, Au, Ni, Tl, Mn, Ba, V, and U in comparison with the other minerals. The high concentrations of Cd and Co are likely caused by the substitution of Cd2+ and Co2+ for Zn2+ in sphalerite. A high concentration of Pb accompanied by a high Ag concentration in sphalerite indicates that Ag occurs as Pb–Ag sulfosalts. Gold is generally low in sphalerite and strongly correlates with Pb, suggesting its presence in microinclusions of galena. The strong correlation of As with Ge in sphalerite from Kairei suggests that they might precipitate at medium temperatures and under moderately reduced conditions. (4) Bornite–digenite has very low concentrations of most trace elements, except for Co, Se, and Bi. Serpentinization in ultramafic-hosted hydrothermal systems might play an important role in Au enrichment in pyrite with low As contents. Compared to felsic-hosted seafloor massive sulfide deposits, sulfide minerals from ultramafic-hosted deposits show higher concentrations of Se and Te, but lower As, Sb, and Au concentrations, the latter often attributed to the contribution of magmatic volatiles. As with typical ultramafic-hosted seafloor massive sulfide deposits, Se enrichment in chalcopyrite from Kairei indicates that the primary factor that controls the Se enrichment is temperature-controlled mobility in vent fluids

Extracellular RNA in melanoma: Advances, challenges, and opportunities

Melanoma, a malignant mass lesion that originates in melanocytes and has a high rate of malignancy, metastasis, and mortality, is defined by these characteristics. Malignant melanoma is a kind of highly malignant tumor that produces melanin and has a high mortality rate. Its incidence accounts for 1%–3% of all malignant tumors and shows an obvious upward trend. The discovery of biomolecules for the diagnosis and treatment of malignant melanoma has important application value. So far, the exact molecular mechanism of melanoma development relevant signal pathway still remains unclear. According to previous studies, extracellular RNAs (exRNAs) have been implicated in tumorigenesis and spread of melanoma. They can influence the proliferation, invasion and metastasis of melanoma by controlling the expression of target genes and can also influence tumor progression by participating in signal transduction mechanisms. Therefore, understanding the relationship between exRNA and malignant melanoma and targeting therapy is of positive significance for its prevention and treatment. In this review, we did an analysis of extracellular vesicles of melanoma which focused on the role of exRNAs (lncRNAs, miRNAs, and mRNAs) and identifies several potential therapeutic targets. In addition, we discuss the typical signaling pathways involved in exRNAs, advances in exRNA detection and how they affect the tumor immune microenvironment in melanoma

Analysis of age-related differences in hypoxia-related factors in yak brain tissue

The brain is an important part of the mammalian nervous system, is highly sensitive to hypoxia, and plays an important role in the adaptation of the body to hypoxic environments. This study was conducted to study the distribution and expression of hypoxia-related factors (hypoxia-inducible factor 1α, HIF-1α; erythropoietin, EPO; vascular endothelial growth factor, VEGF; vascular cell adhesion molecule, VCAM) in the cerebellum, cerebrum, medulla oblongata, and corpora quadrigemina in yaks of different ages (4d, 6-months-old and adult). Paraffin sections were obtained from the cerebellum, cerebrum, medulla oblongata, and corpora quadrigemina of healthy yak for 4-day-old, 6-months-old and adult yaks. Histological characteristics were assessed by haematoxylin staining. Immunohistochemical staining was performed to detect the distribution and expression of HIF-1α, EPO, VEGF and VCAM proteins. Immunohistochemical results showed that HIF-1α, EPO, VEGF, and VCAM were expressed in the pyramidal cell layer of the yak cerebrum, and distributed in the cerebellum granulose cell layer, Purkinje cell layer and medulla layer, and were mainly positive in Purkinje cells and medulla. It is expressed in the cell bodies of the medulla oblongata and the quadrimatous neurons. The expression level in the medulla oblongata was higher, indicating may play a crucial role in functional cohesion. The expression of HIF-1α in 4 d cerebellar tissues was higher than that in other age groups, and the expression of HIF-1α in the medulla oblongata increased with age. In addition, the expression levels of EPO and VEGF in the 6-month-old group were slightly higher than those in the other age groups. It is speculated that EPO and VEGF have obvious protective effects on brain tissue in the 6-month-old age group; VCAM showed no significant differences in the cerebrum, cerebellum, medulla oblongata, or corpora quadrigemina of the yaks. This study provides basic data for further exploration of the adaptive mechanism of plateau yak brain tissue

Real-time Monitoring for the Next Core-Collapse Supernova in JUNO

Core-collapse supernova (CCSN) is one of the most energetic astrophysical events in the Universe. The early and prompt detection of neutrinos before (pre-SN) and during the SN burst is a unique opportunity to realize the multi-messenger observation of the CCSN events. In this work, we describe the monitoring concept and present the sensitivity of the system to the pre-SN and SN neutrinos at the Jiangmen Underground Neutrino Observatory (JUNO), which is a 20 kton liquid scintillator detector under construction in South China. The real-time monitoring system is designed with both the prompt monitors on the electronic board and online monitors at the data acquisition stage, in order to ensure both the alert speed and alert coverage of progenitor stars. By assuming a false alert rate of 1 per year, this monitoring system can be sensitive to the pre-SN neutrinos up to the distance of about 1.6 (0.9) kpc and SN neutrinos up to about 370 (360) kpc for a progenitor mass of 30$M_{\odot}$ for the case of normal (inverted) mass ordering. The pointing ability of the CCSN is evaluated by using the accumulated event anisotropy of the inverse beta decay interactions from pre-SN or SN neutrinos, which, along with the early alert, can play important roles for the followup multi-messenger observations of the next Galactic or nearby extragalactic CCSN.Comment: 24 pages, 9 figure