57 research outputs found
Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs
Large Language Models (LLMs) have proven their exceptional capabilities in
performing language-related tasks. However, their deployment poses significant
challenges due to their considerable memory and storage requirements. In
response to this issue, weight-only quantization, particularly 3 and 4-bit
weight-only quantization, has emerged as one of the most viable solutions. As
the number of bits decreases, the quantization grid broadens, thus emphasizing
the importance of up and down rounding. While previous studies have
demonstrated that fine-tuning up and down rounding with the addition of
perturbations can enhance accuracy in some scenarios, our study is driven by
the precise and limited boundary of these perturbations, where only the
threshold for altering the rounding value is of significance. Consequently, we
propose a concise and highly effective approach for optimizing the weight
rounding task. Our method, named SignRound, involves lightweight block-wise
tuning using signed gradient descent, enabling us to achieve outstanding
results within 400 steps. SignRound competes impressively against recent
methods without introducing additional inference overhead. The source code will
be publicly available at \url{https://github.com/intel/neural-compressor} soon
Modulation Design and Optimization for RIS-Assisted Symbiotic Radios
In reconfigurable intelligent surface (RIS)-assisted symbiotic radio (SR),
the RIS acts as a secondary transmitter by modulating its information bits over
the incident primary signal and simultaneously assists the primary
transmission, then a cooperative receiver is used to jointly decode the primary
and secondary signals. Most existing works of SR focus on using RIS to enhance
the reflecting link while ignoring the ambiguity problem for the joint
detection caused by the multiplication relationship of the primary and
secondary signals. Particularly, in case of a blocked direct link, joint
detection will suffer from severe performance loss due to the ambiguity, when
using the conventional on-off keying and binary phase shift keying modulation
schemes for RIS. To address this issue, we propose a novel modulation scheme
for RIS-assisted SR that divides the phase-shift matrix into two components:
the symbol-invariant and symbol-varying components, which are used to assist
the primary transmission and carry the secondary signal, respectively. To
design these two components, we focus on the detection of the composite signal
formed by the primary and secondary signals, through which a problem of
minimizing the bit error rate (BER) of the composite signal is formulated to
improve both the BER performance of the primary and secondary ones. By solving
the problem, we derive the closed-form solution of the optimal symbol-invariant
and symbol-varying components, which is related to the channel strength ratio
of the direct link to the reflecting link. Moreover, theoretical BER
performance is analyzed. Finally, simulation results show the superiority of
the proposed modulation scheme over its conventional counterpart.Comment: 16 pages,15 figure
Comparison of Efficiencies of Non-invasive Prenatal Testing, Karyotyping, and Chromosomal Micro-Array for Diagnosing Fetal Chromosomal Anomalies in the Second and Third Trimesters
In this study, we aimed to compare the efficiency of non-invasive prenatal testing (NIPT), karyotyping, and chromosomal micro-array (CMA) for the diagnosis of fetal chromosomal anomalies in the second and third trimesters. Pregnant women, who underwent amniocenteses for prenatal genetic diagnoses during their middle and late trimesters, were recruited at the Prenatal Diagnosis Center of Taizhou City. Maternal blood was separated for NIPT, and amniotic fluid cells were cultured for karyotyping and CMA. The diagnostic efficiency of NIPT for detecting fetal imbalanced anomalies was compared with karyotyping and CMA. A total of 69 fetal chromosomal imbalances were confirmed by CMA, 37 were diagnosed by NIPT and 35 were found by karyotyping. The sensitivities of NIPT and karyotyping for diagnosing aneuploidy were 96.3% and 100% respectively. Only one mosaic sexual chromosome monosomy was misdiagnosed by NIPT, whereas the sensitivity of NIPT and karyotyping was 70% and 30%, respectively, for detecting pathogenic deletions and duplications sized from 5–20 Mb. Taken together, our results suggest that the efficiency of NIPT was similar to the formula karyotyping for detecting chromosome imbalance in the second and third trimesters
Effective Quantization for Diffusion Models on CPUs
Diffusion models have gained popularity for generating images from textual
descriptions. Nonetheless, the substantial need for computational resources
continues to present a noteworthy challenge, contributing to time-consuming
processes. Quantization, a technique employed to compress deep learning models
for enhanced efficiency, presents challenges when applied to diffusion models.
These models are notably more sensitive to quantization compared to other model
types, potentially resulting in a degradation of image quality. In this paper,
we introduce a novel approach to quantize the diffusion models by leveraging
both quantization-aware training and distillation. Our results show the
quantized models can maintain the high image quality while demonstrating the
inference efficiency on CPUs. The code is publicly available at:
https://github.com/intel/intel-extension-for-transformers
Coarse-to-fine Knowledge Graph Domain Adaptation based on Distantly-supervised Iterative Training
Modern supervised learning neural network models require a large amount of
manually labeled data, which makes the construction of domain-specific
knowledge graphs time-consuming and labor-intensive. In parallel, although
there has been much research on named entity recognition and relation
extraction based on distantly supervised learning, constructing a
domain-specific knowledge graph from large collections of textual data without
manual annotations is still an urgent problem to be solved. In response, we
propose an integrated framework for adapting and re-learning knowledge graphs
from one coarse domain (biomedical) to a finer-define domain (oncology). In
this framework, we apply distant-supervision on cross-domain knowledge graph
adaptation. Consequently, no manual data annotation is required to train the
model. We introduce a novel iterative training strategy to facilitate the
discovery of domain-specific named entities and triples. Experimental results
indicate that the proposed framework can perform domain adaptation and
construction of knowledge graph efficiently
Trace Metal Distribution in Sulfide Minerals from Ultramafic-Hosted Hydrothermal Systems: Examples from the Kairei Vent Field, Central Indian Ridge
The ultramafic-hosted Kairei vent field is located at 25°19′ S, 70°02′ E, towards the Northern end of segment 1 of the Central Indian Ridge (CIR-S1) at a water depth of ~2450 m. This study aims to investigate the distribution of trace elements among sulfide minerals of differing textures and to examine the possible factors controlling the trace element distribution in those minerals using LA-ICP-MS spot and line scan analyses. Our results show that there are distinct systematic differences in trace element distributions throughout the different minerals, as follows: (1) pyrite is divided into three types at Kairei, including early-stage euhedral pyrite (py-I), sub-euhedral pyrite (py-II), and colloform pyrite (py-III). Pyrite is generally enriched with Mo, Au, As, Tl, Mn, and U. Pyrite-I has high contents of Se, Te, Bi, and Ni when compared to the other types; py-II is enriched in Au relative to py-I and py-III, but poor in Ni; py-III is enriched in Mo, Pb, and U but is poor in Se, Te, Bi, and Au relative to py-I and py-II. Variations in the concentrations of Se, Te, and Bi in pyrite are most likely governed by the strong temperature gradient. There is generally a lower concentration of nickel than Co in pyrite, indicating that our samples precipitated at high temperatures, whereas the extreme Co enrichment is likely from a magmatic heat source combined with an influence of serpentinization reactions. (2) Chalcopyrite is characterized by high concentrations of Co, Se, and Te. The abundance of Se and Te in chalcopyrite over the other minerals is interpreted to have been caused by the high solubilities of Se and Te in the chalcopyrite lattice at high temperatures. The concentrations of Sb, As, and Au are relatively low in chalcopyrite from the Kairei vent field. (3) Sphalerite from Zn-rich chimneys is characterized by high concentrations of Sn, Co, Ga, Ge, Ag, Pb, Sb, As, and Cd, but is depleted in Se, Te, Bi, Mo, Au, Ni, Tl, Mn, Ba, V, and U in comparison with the other minerals. The high concentrations of Cd and Co are likely caused by the substitution of Cd2+ and Co2+ for Zn2+ in sphalerite. A high concentration of Pb accompanied by a high Ag concentration in sphalerite indicates that Ag occurs as Pb–Ag sulfosalts. Gold is generally low in sphalerite and strongly correlates with Pb, suggesting its presence in microinclusions of galena. The strong correlation of As with Ge in sphalerite from Kairei suggests that they might precipitate at medium temperatures and under moderately reduced conditions. (4) Bornite–digenite has very low concentrations of most trace elements, except for Co, Se, and Bi. Serpentinization in ultramafic-hosted hydrothermal systems might play an important role in Au enrichment in pyrite with low As contents. Compared to felsic-hosted seafloor massive sulfide deposits, sulfide minerals from ultramafic-hosted deposits show higher concentrations of Se and Te, but lower As, Sb, and Au concentrations, the latter often attributed to the contribution of magmatic volatiles. As with typical ultramafic-hosted seafloor massive sulfide deposits, Se enrichment in chalcopyrite from Kairei indicates that the primary factor that controls the Se enrichment is temperature-controlled mobility in vent fluids
Extracellular RNA in melanoma: Advances, challenges, and opportunities
Melanoma, a malignant mass lesion that originates in melanocytes and has a high rate of malignancy, metastasis, and mortality, is defined by these characteristics. Malignant melanoma is a kind of highly malignant tumor that produces melanin and has a high mortality rate. Its incidence accounts for 1%–3% of all malignant tumors and shows an obvious upward trend. The discovery of biomolecules for the diagnosis and treatment of malignant melanoma has important application value. So far, the exact molecular mechanism of melanoma development relevant signal pathway still remains unclear. According to previous studies, extracellular RNAs (exRNAs) have been implicated in tumorigenesis and spread of melanoma. They can influence the proliferation, invasion and metastasis of melanoma by controlling the expression of target genes and can also influence tumor progression by participating in signal transduction mechanisms. Therefore, understanding the relationship between exRNA and malignant melanoma and targeting therapy is of positive significance for its prevention and treatment. In this review, we did an analysis of extracellular vesicles of melanoma which focused on the role of exRNAs (lncRNAs, miRNAs, and mRNAs) and identifies several potential therapeutic targets. In addition, we discuss the typical signaling pathways involved in exRNAs, advances in exRNA detection and how they affect the tumor immune microenvironment in melanoma
Analysis of age-related differences in hypoxia-related factors in yak brain tissue
The brain is an important part of the mammalian nervous system, is highly sensitive to hypoxia, and plays an important role in the adaptation of the body to hypoxic environments. This study was conducted to study the distribution and expression of hypoxia-related factors (hypoxia-inducible factor 1α, HIF-1α; erythropoietin, EPO; vascular endothelial growth factor, VEGF; vascular cell adhesion molecule, VCAM) in the cerebellum, cerebrum, medulla oblongata, and corpora quadrigemina in yaks of different ages (4d, 6-months-old and adult). Paraffin sections were obtained from the cerebellum, cerebrum, medulla oblongata, and corpora quadrigemina of healthy yak for 4-day-old, 6-months-old and adult yaks. Histological characteristics were assessed by haematoxylin staining. Immunohistochemical staining was performed to detect the distribution and expression of HIF-1α, EPO, VEGF and VCAM proteins. Immunohistochemical results showed that HIF-1α, EPO, VEGF, and VCAM were expressed in the pyramidal cell layer of the yak cerebrum, and distributed in the cerebellum granulose cell layer, Purkinje cell layer and medulla layer, and were mainly positive in Purkinje cells and medulla. It is expressed in the cell bodies of the medulla oblongata and the quadrimatous neurons. The expression level in the medulla oblongata was higher, indicating may play a crucial role in functional cohesion. The expression of HIF-1α in 4 d cerebellar tissues was higher than that in other age groups, and the expression of HIF-1α in the medulla oblongata increased with age. In addition, the expression levels of EPO and VEGF in the 6-month-old group were slightly higher than those in the other age groups. It is speculated that EPO and VEGF have obvious protective effects on brain tissue in the 6-month-old age group; VCAM showed no significant differences in the cerebrum, cerebellum, medulla oblongata, or corpora quadrigemina of the yaks. This study provides basic data for further exploration of the adaptive mechanism of plateau yak brain tissue
Real-time Monitoring for the Next Core-Collapse Supernova in JUNO
Core-collapse supernova (CCSN) is one of the most energetic astrophysical
events in the Universe. The early and prompt detection of neutrinos before
(pre-SN) and during the SN burst is a unique opportunity to realize the
multi-messenger observation of the CCSN events. In this work, we describe the
monitoring concept and present the sensitivity of the system to the pre-SN and
SN neutrinos at the Jiangmen Underground Neutrino Observatory (JUNO), which is
a 20 kton liquid scintillator detector under construction in South China. The
real-time monitoring system is designed with both the prompt monitors on the
electronic board and online monitors at the data acquisition stage, in order to
ensure both the alert speed and alert coverage of progenitor stars. By assuming
a false alert rate of 1 per year, this monitoring system can be sensitive to
the pre-SN neutrinos up to the distance of about 1.6 (0.9) kpc and SN neutrinos
up to about 370 (360) kpc for a progenitor mass of 30 for the case
of normal (inverted) mass ordering. The pointing ability of the CCSN is
evaluated by using the accumulated event anisotropy of the inverse beta decay
interactions from pre-SN or SN neutrinos, which, along with the early alert,
can play important roles for the followup multi-messenger observations of the
next Galactic or nearby extragalactic CCSN.Comment: 24 pages, 9 figure
- …