Exploring Post-Training Quantization of Protein Language Models

Recent advancements in unsupervised protein language models (ProteinLMs), like ESM-1b and ESM-2, have shown promise in different protein prediction tasks. However, these models face challenges due to their high computational demands, significant memory needs, and latency, restricting their usage on devices with limited resources. To tackle this, we explore post-training quantization (PTQ) for ProteinLMs, focusing on ESMFold, a simplified version of AlphaFold based on ESM-2 ProteinLM. Our study is the first attempt to quantize all weights and activations of ProteinLMs. We observed that the typical uniform quantization method performs poorly on ESMFold, causing a significant drop in TM-Score when using 8-bit quantization. We conducted extensive quantization experiments, uncovering unique challenges associated with ESMFold, particularly highly asymmetric activation ranges before Layer Normalization, making representation difficult using low-bit fixed-point formats. To address these challenges, we propose a new PTQ method for ProteinLMs, utilizing piecewise linear quantization for asymmetric activation values to ensure accurate approximation. We demonstrated the effectiveness of our method in protein structure prediction tasks, demonstrating that ESMFold can be accurately quantized to low-bit widths without compromising accuracy. Additionally, we applied our method to the contact prediction task, showcasing its versatility. In summary, our study introduces an innovative PTQ method for ProteinLMs, addressing specific quantization challenges and potentially leading to the development of more efficient ProteinLMs with significant implications for various protein-related applications.Comment: 8 pages, 4 figure

Cerebral hemodynamic characteristics of acute mountain sickness upon acute high-altitude exposure at 3,700 m in young Chinese men.

PURPOSE: We aimed at identifying the cerebral hemodynamic characteristics of acute mountain sickness (AMS). METHODS: Transcranial Doppler (TCD) sonography examinations were performed between 18 and 24 h after arrival at 3,700 m via plane from 500 m (n = 454). A subgroup of 151 subjects received TCD examinations at both altitudes. RESULTS: The velocities of the middle cerebral artery, vertebral artery (VA) and basilar artery (BA) increased while the pulsatility indexes (PIs) and resistance indexes (RIs) decreased significantly (all p < 0.05). Velocities of BA were higher in AMS (AMS+) individuals when compared with non-AMS (AMS-) subjects (systolic velocity: 66 ± 12 vs. 69 ± 15 cm/s, diastolic velocity: 29 ± 7 vs. 31 ± 8 cm/s and mean velocity, 42 ± 9 vs. 44 ± 10 cm/s). AMS was characterized by higher diastolic velocity [V d_VA (26 ± 4 vs. 25 ± 4, p = 0.013)] with lower PI and RI (both p = 0.004) in VA. Furthermore, the asymmetry index (AI) of VAs was significantly lower in the AMS + group [-5.7 % (21.0 %) vs. -2.5 % (17.8 %), p = 0.016]. The AMS score was closely correlated with the hemodynamic parameters of BA and the V d_VA, PI, RI and AI of VA. CONCLUSION: AMS is associated with alterations in cerebral hemodynamics in the posterior circulation rather than the anterior one, and is characterized by higher blood velocity with lower resistance. In addition, the asymmetry of VAs may be involved in AMS

Anatomic and functional evaluation of the lymphatics and lymph nodes in diagnosis of lymphatic circulation disorders with contrast magnetic resonance lymphangiography

ObjectivesOwing to its structural and anatomic characteristics, imaging of the lymphatic system has been difficult. The conventional diagnostic method of radionuclide-based imaging has the disadvantage of poor resolution. Recent work has shown that magnetic resonance imaging (MRI) can depict lymphatic channels in patients with lymphedema. This study evaluated the anatomic and functional images of contrast MR lymphangiography in the diagnosis of limb lymphatic circulation disorders.MethodsThe study enrolled 27 patients with primary lymphedema. Four patients had bilateral disease, and 23 had unilateral disease. Contrast-enhanced lymphangiography was performed with a 3.0-T MR unit after the intracutaneous injection of gadobenate dimeglumine into the interdigital webs of the dorsal foot. The kinetics of enhanced lymph flow within the lymphatic system were calculated using the formula [speed in cm = total length of visualized lymph vessel in cm/inspection time in minutes] and by comparing dynamic nodal enhancement and time-signal intensity curves between edematous and contralateral limbs. Morphologic abnormalities of the lymphatic system were also evaluated.ResultsExamination of the MRIs after injection of the contrast agent showed enhanced lymphatic channels consistently visualized in all clinical lymphedematous limbs and in five contralateral limbs of unilateral lymphedema patients. The speed of flow within the lymphatics of lymphedematous limbs was 0.3 to 1.48 cm/min. Contrast enhancement in inguinal nodes of edematous limbs was significantly less than that of contralateral limbs (P < .01). Dynamic measurement of contrast enhancement showed a remarkable lowering of peak time (P < .01) and peak enhancement (P < .01), and a delay in outflow in inguinal nodes of affected limbs compared with that of control limbs. Postcontrast MRI also depicted varied distribution patterns of lymphatics and abnormal lymph flow pathways within lymph nodes in the limbs with lymphatic circulation disorders.ConclusionContrast MR lymphangiography with gadobenate dimeglumine is capable of visualizing the precise anatomy of lymphatic vessels and lymph nodes in lymphedematous limbs. It also provides information concerning the functional status of lymph flow transport in the lymphatic vessels and lymph nodes of these limbs

Stem cell factor SALL4, a potential prognostic marker for myelodysplastic syndromes

Background: Myelodysplastic syndromes (MDS) are a group of heterogeneous diseases with variable clinical course. Predicting disease progression is difficult due to lack of specific molecular marker(s). SALL4 plays important roles in normal hematopoiesis and leukemogenesis. SALL4 transgenic mice develop MDS prior to acute myeloid leukemia (AML) transformation. However, the role of SALL4 in human MDS has not been extensively investigated. In this study, we evaluate the diagnostic/prognostic value of SALL4 in MDS by examining its expression levels in a cohort of MDS patients. Methods: Fifty-five newly diagnosed MDS, twenty MDS-AML, and sixteen post-treatment MDS patients were selected for our study along with ten healthy donors. Results: We demonstrated that SALL4 was over-expressed in MDS patients and proportionally increased in MDS patients with high grade/IPSS scores. This expression pattern was similar to that of Bmi-1, an important marker in predicting MDS/AML progression. In addition, the level of SALL4 was positively correlated with increased blast counts, high-risk keryotypes and increased significantly in MDS-AML transformation. Furthermore, higher level of SALL4 expression was associated with worse survival rates and SALL4 level decreased following effective therapy. Conclusions: To the best of our knowledge, this is the largest series and the first to report the expression pattern of SALL4 in detail in various subtypes of MDS in comparison to that of Bmi-1. We conclude that SALL4 is a potential molecular marker in predicting the prognosis of MDS

Radiative transitions in charmonium from Nf=2N_f=2 twisted mass lattice QCD

We present a study for charmonium radiative transitions: $J/\psi\rightarrow\eta_c\gamma$, $\chi_{c0}\rightarrow J/\Psi\gamma$ and $h_c\rightarrow\eta_c\gamma$ using $N_f=2$ twisted mass lattice QCD gauge configurations. The single-quark vector form factors for $\eta_c$ and $\chi_{c0}$ are also determined. The simulation is performed at a lattice spacing of $a= 0.06666$ fm and the lattice size is $32^3\times 64$. After extrapolation of lattice data at nonzero $Q^2$ to 0, we compare our results with previous quenched lattice results and the available experimental values.Comment: typeset with revtex, 15 pages, 11 figures, 4 table

Galactic Disk Bulk Motions as Revealed by the LSS-GAC DR2

We report a detailed investigation of the bulk motions of the nearby Galactic stellar disk, based on three samples selected from the LSS-GAC DR2: a global sample containing 0.57 million FGK dwarfs out to $\sim$ 2 kpc, a local subset of the global sample consisting $\sim$ 5,400 stars within 150 pc, and an anti-center sample containing $\sim$ 4,400 AFGK dwarfs and red clump stars within windows of a few degree wide centered on the Galactic anti-center. The global sample is used to construct a three-dimensional map of bulk motions of the Galactic disk from the solar vicinity out to $\sim$ 2 kpc with a spatial resolution of $\sim$ 250 pc. Typical values of the radial and vertical components of bulk motion range from $-$15 km s$^{-1}$ to 15 km s$^{-1}$, while the lag behind the circular speed dominates the azimuthal component by up to $\sim$ 15 km s$^{-1}$. The map reveals spatially coherent, kpc-scale stellar flows in the disk, with typical velocities of a few tens km s$^{-1}$. Bending- and breathing-mode perturbations are clearly visible, and vary smoothly across the disk plane. Our data also reveal higher-order perturbations, such as breaks and ripples, in the profiles of vertical motion versus height. From the local sample, we find that stars of different populations exhibit very different patterns of bulk motion. Finally, the anti-center sample reveals a number of peaks in stellar number density in the line-of-sight velocity versus distance distribution, with the nearer ones apparently related to the known moving groups. The "velocity bifurcation" reported by Liu et al. (2012) at Galactocentric radii 10--11 kpc is confirmed. However, just beyond this distance, our data also reveal a new triple-peaked structure.Comment: 27 pages, 17 figures, Accepted for publication in a special issue of Research in Astronomy and Astrophysics on LAMOST science

Amelioration of the neuroinhibitory local environment after ischemic injury through in situ astrocyte-to-neuron conversion

Ischemic injury in central nervous system (CNS) often causes severe neuronal loss and activates glial cells. We showed earlier that NeuroD1-mediated astrocyte-to-neuron (AtN) conversion can regenerate a substantial proportion of neurons (~40% of the total) and reconstruct the ischemic injured neural circuits. In this study, we focus on glial changes and blood vessel recovery following AtN conversion. Specifically, we found that ectopic expression of NeuroD1 in the reactive astrocytes after ischemic injury significantly reduced glial reactivity, as shown by less hypertrophic morphology, along with reduced secretion of neuroinhibitory factors such as CSPG and LCN2. As for microglia, we found less amoeboid shape of reactive microglia with reduced inflammatory factors such as IL-1β, TNFα. Moreover, blood vessels in the injured areas were repaired after AtN conversion and the blood-brain-barrier structure was restored. Whole tissue transcriptome sequencing identified significantly reduced reactive astrocyte genes and proinflammatory genes, as well as an upregulation of neurogenesis pathway and angiogenesis genes. Together, we demonstrate that NeuroD1-mediated astrocyte-to-neuron (AtN) conversion can alleviate glial scarring and inflammation to create a more neuropermissive micro-environment for functional recovery