Exploring Post-Training Quantization of Protein Language Models

Recent advancements in unsupervised protein language models (ProteinLMs), like ESM-1b and ESM-2, have shown promise in different protein prediction tasks. However, these models face challenges due to their high computational demands, significant memory needs, and latency, restricting their usage on devices with limited resources. To tackle this, we explore post-training quantization (PTQ) for ProteinLMs, focusing on ESMFold, a simplified version of AlphaFold based on ESM-2 ProteinLM. Our study is the first attempt to quantize all weights and activations of ProteinLMs. We observed that the typical uniform quantization method performs poorly on ESMFold, causing a significant drop in TM-Score when using 8-bit quantization. We conducted extensive quantization experiments, uncovering unique challenges associated with ESMFold, particularly highly asymmetric activation ranges before Layer Normalization, making representation difficult using low-bit fixed-point formats. To address these challenges, we propose a new PTQ method for ProteinLMs, utilizing piecewise linear quantization for asymmetric activation values to ensure accurate approximation. We demonstrated the effectiveness of our method in protein structure prediction tasks, demonstrating that ESMFold can be accurately quantized to low-bit widths without compromising accuracy. Additionally, we applied our method to the contact prediction task, showcasing its versatility. In summary, our study introduces an innovative PTQ method for ProteinLMs, addressing specific quantization challenges and potentially leading to the development of more efficient ProteinLMs with significant implications for various protein-related applications.Comment: 8 pages, 4 figure

Quantitative and Qualitative Analysis of Aconitum Alkaloids in Raw and Processed Chuanwu and Caowu by HPLC in Combination with Automated Analytical System and ESI/MS/MS

HPLC in combination with automated analytical system and ESI/MS/MS was used to analyze aconitine (A), mesaconitine (MA), hypaconitine (HA), and their benzoyl analogs in the Chinese herbs Caowu and Chuanwu. First, an HPLC method was developed and validated to determine A, MA, and HA in raw and processed Caowu and Chuanwu. Then an automated analytical system and ESI/MS/MS were applied to analyze these alkaloids and their semihydrolyzed products. The results obtained from automated analytical system are identical to those from ESI/MS/MS, which indicated that the method is a convenient and rapid tool for the qualitative analysis of herbal preparations. Furthermore, HA was little hydrolyzed by heating processes and thus it might account more for the toxicity of processed aconites. Hence, HA could be used as an indicator when one alkaloid is required as a reference to monitor the quality of raw and processed Chuanwu and Caowu. In addition, the raw and processed Chuanwu and Caowu can be distinguished by monitoring the ratio of A and MA to HA

Simultaneous Bayesian Clustering and Feature Selection Through Student’s t{t} Mixtures Model

In this paper, we proposed a generative model for feature selection under the unsupervised learning context. The model assumes that data are independently and identically sampled from a finite mixture of Student?s t distributions, which can reduce the sensitiveness to outliers. Latent random variables that represent the features? salience are included in the model for the indication of the relevance of features. As a result, the model is expected to simultaneously realize clustering, feature selection, and outlier detection. Inference is carried out by a tree-structured variational Bayes algorithm. Full Bayesian treatment is adopted in the model to realize automatic model selection. Controlled experimental studies showed that the developed model is capable of modeling the data set with outliers accurately. Further- more, experiment results showed that the developed algorithm compares favorably against existing unsupervised probability model-based Bayesian feature selection algorithms on artificial and real data sets. Moreover, the application of the developed algorithm on real leukemia gene expression data indicated that it is able to identify the discriminating genes successfully

Cancer Nanotechnology: Enhancing Tumor Cell Response to Chemotherapy for Hepatocellular Carcinoma Therapy

Abstract Hepatocellular carcinoma (HCC) is one of the deadliest cancers due to its complexities, reoccurrence after surgical resection, metastasis and heterogeneity. In addition to sorafenib and lenvatinib for the treatment of HCC approved by FDA, various strategies including transarterial chemoembolization, radiotherapy, locoregional therapy and chemotherapy have been investigated in clinics. Recently, cancer nanotechnology has got great attention for the treatment of various cancers including HCC. Both passive and active targetings are progressing at a steady rate. Herein, we describe the lessons learned from pathogenesis of HCC and the understanding of targeted and non-targeted nanoparticles used for the delivery of small molecules, monoclonal antibodies, miRNAs and peptides. Exploring current efficacy is to enhance tumor cell response of chemotherapy. It highlights the opportunities and challenges faced by nanotechnologies in contemporary hepatocellular carcinoma therapy, where personalized medicine is increasingly becoming the mainstay. Overall objective of this review is to enhance our understanding in the design and development of nanotechnology for treatment of HCC

Investigation of genetic diversity and population structure of common wheat cultivars in northern China using DArT markers

<p>Abstract</p> <p>Background</p> <p>In order to help establish heterotic groups of Chinese northern wheat cultivars (lines), Diversity arrays technology (DArT) markers were used to investigate the genetic diversity and population structure of Chinese common wheat (<it>Triticum aestivum </it>L.).</p> <p>Results</p> <p>In total, 1637 of 7000 DArT markers were polymorphic and scored with high confidence among a collection of 111 lines composed mostly of cultivars and breeding lines from northern China. The polymorphism information content (PIC) of DArT markers ranged from 0.03 to 0.50, with an average of 0.40, with P > 80 (reliable markers). With principal-coordinates analysis (PCoA) of DArT data either from the whole genome or from the B-genome alone, all lines fell into one of two major groups reflecting 1RS/1BL type (1RS/1BL and non-1RS/1BL). Evidence of geographic clustering of genotypes was also observed using DArT markers from the A genome. Cluster analysis based on the unweighted pair-group method with algorithmic mean suggested the existence of two subgroups within the non-1RS/1BL group and four subgroups within the 1RS/1BL group. Furthermore, analysis of molecular variance (AMOVA) revealed highly significant (<it>P </it>< 0.001) genetic variance within and among subgroups and among groups.</p> <p>Conclusion</p> <p>These results provide valuable information for selecting crossing parents and establishing heterotic groups in the Chinese wheat-breeding program.</p

Insufficient Radiofrequency Ablation Promotes Angiogenesis of Residual Hepatocellular Carcinoma Via HIF-1α/VEGFA

Background: The mechanism of rapid growth of the residual tumor after radiofrequency (RF) ablation is poorly understood. In this study, we investigated the effect of hyperthermia on HepG2 cells and generated a subline with enhanced viability and dys-regulated angiogenesis in vivo, which was used as a model to further determine the molecular mechanism of the rapid growth of residual HCC after RF ablation. Methodology/Principal Findings: Heat treatment was used to establish sublines of HepG2 cells. A subline (HepG2 k) with a relatively higher viability and significant heat tolerance was selected. The cellular protein levels of VEGFA, HIF-1α and p-Akt, VEGFA mRNA and secreted VEGFA were measured, and all of these were up-regulated in this subline compared to parental HepG2 cells. HIF-1α inhibitor YC-1 and VEGFA siRNA inhibited the high viability of the subline. The conditioned media from the subline exerted stronger pro-angiogenic effects. Bevacizumab, VEGFA siRNA and YC-1 inhibited proangiogenic effects of the conditioned media of HepG2 k cells and abolished the difference between parental HepG2 cells and HepG2 k cells. For in vivo studies, a nude mouse model was used, and the efficacy of bavacizumab was determined. HepG2 k tumor had stronger pro-angiogenic effects than parental HepG2 tumor. Bevacizumab could inhibit the tumor growth and angiogenesis, and also eliminate the difference in tumor growth and angiogenesis between parental HepG2 tumor and HepG2 k tumor in vivo. Conclusions/Significance: The angiogenesis induced by HIF1α/VEGFA produced by altered cells after hyperthermia treatment may play an important role in the rapid growth of residual HCC after RF ablation. Bevacizumab may be a good candidate drug for preventing and treating the process

Snoring, Inflammatory Markers, Adipokines and Metabolic Syndrome in Apparently Healthy Chinese

OBJECTIVE: Chronic low-grade inflammation and adipokines dysregulation are linked to mechanisms underscoring the pathogenesis of obesity-related metabolic disorders. Little is known about roles of these cytokines on the association between snoring and metabolic syndrome (MetS). We aimed to investigate whether a cluster of cytokines are related to snoring frequency and its association with MetS in apparently healthy Chinese. METHODS: Current analyses used a population-based sample including 1059 Shanghai residents aged 35-54 years. Self-reported snoring frequency was classified as never, occasionally and regularly. Fasting plasma glucose, lipid profile, insulin, C-reactive protein, interleukin-6, interleukin-18, lipopolysaccharide binding protein, high-molecular-weight adiponectin and leptin were measured. MetS was defined by the updated National Cholesterol Education Program Adult Treatment Panel III criteria for Asian-Americans. RESULTS: Overweight/obese subjects had significantly higher prevalence of regular snorers than their normal-weight counterparts (34.8% vs. 11.5%, P<0.001). Regular snoring was associated with unfavorable profile of inflammatory markers and adipokines. However, those associations were abolished after adjustment for body mass index (BMI) or waist circumference. The MetS risk (multivariate-adjusted odds ratio 5.41, 95% confidence interval 3.72-7.88) was substantially higher in regular snorers compared with non-snorers. Controlling for BMI remarkably attenuated the association (2.03, 1.26-3.26), while adjusting for inflammatory markers and adipokines showed little effects. CONCLUSION: Frequent snoring was associated with an elevated MetS risk independent of lifestyle factors, adiposity, inflammatory markers and adipokines in apparently healthy Chinese. Whether snoring pattern is an economic and no-invasive indicator for screening high-risk persons needs to be addressed prospectively

Hyperpolarization-activated cyclic nucleotide-gated cation channel 3 promotes HCC development in a female-biased manner

Sex differences in hepatocellular carcinoma (HCC) development are regulated by sex and non-sex chromosomes, sex hormones, and environmental factors. We previously reported that Ncoa5(+/-) mice develop HCC in a male-biased manner. Here we show that NCOA5 expression is reduced in male patient HCCs while the expression of an NCOA5-interacting tumor suppressor, TIP30, is lower in female HCCs. Tip30 heterozygous deletion does not change HCC incidence in Ncoa5(+/-) male mice but dramatically increases HCC incidence in Ncoa5(+/-) female mice, accompanied by hepatic hyperpolarization-activated cyclic nucleotide-gated cation channel 3 (HCN3) overexpression. HCN3 overexpression cooperates with MYC to promote mouse HCC development, whereas Hcn3 knockout preferentially hinders HCC development in female mice. Furthermore, HCN3 amplification and overexpression occur in human HCCs and correlate with a poorer prognosis of patients in a female-biased manner. Our results suggest that TIP30 and NCOA5 protect against female liver oncogenesis and that HCN3 is a female-biased HCC driver

Globally Profiling Sialylation Status of Macrophages Upon Statin Treatment

Sialic acids (SAs) are widely expressed on immune cells and their levels and linkages named as sialylation status vary upon cellular environment changes related to both physiological and pathological processes. In this study, we performed a global profiling of the sialylation status of macrophages and their release of SAs in the cell culture medium by using flow cytometry, confocal microscopy and liquid chromatography tandem mass spectrometry (LC-MS/MS). Both flow cytometry and confocal microscopy results showed that cell surface α-2,3-linked SAs were predominant in the normal culture condition and changed slightly upon treatment with atorvastatin for 24 h, whereas α-2,6-linked SAs were negligible in the normal culture condition but significantly increased after treatment. Meanwhile, the amount of total cellular SAs increased about three times (from 369 ± 29 to 1080 ± 50 ng/mL) upon treatment as determined by the LC-MS/MS method. On the other hand, there was no significant change for secreted free SAs and conjugated SAs in the medium. These results indicated that the cell surface α-2,6 sialylation status of macrophages changes distinctly upon atorvastatin stimulation, which may reflect on the biological functions of the cells