Integration of Pre-trained Protein Language Models into Geometric Deep Learning Networks

Geometric deep learning has recently achieved great success in non-Euclidean domains, and learning on 3D structures of large biomolecules is emerging as a distinct research area. However, its efficacy is largely constrained due to the limited quantity of structural data. Meanwhile, protein language models trained on substantial 1D sequences have shown burgeoning capabilities with scale in a broad range of applications. Several previous studies consider combining these different protein modalities to promote the representation power of geometric neural networks, but fail to present a comprehensive understanding of their benefits. In this work, we integrate the knowledge learned by well-trained protein language models into several state-of-the-art geometric networks and evaluate a variety of protein representation learning benchmarks, including protein-protein interface prediction, model quality assessment, protein-protein rigid-body docking, and binding affinity prediction. Our findings show an overall improvement of 20% over baselines. Strong evidence indicates that the incorporation of protein language models' knowledge enhances geometric networks' capacity by a significant margin and can be generalized to complex tasks

InstructBio: A Large-scale Semi-supervised Learning Paradigm for Biochemical Problems

In the field of artificial intelligence for science, it is consistently an essential challenge to face a limited amount of labeled data for real-world problems. The prevailing approach is to pretrain a powerful task-agnostic model on a large unlabeled corpus but may struggle to transfer knowledge to downstream tasks. In this study, we propose InstructMol, a semi-supervised learning algorithm, to take better advantage of unlabeled examples. It introduces an instructor model to provide the confidence ratios as the measurement of pseudo-labels' reliability. These confidence scores then guide the target model to pay distinct attention to different data points, avoiding the over-reliance on labeled data and the negative influence of incorrect pseudo-annotations. Comprehensive experiments show that InstructBio substantially improves the generalization ability of molecular models, in not only molecular property predictions but also activity cliff estimations, demonstrating the superiority of the proposed method. Furthermore, our evidence indicates that InstructBio can be equipped with cutting-edge pretraining methods and used to establish large-scale and task-specific pseudo-labeled molecular datasets, which reduces the predictive errors and shortens the training process. Our work provides strong evidence that semi-supervised learning can be a promising tool to overcome the data scarcity limitation and advance molecular representation learning

Interaction between Dysfunctional Connectivity at Rest and Heroin Cues-Induced Brain Responses in Male Abstinent Heroin-Dependent Individuals

BACKGROUND: The majority of previous heroin cue-reactivity functional magnetic resonance imaging (fMRI) studies focused on local function impairments, such as inhibitory control, decision-making and stress regulation. Our previous studies have demonstrated that these brain circuits also presented dysfunctional connectivity during the resting state. Yet few studies considered the relevance of resting state dysfunctional connectivity to task-related neural activity in the same chronic heroin user (CHU). METHODOLOGY/PRINCIPAL FINDINGS: We employed the method of graph theory analysis, which detected the abnormality of brain regions and dysregulation of brain connections at rest between 16 male abstinent chronic heroin users (CHUs) and 16 non-drug users (NDUs). Using a cue-reactivity task, we assessed the relationship between drug-related cue-induced craving activity and the abnormal topological properties of the CHUs' resting networks. Comparing NDUs' brain activity to that of CHUs, the intensity of functional connectivity of the medial frontal gyrus (meFG) in patients' resting state networks was prominently greater and positively correlated with the same region's neural activity in the heroin-related task; decreased functional connectivity intensity of the anterior cingulate cortex (ACC) in CHUs at rest was associated with more drug-related cue-induced craving activities. CONCLUSIONS: These results may indicate that there exist two brain systems interacting simultaneously in the heroin-addicted brain with regards to a cue-reactivity task. The current study may shed further light on the neural architecture that supports craving responses in heroin dependence

Gender-Related Differences in the Dysfunctional Resting Networks of Migraine Suffers

BACKGROUND: Migraine shows gender-specific incidence and has a higher prevalence in females. However, little is known about gender-related differences in dysfunctional brain organization, which may account for gender-specific vulnerability and characteristics of migraine. In this study, we considered gender-related differences in the topological property of resting functional networks. METHODOLOGY/PRINCIPAL FINDINGS: Data was obtained from 38 migraine patients (18 males and 20 females) and 38 healthy subjects (18 males and 20 females). We used the graph theory analysis, which becomes a powerful tool in investigating complex brain networks on a whole brain scale and could describe functional interactions between brain regions. Using this approach, we compared the brain functional networks between these two groups, and several network properties were investigated, such as small-worldness, network resilience, nodal centrality, and interregional connections. In our findings, these network characters were all disrupted in patients suffering from chronic migraine. More importantly, these functional damages in the migraine-affected brain had a skewed balance between males and females. In female patients, brain functional networks showed worse resilience, more regions exhibited decreased nodal centrality, and more functional connections revealed abnormalities than in male patients. CONCLUSIONS: These results indicated that migraine may have an additional influence on females and lead to more dysfunctional organization in their resting functional networks

Assessing the Microbial Community and Functional Genes in a Vertical Soil Profile with Long-Term Arsenic Contamination

Conceived and designed the experiments: GW. Performed the experiments: JX GL. Analyzed the data: JX JZ GW. Contributed reagents/materials/analysis tools: ST JZ GW. Wrote the paper: JX ZH JDVN JZ GW.Arsenic (As) contamination in soil and groundwater has become a serious problem to public health. To examine how microbial communities and functional genes respond to long-term arsenic contamination in vertical soil profile, soil samples were collected from the surface to the depth of 4 m (with an interval of 1 m) after 16-year arsenic downward infiltration. Integrating BioLog and functional gene microarray (GeoChip 3.0) technologies, we showed that microbial metabolic potential and diversity substantially decreased, and community structure was markedly distinct along the depth. Variations in microbial community functional genes, including genes responsible for As resistance, carbon and nitrogen cycling, phosphorus utilization and cytochrome c oxidases were detected. In particular, changes in community structures and activities were correlated with the biogeochemical features along the vertical soil profile when using the rbcL and nifH genes as biomarkers, evident for a gradual transition from aerobic to anaerobic lifestyles. The C/N showed marginally significant correlations with arsenic resistance (p = 0.069) and carbon cycling genes (p = 0.073), and significant correlation with nitrogen fixation genes (p = 0.024). The combination of C/N, NO3− and P showed the highest correlation (r = 0.779, p = 0.062) with the microbial community structure. Contradict to our hypotheses, a long-term arsenic downward infiltration was not the primary factor, while the spatial isolation and nutrient availability were the key forces in shaping the community structure. This study provides new insights about the heterogeneity of microbial community metabolic potential and future biodiversity preservation for arsenic bioremediation management.Yeshttp://www.plosone.org/static/editorial#pee