CBEAF-Adapting: Enhanced Continual Pretraining for Building Chinese Biomedical Language Model

Continual pretraining is a standard way of building a domain-specific pretrained language model from a general-domain language model. However, sequential task training may cause catastrophic forgetting, which affects the model performance in downstream tasks. In this paper, we propose a continual pretraining method for the BERT-based model, named CBEAF-Adapting (Chinese Biomedical Enhanced Attention-FFN Adapting). Its main idea is to introduce a small number of attention heads and hidden units inside each self-attention layer and feed-forward network. Using the Chinese biomedical domain as a running example, we trained a domain-specific language model named CBEAF-RoBERTa. We conduct experiments by applying models to downstream tasks. The results demonstrate that with only about 3% of model parameters trained, our method could achieve about 0.5%, 2% average performance gain compared to the best performing model in baseline and the domain-specific model, PCL-MedBERT, respectively. We also examine the forgetting problem of different pretraining methods. Our method alleviates the problem by about 13% compared to fine-tuning

Spinal nerve segmentation method and dataset construction in endoscopic surgical scenarios

Endoscopic surgery is currently an important treatment method in the field of spinal surgery and avoiding damage to the spinal nerves through video guidance is a key challenge. This paper presents the first real-time segmentation method for spinal nerves in endoscopic surgery, which provides crucial navigational information for surgeons. A finely annotated segmentation dataset of approximately 10,000 consec-utive frames recorded during surgery is constructed for the first time for this field, addressing the problem of semantic segmentation. Based on this dataset, we propose FUnet (Frame-Unet), which achieves state-of-the-art performance by utilizing inter-frame information and self-attention mechanisms. We also conduct extended exper-iments on a similar polyp endoscopy video dataset and show that the model has good generalization ability with advantageous performance. The dataset and code of this work are presented at: https://github.com/zzzzzzpc/FUnet .Comment: Accepted by MICCAI 202

The Applications of Finite Element Analysis in Proximal Humeral Fractures

Proximal humeral fractures are common and most challenging, due to the complexity of the glenohumeral joint, especially in the geriatric population with impacted fractures, that the development of implants continues because currently the problems with their fixation are not solved. Pre-, intra-, and postoperative assessments are crucial in management of those patients. Finite element analysis, as one of the valuable tools, has been implemented as an effective and noninvasive method to analyze proximal humeral fractures, providing solid evidence for management of troublesome patients. However, no review article about the applications and effects of finite element analysis in assessing proximal humeral fractures has been reported yet. This review article summarized the applications, contribution, and clinical significance of finite element analysis in assessing proximal humeral fractures. Furthermore, the limitations of finite element analysis, the difficulties of more realistic simulation, and the validation and also the creation of validated FE models were discussed. We concluded that although some advancements in proximal humeral fractures researches have been made by using finite element analysis, utility of this powerful tool for routine clinical management and adequate simulation requires more state-of-the-art studies to provide evidence and bases

Potential metabolic mechanism of girls' central precocious puberty: a network analysis on urine metabonomics data

BACKGROUND: Central precocious puberty (CPP) is a common pediatric endocrine disease caused by early activation of hypothalamic-putuitary-gonadal (HPG) axis, yet the exact mechanism was poorly understood. Although there were some proofs that an altered metabolic profile was involved in CPP, interpreting the biological implications at a systematic level is still in pressing need. To gain a systematic understanding of the biological implications, this paper analyzed the CPP differential urine metabolites from a network point of view. RESULTS: In this study, differential urine metabolites between CPP girls and age-matched normal ones were identified by LC-MS. Their basic topological parameters were calculated in the background network. The network decomposition suggested that CPP differential urine metabolites were most relevant to amino acid metabolism. Further proximity analysis of CPP differential urine metabolites and neuro-endocrine metabolites showed a close relationship between CPP metabolism and neuro-endocrine system. Then the core metabolic network of CPP was successfully constructed among all these differential urine metabolites. As can be demonstrated in the core network, abnormal aromatic amino acid metabolism might influence the activity of HPG and hypothalamic pituitary adrenal (HPA) axis. Several adjustments to the early activation of puberty in CPP girls could also be revealed by urine metabonomics. CONCLUSIONS: The present article demonstrated the ability of urine metabonomics to provide several potential metabolic clues for CPP's mechanism. It was revealed that abnormal metabolism of amino acid, especially aromatic amino acid, might have a close correlation with CPP's pathogenesis by activating HPG axis and suppressing HPA axis. Such a method of network-based analysis could also be applied to other metabonomics analysis to provide an overall perspective at a systematic level

Preservation of Litchi Fruit with Nanosilver Composite Particles (Ag-NP) and Resistance against <i>Peronophythora litchi</i>

Litchi (Litchi chinensis Sonn.) is susceptible to infection by Peronophythora litchi post storage, which rapidly decreases the sensory and nutritional quality of the fruit. In this study, the effects of nanosilver (Ag-NP) solution treatment on the shelf life of litchi fruit and the inhibition of P. litchi were examined, and the underlying mechanisms were discussed. For investigations, we used one variety of litchi (‘Feizixiao’), dipping it in different concentrations of Ag-NP solution after harvesting. Meanwhile, we treated P. litchi with different concentrations of Ag-NP solution. According to the data analysis, litchi treated with 400 μg/mL Ag-NPs and stored at 4 °C had the highest health rate and the lowest browning index among all the samples. In the same trend, treatment with 400 μg/mL Ag-NPs produced the best results for anthocyanin content, total soluble solids content, and titratable acidity content. Additionally, according to the results of the inhibition test, 800 μg/mL Ag-NP solution had a 94.97% inhibition rate against P. litchi. Within 2–10 h following exposure to 400 μg/mL Ag-NP solution, the contents of superoxide dismutase, peroxidase, and catalase in P. litchi gradually increased and peaked, followed by a gradual decline. At this time, the integrity of the cell membrane of P. litchi could be broken by Ag-NP solution, and the sporangia showed deformed germ tubes and abnormal shapes. Taken together, these results suggested that Ag-NP treatment inhibited respiration and P. litchi activity, which might attenuate litchi pericarp browning and prolong the shelf life of litchi. Accordingly, Ag-NPs could be used as an effective antistaling agent in litchi fruit and as an ecofriendly fungicide for the post-harvest control of litchi downy blight. This study provides new insights into the application of Ag-NP as an antistaling agent for fruit storage and as an ecofriendly fungicide

Mutational burden and potential oligogenic model of TBX6

Abstract Background Congenital scoliosis (CS) is a spinal deformity due to vertebral malformations. Although insufficiency of TBX6 dosage contributes to a substantial proportion of CS, the molecular etiology for the majority of CS remains largely unknown. TBX6‐mediated genes involved in the process of somitogenesis represent promising candidates. Methods Individuals affected with CS and without a positive genetic finding were referred to this study. Proband‐only exome sequencing (ES) were performed on the recruited individuals, followed by analysis of TBX6‐mediated candidate genes, namely MEOX1, MEOX2, MESP2, MYOD1, MYF5, RIPPLY1, and RIPPLY2. Results A total of 584 patients with CS of unknown molecular etiology were recruited. After ES analysis, protein‐truncating variants in RIPPLY1 and MYF5 were identified from two individuals, respectively. In addition, we identified five deleterious missense variants (MYOD1, n = 4; RIPPLY2, n = 1) in TBX6‐mediated genes. We observed a significant mutational burden of MYOD1 in CS (p = 0.032) compared with the in‐house controls (n = 1854). Moreover, a potential oligogenic disease‐causing mode was proposed based on the observed mutational co‐existence of MYOD1/MEOX1 and MYOD1/RIPPLY1. Conclusion Our study characterized the mutational spectrum of TBX6‐mediated genes, prioritized core candidate genes/variants, and provided insight into a potential oligogenic disease‐causing mode in CS