How to get better embeddings with code pre-trained models? An empirical study

Pre-trained language models have demonstrated powerful capabilities in the field of natural language processing (NLP). Recently, code pre-trained model (PTM), which draw from the experiences of the NLP field, have also achieved state-of-the-art results in many software engineering (SE) downstream tasks. These code PTMs take into account the differences between programming languages and natural languages during pre-training and make adjustments to pre-training tasks and input data. However, researchers in the SE community still inherit habits from the NLP field when using these code PTMs to generate embeddings for SE downstream classification tasks, such as generating semantic embeddings for code snippets through special tokens and inputting code and text information in the same way as pre-training the PTMs. In this paper, we empirically study five different PTMs (i.e. CodeBERT, CodeT5, PLBART, CodeGPT and CodeGen) with three different architectures (i.e. encoder-only, decoder-only and encoder-decoder) on four SE downstream classification tasks (i.e. code vulnerability detection, code clone detection, just-in-time defect prediction and function docstring mismatch detection) with respect to the two aforementioned aspects. Our experimental results indicate that (1) regardless of the architecture of the code PTMs used, embeddings obtained through special tokens do not sufficiently aggregate the semantic information of the entire code snippet; (2) the quality of code embeddings obtained by combing code data and text data in the same way as pre-training the PTMs is poor and cannot guarantee richer semantic information; (3) using the method that aggregates the vector representations of all code tokens, the decoder-only PTMs can obtain code embeddings with semantics as rich as or even better quality than those obtained from the encoder-only and encoder-decoder PTMs

Noninvasive prenatal diagnosis of 21-Hydroxylase deficiency using target capture sequencing of maternal plasma DNA.

Here, we aimed to validate a noninvasive method using capture sequencing for prenatal diagnosis of congenital adrenal hyperplasia due to 21-Hydroxylase deficiency (21-OHD). Noninvasive prenatal diagnosis (NIPD) of 21-OHD was based on 14 plasma samples collected from 12 families, including four plasma sample collected during the first trimester. Targeted capture sequencing was performed using genomic DNA from the parents and child trios to determine the pathogenic and wild-type alleles associated with the haplotypes. Maternal plasma DNA was also sequenced to determine the fetal inheritance of the allele using hidden Markov model-based haplotype linkage analysis. The effect of fetal DNA fraction and sequencing depth on the accuracy of NIPD was investigated. The lower limit of fetal DNA fraction was 2% and the threshold mean sequence depth was 38, suggesting potential advantage if used in early gestation. The CYP21A2 genotype of the fetus was accurately determined in all the 14 plasma samples as early as day 1 and 8 weeks of gestation. Results suggest the accuracy and feasibility of NIPD of 21-OHD using a small target capture region with a low threshold for fetal DNA fraction and sequence depth. Our method is cost-effective and suggests diagnostic applications in clinical practice

Biomechanical evaluation of a novel dualplate fixation method for proximal humeral fractures without medial support

Abstract Background Comminuted fractures of the proximal humerus are generally treated with the locking plate system, and clinical results are satisfactory. However, unstable support of the medial column results in varus malunion and screw perforation. We designed a novel medial anatomical locking plate (MLP) to directly support the medial column. Theoretically, the combined application of locking plate and MLP (LPMP) would directly provide strong dual-column stability. We hypothesized that the LPMP could provide greater construct stability than the locking plate alone (LP), locking plate combined with a fibular graft (LPSG), and locking plate combined with a distal radius plate (LPDP). Methods LP, LPMP, LPSG, and LPDP implants were instrumented into the finite element model of a proximal humeral fracture. Axial, shear, and rotational loads were applied to the models under normal and osteoporotic bone conditions. The whole simulation was repeated five times for each fixator. To assess the biomechanical characteristics, the construct stiffness, fracture micromotion, stress distribution, and neck-shaft angle (NSA) were compared. Results The LPMP group showed significantly greater integral and regional construct stiffness, and endured less von Mises stresses, than the other three fixation methods. The stresses on the lateral locking plate were dispersed by the MLP. The LPMP group showed the least change in NSA. Conclusions From the finite element viewpoint, the LPMP method provided both lateral and medial direct support. The LPMP system was effective in treating proximal humeral fracture with an unstable medial column

Investigation of the Tribological Behavior and Microstructure of Plasma-Cladded Fe–Cr–Mo–Ni–B Coating

In this study, an Fe–Cr–Mo–Ni–B coating was prepared using plasma cladding on Cr5 steel substrate. The microstructure, phase evolution and tribological performance of the Fe–Cr–Mo–Ni–B coating were investigated. The microstructure is mainly composed of Mo2FeB2, Fe2B, α-Fe, γ-Fe and MoB. The process of phase evolution in the coating was observed in situ by HT-CLSM. The Mo2FeB2 phase with good thermodynamic stability can exist in the high-temperature liquid phase. It also has a phenomenon of connection and merging and turns into different morphology during the plasma cladding process. The hardness value of coating was much higher than the base metal, and the hardness value of Mo2FeB2 (785.5 HV) was higher than the eutectic matrix (693.2 HV). The wear mechanisms of the cladding under dry sliding were primarily caused by adhesive wear, accompanying slight oxidation wear. The Mo2FeB2 phase has an important effect on the wear resistance property

Percutaneous Endoscopic Transforaminal Lumbar Interbody Fusion for the Treatment of Lumbar Spinal Stenosis: Preliminary Report of Seven Cases with 12-Month Follow-Up

Purpose. The objective of this study was to investigate the preliminary effectiveness of percutaneous endoscopic transforaminal lumbar interbody fusion (PE-TLIF) for the treatment of lumbar spinal stenosis (LSS). Methods. From September 2016 to June 2017, a series of seven patients consisting of six females and one male with an average age of 55.25 years (range 43–77 years) who were diagnosed with LSS were involved in this study. All patients were treated by PE-TLIF. During perioperative and follow-up period, demographic data, operation time, intraoperative blood loss, Visual Analogue Scale (VAS), Oswestry Disability Index (ODI), and modified MacNab criteria were evaluated and perioperative complications were documented. Results. All patients were followed up for more than 12 months, with an average follow-up time of 15 (range 12-21) months. The mean VAS of back pain was 7.43 (range 6-8) preoperatively and 0.86 (range 0-2) at the final follow-up. The mean VAS of leg pain was 6.14 (range 4-9) preoperatively and 0.71 (range 0-1) at the final follow-up. The mean ODI was 53.57% (range 38%-63%) preoperatively and 15.57% (range 5%-26%) at the final follow-up. In three-month follow-up, continuous bone trabeculae bridging between intervertebral bodies was seen in 3 cases, and the remaining 4 cases could identify continuous bone trabeculae bridging at 6-month follow-up, reaching the standard of spinal intervertebral fusion. At the final follow-up, 4 patients were rated as excellent (4/7) and 3 patients were rated as good (3/7) according to the modified MacNab criteria. Conclusions. Our study suggested that percutaneous endoscopic transforaminal lumbar interbody fusion could acquire satisfactory treatment effects for the patients with lumbar spinal stenosis, even for the patient who could not afford general anesthesia

Three-dimensional printing model improves morphological understanding in acetabular fracture learning: A multicenter, randomized, controlled study - Fig 5

<p>The results of identifying major fracture lines (A) and classification (B).</p

Example of questions used in the first level test.

<p>Example of questions used in the first level test.</p

Three-dimensional printing model improves morphological understanding in acetabular fracture learning: A multicenter, randomized, controlled study - Fig 1

<p>Example of three-dimensional printing (A) and virtual reality (B) models of acetabular factures.</p

The characteristic of participants.

<p>The characteristic of participants.</p