How to get better embeddings with code pre-trained models? An empirical study

Pre-trained language models have demonstrated powerful capabilities in the field of natural language processing (NLP). Recently, code pre-trained model (PTM), which draw from the experiences of the NLP field, have also achieved state-of-the-art results in many software engineering (SE) downstream tasks. These code PTMs take into account the differences between programming languages and natural languages during pre-training and make adjustments to pre-training tasks and input data. However, researchers in the SE community still inherit habits from the NLP field when using these code PTMs to generate embeddings for SE downstream classification tasks, such as generating semantic embeddings for code snippets through special tokens and inputting code and text information in the same way as pre-training the PTMs. In this paper, we empirically study five different PTMs (i.e. CodeBERT, CodeT5, PLBART, CodeGPT and CodeGen) with three different architectures (i.e. encoder-only, decoder-only and encoder-decoder) on four SE downstream classification tasks (i.e. code vulnerability detection, code clone detection, just-in-time defect prediction and function docstring mismatch detection) with respect to the two aforementioned aspects. Our experimental results indicate that (1) regardless of the architecture of the code PTMs used, embeddings obtained through special tokens do not sufficiently aggregate the semantic information of the entire code snippet; (2) the quality of code embeddings obtained by combing code data and text data in the same way as pre-training the PTMs is poor and cannot guarantee richer semantic information; (3) using the method that aggregates the vector representations of all code tokens, the decoder-only PTMs can obtain code embeddings with semantics as rich as or even better quality than those obtained from the encoder-only and encoder-decoder PTMs

Shrinking the Semantic Gap: Spatial Pooling of Local Moment Invariants for Copy-Move Forgery Detection

Copy-move forgery is a manipulation of copying and pasting specific patches from and to an image, with potentially illegal or unethical uses. Recent advances in the forensic methods for copy-move forgery have shown increasing success in detection accuracy and robustness. However, for images with high self-similarity or strong signal corruption, the existing algorithms often exhibit inefficient processes and unreliable results. This is mainly due to the inherent semantic gap between low-level visual representation and high-level semantic concept. In this paper, we present a very first study of trying to mitigate the semantic gap problem in copy-move forgery detection, with spatial pooling of local moment invariants for midlevel image representation. Our detection method expands the traditional works on two aspects: 1) we introduce the bag-of-visual-words model into this field for the first time, may meaning a new perspective of forensic study; 2) we propose a word-to-phrase feature description and matching pipeline, covering the spatial structure and visual saliency information of digital images. Extensive experimental results show the superior performance of our framework over state-of-the-art algorithms in overcoming the related problems caused by the semantic gap.Comment: 13 pages, 11 figure

From Benchside to Community Research: Development of Affordable and Accessible Probiotic Foods in East Africa

Probiotics are live microorganisms that, when ingested in adequate amounts, confer health benefits. In 2004, Western Heads East brought Fiti to East Africa and trained women how to produce Fiti probiotic yogurt. Fiti is composed of a probiotic culture, Lactobacillus rhamnosus GR-1, and a starter strain, Streptococcus thermophilus C1062. This initiative has since empowered low-income groups to gain financial independence, particularly women. There are currently ~250 kitchens feeding over 250,000 consumers daily in East Africa. A challenge with accessing Fiti yoghurt is the fluctuating cost of milk and inconsistent supply of high quality milk. A potential solution is the consumption of Fiti through affordable non-dairy foods. Pilot studies reveal bacterial viability of Fiti probiotics in probiotic mango juice, orange juice, pineapple juice, mango juice, and millet porridge; there remains no sensory data on these products from East African populations. The research questions sought to examine how individuals in Tanzania and Kenya rated different non-dairy probiotic foods; how these ratings compared to probiotic yoghurt; and how these rating correlated with the willingness of individuals to consume non-dairy probiotic products

Biomechanical evaluation of a novel dualplate fixation method for proximal humeral fractures without medial support

Abstract Background Comminuted fractures of the proximal humerus are generally treated with the locking plate system, and clinical results are satisfactory. However, unstable support of the medial column results in varus malunion and screw perforation. We designed a novel medial anatomical locking plate (MLP) to directly support the medial column. Theoretically, the combined application of locking plate and MLP (LPMP) would directly provide strong dual-column stability. We hypothesized that the LPMP could provide greater construct stability than the locking plate alone (LP), locking plate combined with a fibular graft (LPSG), and locking plate combined with a distal radius plate (LPDP). Methods LP, LPMP, LPSG, and LPDP implants were instrumented into the finite element model of a proximal humeral fracture. Axial, shear, and rotational loads were applied to the models under normal and osteoporotic bone conditions. The whole simulation was repeated five times for each fixator. To assess the biomechanical characteristics, the construct stiffness, fracture micromotion, stress distribution, and neck-shaft angle (NSA) were compared. Results The LPMP group showed significantly greater integral and regional construct stiffness, and endured less von Mises stresses, than the other three fixation methods. The stresses on the lateral locking plate were dispersed by the MLP. The LPMP group showed the least change in NSA. Conclusions From the finite element viewpoint, the LPMP method provided both lateral and medial direct support. The LPMP system was effective in treating proximal humeral fracture with an unstable medial column

Cloning, Characterization and Functional Analysis of <i>Caspase 8-like</i> Gene in Apoptosis of <i>Crassostrea hongkongensis</i> Response to Hyper-Salinity Stress

Caspase-8, a member of the caspase family, is an initiating caspase and plays a crucial role in apoptosis. In this study, the full-length cDNA of caspase8-like (CASP8-like) was isolated from Crassostrea hongkongensis (C. hongkongensis) by RACE-PCR. ChCASP8-like contained a 1599-bp open reading frame (ORF) encoding 533 amino acids with two conserved death effector domains (DEDs) and a cysteine aspartase cysteine structural domain (CASc). Amino acid sequence comparison showed that ChCASP8-like shared the highest identity (85.4%) with CASP8-like of C. angulata. The tissue expression profile showed that ChCASP8-like was constitutively expressed in gills, hepatopancreas, mantle, adductor muscle, hemocytes and gonads, and was significantly upregulated in hemocytes, hepatopancreas and gills under hyper-salinity stress. The apoptosis-related genes, including ATR, CHK1, BCL-XL, CASP8-like, CASP9 and CASP3, were significantly activated by hyper-salinity stress, but were remarkably inhibited by ChCASP8-like silencing. The caspase 8 activity was increased by 1.7-fold after hyper-salinity stress, and was inhibited by 9.4% by ChCASP8-like silencing. Moreover, ChCASP8-like silencing clearly alleviated the apoptosis resulting from hyper-salinity stress. These results collectively demonstrated that ChCASP8-like played a crucial role in inducing apoptosis against hyper-salinity stress

Changes of Inundation Frequency in the Yellow River Delta and Its Response to Wetland Vegetation

The spatiotemporal changes of inundation frequency in the Yellow River Delta (YRD) have profound influences on sustainable ecological protection and are also closely relevant to economic development scarcity on the coast of China. However, long-term changes of inundation frequency have remained poorly characterized. Using the Google Earth Engine (GEE) cloud platform, this study processed Landsat images to explore inundation frequency changes from 1990 to 2020 in the YRD. The results indicated that (1) The existing water index combining the classification results based on the MLM (Maximum Likelihood Method) is suitable for the mapping of the long-term water bodies, especially in the coastal regions; (2) The inundation frequency showed a clearly uneven temporal–spatial distribution. The low inundation area (LIA) is mainly intertidal natural wetlands with a declining trend, while the high inundation area (HIA) is dominated by constructed wetlands with a rising trend; (3) The use frequency of artificial ponds determines the inundation frequency of the constructed wetland. The industry development has gradually matured, causing the inundation frequency from dispersion to concentration in the constructed wetland; and (4) In the natural wetland, the LIA have increased since 2010 and have accounted for 30% in 2020. The large-scale appearance of LIA occurs with the emergence of high vegetation abundance. Spartina alterniflora salt marshes with strong reproductive ability and high abundance lead to the difficulty of wetland flooding and reduce the inundation frequency

Three-dimensional printing model improves morphological understanding in acetabular fracture learning: A multicenter, randomized, controlled study - Fig 5

<p>The results of identifying major fracture lines (A) and classification (B).</p

Example of questions used in the first level test.

<p>Example of questions used in the first level test.</p

Three-dimensional printing model improves morphological understanding in acetabular fracture learning: A multicenter, randomized, controlled study - Fig 1

<p>Example of three-dimensional printing (A) and virtual reality (B) models of acetabular factures.</p

The characteristic of participants.

<p>The characteristic of participants.</p