128 research outputs found
بررسی تاثیر حذف مواد اسیدی از رژیم غذایی در درمان بیماران مبتلا به حساسیت دندانی : کارآزمایی بالینی تصادفی دو سویه کور
Model-Agnostic Syntactical Information for Pre-Trained Programming Language Models
Pre-trained Programming Language Models (PPLMs) achieved many recent states
of the art results for many code-related software engineering tasks. Though
some studies use data flow or propose tree-based models that utilize Abstract
Syntax Tree (AST), most PPLMs do not fully utilize the rich syntactical
information in source code. Still, the input is considered a sequence of
tokens. There are two issues; the first is computational inefficiency due to
the quadratic relationship between input length and attention complexity.
Second, any syntactical information, when needed as an extra input to the
current PPLMs, requires the model to be pre-trained from scratch, wasting all
the computational resources already used for pre-training the current models.
In this work, we propose Named Entity Recognition (NER) adapters, lightweight
modules that can be inserted into Transformer blocks to learn type information
extracted from the AST. These adapters can be used with current PPLMs such as
CodeBERT, GraphCodeBERT, and CodeT5. We train the NER adapters using a novel
Token Type Classification objective function (TTC). We insert our proposed work
in CodeBERT, building CodeBERTER, and evaluate the performance on two tasks of
code refinement and code summarization. CodeBERTER improves the accuracy of
code refinement from 16.4 to 17.8 while using 20% of training parameter budget
compared to the fully fine-tuning approach, and the BLEU score of code
summarization from 14.75 to 15.90 while reducing 77% of training parameters
compared to the fully fine-tuning approach.Comment: 11 pages, 5 Figures, Has been accepted on ICSE 202
ReviewViz: Assisting Developers Perform Empirical Study on Energy Consumption Related Reviews for Mobile Applications
Improving the energy efficiency of mobile applications is a topic that has
gained a lot of attention recently. It has been addressed in a number of ways
such as identifying energy bugs and developing a catalog of energy patterns.
Previous work shows that users discuss the battery-related issues (energy
inefficiency or energy consumption) of the apps in their reviews. However,
there is no work that addresses the automatic extraction of battery-related
issues from users' feedback. In this paper, we report on a visualization tool
that is developed to empirically study machine learning algorithms and text
features to automatically identify the energy consumption specific reviews with
the highest accuracy. Other than the common machine learning algorithms, we
utilize deep learning models with different word embeddings to compare the
results. Furthermore, to help the developers extract the main topics that are
discussed in the reviews, two states of the art topic modeling algorithms are
applied. The visualizations of the topics represent the keywords that are
extracted for each topic along with a comparison with the results of string
matching. The developed web-browser based interactive visualization tool is a
novel framework developed with the intention of giving the app developers
insights about running time and accuracy of machine learning and deep learning
models as well as extracted topics. The tool makes it easier for the developers
to traverse through the extensive result set generated by the text
classification and topic modeling algorithms. The dynamic-data structure used
for the tool stores the baseline-results of the discussed approaches and is
updated when applied on new datasets. The tool is open-sourced to replicate the
research results.Comment: 4 pages, 5 figure
The effect of aerobic exercise and barley β-glucan on blood glucose, body composition and blood pressure of diabetic women
Background: The incidence of type 2 diabetes increases with aging, unhealthy diets, obesity and sedentary lifestyles. The aim of this study was to investigate the combinational effect of a 12-week aerobic exercise and barley β-glucan (BBG) on blood glucose, body composition and blood pressure in women with type 2 diabetes. Materials and Methods: In this semi-experimental study, 24 women with the mean age of 49 years and a blood glucose level of 110-280 mg/dl were purposefully selected and randomly divided into three groups: a) group of aerobic exercise with diet (n=8), b) diet group (n=8) c) control group (n=8). The diet group consumed one barley bread, containing 4 g of β glucan, each day for 12 weeks. The group of aerobic exercise, who was on diet, participated in a progressive walking program with the intensity of 60-70 of maximal heart rate in addition to diet program (barley bread). Blood glucose, weight, fat percentage, and systolic and diastolic blood pressure levels were measured in pre-and post-training. Results: Results showed a significant decrease in the blood glucose level in the experimental groups compared to the control group, while no major changes were observed in body composition and blood pressure. Conclusion: It seems that the combined program (aerobic training with diet) or consumption of β-glucan alone can decrease blood glucose in patients with diabetes
An Exploratory Study on Code Attention in BERT
Many recent models in software engineering introduced deep neural models
based on the Transformer architecture or use transformer-based Pre-trained
Language Models (PLM) trained on code. Although these models achieve the state
of the arts results in many downstream tasks such as code summarization and bug
detection, they are based on Transformer and PLM, which are mainly studied in
the Natural Language Processing (NLP) field. The current studies rely on the
reasoning and practices from NLP for these models in code, despite the
differences between natural languages and programming languages. There is also
limited literature on explaining how code is modeled.
Here, we investigate the attention behavior of PLM on code and compare it
with natural language. We pre-trained BERT, a Transformer based PLM, on code
and explored what kind of information it learns, both semantic and syntactic.
We run several experiments to analyze the attention values of code constructs
on each other and what BERT learns in each layer. Our analyses show that BERT
pays more attention to syntactic entities, specifically identifiers and
separators, in contrast to the most attended token [CLS] in NLP. This
observation motivated us to leverage identifiers to represent the code sequence
instead of the [CLS] token when used for code clone detection. Our results show
that employing embeddings from identifiers increases the performance of BERT by
605% and 4% F1-score in its lower layers and the upper layers, respectively.
When identifiers' embeddings are used in CodeBERT, a code-based PLM, the
performance is improved by 21-24% in the F1-score of clone detection. The
findings can benefit the research community by using code-specific
representations instead of applying the common embeddings used in NLP, and open
new directions for developing smaller models with similar performance.Comment: Accepted in ICPC 202
On the Transferability of Pre-trained Language Models for Low-Resource Programming Languages
A recent study by Ahmed and Devanbu reported that using a corpus of code
written in multilingual datasets to fine-tune multilingual Pre-trained Language
Models (PLMs) achieves higher performance as opposed to using a corpus of code
written in just one programming language. However, no analysis was made with
respect to fine-tuning monolingual PLMs. Furthermore, some programming
languages are inherently different and code written in one language usually
cannot be interchanged with the others, i.e., Ruby and Java code possess very
different structure. To better understand how monolingual and multilingual PLMs
affect different programming languages, we investigate 1) the performance of
PLMs on Ruby for two popular Software Engineering tasks: Code Summarization and
Code Search, 2) the strategy (to select programming languages) that works well
on fine-tuning multilingual PLMs for Ruby, and 3) the performance of the
fine-tuned PLMs on Ruby given different code lengths.
In this work, we analyze over a hundred of pre-trained and fine-tuned models.
Our results show that 1) multilingual PLMs have a lower Performance-to-Time
Ratio (the BLEU, METEOR, or MRR scores over the fine-tuning duration) as
compared to monolingual PLMs, 2) our proposed strategy to select target
programming languages to fine-tune multilingual PLMs is effective: it reduces
the time to fine-tune yet achieves higher performance in Code Summarization and
Code Search tasks, and 3) our proposed strategy consistently shows good
performance on different code lengths.Comment: Accepted in ICPC 202
- …