1,617 research outputs found
Adversarial Robustness of Deep Code Comment Generation
Deep neural networks (DNNs) have shown remarkable performance in a variety of
domains such as computer vision, speech recognition, or natural language
processing. Recently they also have been applied to various software
engineering tasks, typically involving processing source code. DNNs are
well-known to be vulnerable to adversarial examples, i.e., fabricated inputs
that could lead to various misbehaviors of the DNN model while being perceived
as benign by humans. In this paper, we focus on the code comment generation
task in software engineering and study the robustness issue of the DNNs when
they are applied to this task. We propose ACCENT, an identifier substitution
approach to craft adversarial code snippets, which are syntactically correct
and semantically close to the original code snippet, but may mislead the DNNs
to produce completely irrelevant code comments. In order to improve the
robustness, ACCENT also incorporates a novel training method, which can be
applied to existing code comment generation models. We conduct comprehensive
experiments to evaluate our approach by attacking the mainstream
encoder-decoder architectures on two large-scale publicly available datasets.
The results show that ACCENT efficiently produces stable attacks with
functionality-preserving adversarial examples, and the generated examples have
better transferability compared with baselines. We also confirm, via
experiments, the effectiveness in improving model robustness with our training
method
Augmenting Machine Learning with Information Retrieval to Recommend Real Cloned Code Methods for Code Completion
Software developers frequently reuse source code from repositories as it
saves development time and effort. Code clones accumulated in these
repositories hence represent often repeated functionalities and are candidates
for reuse in an exploratory or rapid development. In previous work, we
introduced DeepClone, a deep neural network model trained by fine tuning GPT-2
model over the BigCloneBench dataset to predict code clone methods. The
probabilistic nature of DeepClone output generation can lead to syntax and
logic errors that requires manual editing of the output for final reuse. In
this paper, we propose a novel approach of applying an information retrieval
(IR) technique on top of DeepClone output to recommend real clone methods
closely matching the predicted output. We have quantitatively evaluated our
strategy, showing that the proposed approach significantly improves the quality
of recommendation
Can We Automatically Generate Class Comments in Pharo?
Code comments support developers in understanding and maintaining codebases. Specifically in the Pharo environment, code comments serve as the main form of code documentation and usually convey information ranging from high-level design descriptions to low-level implementation details. Nevertheless, numerous important classes in Pharo still lack comments as developers find writing comments to be a tedious and effort-intensive task. Previous works in Java have recommended generating comments automatically to reduce commenting effort and save developers time. There exist several approaches to achieve this goal. One such popular approach is based on identifying stereotypes, \ie a generalized set of characteristics supposed to represent an entity (object, class). However, this approach has not been tested for other programming languages. In this paper, we adopt the stereotype-based approach to automatically generate class comments in the Pharo programming environment. Specifically, we generated information about the class type, collaborators and key methods. We surveyed seven developers to evaluate the generated comments for 24 classes. The responses suggest that, although more information could be added to the comments, the generated class comments are readable and understandable, and the majority of comments do not contain unnecessary information
A Decade of Code Comment Quality Assessment: A Systematic Literature Review
Code comments are important artifacts in software systems and play a
paramount role in many software engineering (SE) tasks related to maintenance
and program comprehension. However, while it is widely accepted that high
quality matters in code comments just as it matters in source code, assessing
comment quality in practice is still an open problem. First and foremost, there
is no unique definition of quality when it comes to evaluating code comments.
The few existing studies on this topic rather focus on specific attributes of
quality that can be easily quantified and measured. Existing techniques and
corresponding tools may also focus on comments bound to a specific programming
language, and may only deal with comments with specific scopes and clear goals
(e.g., Javadoc comments at the method level, or in-body comments describing
TODOs to be addressed). In this paper, we present a Systematic Literature
Review (SLR) of the last decade of research in SE to answer the following
research questions: (i) What types of comments do researchers focus on when
assessing comment quality? (ii) What quality attributes (QAs) do they consider?
(iii) Which tools and techniques do they use to assess comment quality?, and
(iv) How do they evaluate their studies on comment quality assessment in
general? Our evaluation, based on the analysis of 2353 papers and the actual
review of 47 relevant ones, shows that (i) most studies and techniques focus on
comments in Java code, thus may not be generalizable to other languages, and
(ii) the analyzed studies focus on four main QAs of a total of 21 QAs
identified in the literature, with a clear predominance of checking consistency
between comments and the code. We observe that researchers rely on manual
assessment and specific heuristics rather than the automated assessment of the
comment quality attributes
- …