74 research outputs found
CARBON: A Counterfactual Reasoning based Framework for Neural Code Comprehension Debiasing
Previous studies have demonstrated that code intelligence models are
sensitive to program transformation among which identifier renaming is
particularly easy to apply and effective. By simply renaming one identifier in
source code, the models would output completely different results. The prior
research generally mitigates the problem by generating more training samples.
Such an approach is less than ideal since its effectiveness depends on the
quantity and quality of the generated samples. Different from these studies, we
are devoted to adjusting models for explicitly distinguishing the influence of
identifier names on the results, called naming bias in this paper, and thereby
making the models robust to identifier renaming. Specifically, we formulate the
naming bias with a structural causal model (SCM), and propose a counterfactual
reasoning based framework named CARBON for eliminating the naming bias in
neural code comprehension. CARBON explicitly captures the naming bias through
multi-task learning in the training stage, and reduces the bias by
counterfactual inference in the inference stage. We evaluate CARBON on three
neural code comprehension tasks, including function naming, defect detection
and code classification. Experiment results show that CARBON achieves
relatively better performance (e.g., +0.5% on the function naming task at F1
score) than the baseline models on the original benchmark datasets, and
significantly improvement (e.g., +37.9% on the function naming task at F1
score) on the datasets with identifiers renamed. The proposed framework
provides a causal view for improving the robustness of code intelligence
models
Domain Knowledge Matters: Improving Prompts with Fix Templates for Repairing Python Type Errors
Although the dynamic type system of Python facilitates the developers in
writing Python programs, it also brings type errors at run-time. There exist
rule-based approaches for automatically repairing Python type errors. The
approaches can generate accurate patches but they require domain experts to
design patch synthesis rules and suffer from low template coverage of
real-world type errors. Learning-based approaches alleviate the manual efforts
in designing patch synthesis rules. Among the learning-based approaches, the
prompt-based approach which leverages the knowledge base of code pre-trained
models via pre-defined prompts, obtains state-of-the-art performance in general
program repair tasks. However, such prompts are manually defined and do not
involve any specific clues for repairing Python type errors, resulting in
limited effectiveness. How to automatically improve prompts with the domain
knowledge for type error repair is challenging yet under-explored. In this
paper, we present TypeFix, a novel prompt-based approach with fix templates
incorporated for repairing Python type errors. TypeFix first mines generalized
fix templates via a novel hierarchical clustering algorithm. The identified fix
templates indicate the common edit patterns and contexts of existing type error
fixes. TypeFix then generates code prompts for code pre-trained models by
employing the generalized fix templates as domain knowledge, in which the masks
are adaptively located for each type error instead of being pre-determined.
Experiments on two benchmarks, including BugsInPy and TypeBugs, show that
TypeFix successfully repairs 26 and 55 type errors, outperforming the best
baseline approach by 9 and 14, respectively. Besides, the proposed fix template
mining approach can cover 75% of developers' patches in both benchmarks,
increasing the best rule-based approach PyTER by more than 30%.Comment: This paper has been accepted by ICSE'2
SCALE: Constructing Structured Natural Language Comment Trees for Software Vulnerability Detection
Recently, there has been a growing interest in automatic software
vulnerability detection. Pre-trained model-based approaches have demonstrated
superior performance than other Deep Learning (DL)-based approaches in
detecting vulnerabilities. However, the existing pre-trained model-based
approaches generally employ code sequences as input during prediction, and may
ignore vulnerability-related structural information, as reflected in the
following two aspects. First, they tend to fail to infer the semantics of the
code statements with complex logic such as those containing multiple operators
and pointers. Second, they are hard to comprehend various code execution
sequences, which is essential for precise vulnerability detection.
To mitigate the challenges, we propose a Structured Natural Language Comment
tree-based vulnerAbiLity dEtection framework based on the pre-trained models,
named SCALE. The proposed Structured Natural Language Comment Tree (SCT)
integrates the semantics of code statements with code execution sequences based
on the Abstract Syntax Trees (ASTs). Specifically, SCALE comprises three main
modules: (1) Comment Tree Construction, which aims at enhancing the model's
ability to infer the semantics of code statements by first incorporating Large
Language Models (LLMs) for comment generation and then adding the comment node
to ASTs. (2) Structured Natural Language Comment Tree Construction}, which aims
at explicitly involving code execution sequence by combining the code syntax
templates with the comment tree. (3) SCT-Enhanced Representation, which finally
incorporates the constructed SCTs for well capturing vulnerability patterns.Comment: Accepted by ISSTA 202
Dynamically Relative Position Encoding-Based Transformer for Automatic Code Edit
Adapting Deep Learning (DL) techniques to automate non-trivial coding
activities, such as code documentation and defect detection, has been
intensively studied recently. Learning to predict code changes is one of the
popular and essential investigations. Prior studies have shown that DL
techniques such as Neural Machine Translation (NMT) can benefit meaningful code
changes, including bug fixing and code refactoring. However, NMT models may
encounter bottleneck when modeling long sequences, thus are limited in
accurately predicting code changes. In this work, we design a Transformer-based
approach, considering that Transformer has proven effective in capturing
long-term dependencies. Specifically, we propose a novel model named DTrans.
For better incorporating the local structure of code, i.e., statement-level
information in this paper, DTrans is designed with dynamically relative
position encoding in the multi-head attention of Transformer. Experiments on
benchmark datasets demonstrate that DTrans can more accurately generate patches
than the state-of-the-art methods, increasing the performance by at least
5.45\%-46.57\% in terms of the exact match metric on different datasets.
Moreover, DTrans can locate the lines to change with 1.75\%-24.21\% higher
accuracy than the existing methods
- …