91 research outputs found
Identifying Patch Correctness in Test-Based Program Repair
Test-based automatic program repair has attracted a lot of attention in
recent years. However, the test suites in practice are often too weak to
guarantee correctness and existing approaches often generate a large number of
incorrect patches.
To reduce the number of incorrect patches generated, we propose a novel
approach that heuristically determines the correctness of the generated
patches. The core idea is to exploit the behavior similarity of test case
executions. The passing tests on original and patched programs are likely to
behave similarly while the failing tests on original and patched programs are
likely to behave differently. Also, if two tests exhibit similar runtime
behavior, the two tests are likely to have the same test results. Based on
these observations, we generate new test inputs to enhance the test suites and
use their behavior similarity to determine patch correctness.
Our approach is evaluated on a dataset consisting of 139 patches generated
from existing program repair systems including jGenProg, Nopol, jKali, ACS and
HDRepair. Our approach successfully prevented 56.3\% of the incorrect patches
to be generated, without blocking any correct patches.Comment: ICSE 201
Automating Method Naming with Context-Aware Prompt-Tuning
Method names are crucial to program comprehension and maintenance. Recently,
many approaches have been proposed to automatically recommend method names and
detect inconsistent names. Despite promising, their results are still
sub-optimal considering the three following drawbacks: 1) These models are
mostly trained from scratch, learning two different objectives simultaneously.
The misalignment between two objectives will negatively affect training
efficiency and model performance. 2) The enclosing class context is not fully
exploited, making it difficult to learn the abstract function of the method. 3)
Current method name consistency checking methods follow a generate-then-compare
process, which restricts the accuracy as they highly rely on the quality of
generated names and face difficulty measuring the semantic consistency.
In this paper, we propose an approach named AUMENA to AUtomate MEthod NAming
tasks with context-aware prompt-tuning. Unlike existing deep learning based
approaches, our model first learns the contextualized representation(i.e.,
class attributes) of PL and NL through the pre-training model, then fully
exploits the capacity and knowledge of large language model with prompt-tuning
to precisely detect inconsistent method names and recommend more accurate
names. To better identify semantically consistent names, we model the method
name consistency checking task as a two-class classification problem, avoiding
the limitation of previous similarity-based consistency checking approaches.
The experimental results reflect that AUMENA scores 68.6%, 72.0%, 73.6%, 84.7%
on four datasets of method name recommendation, surpassing the state-of-the-art
baseline by 8.5%, 18.4%, 11.0%, 12.0%, respectively. And our approach scores
80.8% accuracy on method name consistency checking, reaching an 5.5%
outperformance. All data and trained models are publicly available.Comment: Accepted by ICPC-202
How Early Participation Determines Long-Term Sustained Activity in GitHub Projects?
Although the open source model bears many advantages in software development,
open source projects are always hard to sustain. Previous research on open
source sustainability mainly focuses on projects that have already reached a
certain level of maturity (e.g., with communities, releases, and downstream
projects). However, limited attention is paid to the development of
(sustainable) open source projects in their infancy, and we believe an
understanding of early sustainability determinants is crucial for project
initiators, incubators, newcomers, and users.
In this paper, we aim to explore the relationship between early participation
factors and long-term project sustainability. We leverage a novel methodology
combining the Blumberg model of performance and machine learning to predict the
sustainability of 290,255 GitHub projects. Specificially, we train an XGBoost
model based on early participation (first three months of activity) in 290,255
GitHub projects and we interpret the model using LIME. We quantitatively show
that early participants have a positive effect on project's future sustained
activity if they have prior experience in OSS project incubation and
demonstrate concentrated focus and steady commitment. Participation from
non-code contributors and detailed contribution documentation also promote
project's sustained activity. Compared with individual projects, building a
community that consists of more experienced core developers and more active
peripheral developers is important for organizational projects. This study
provides unique insights into the incubation and recognition of sustainable
open source projects, and our interpretable prediction approach can also offer
guidance to open source project initiators and newcomers.Comment: The 31st ACM Joint European Software Engineering Conference and
Symposium on the Foundations of Software Engineering (ESEC/FSE 2023
Opinion Mining for Software Development: A Systematic Literature Review
Opinion mining, sometimes referred to as sentiment analysis, has gained increasing attention in software engineering (SE) studies.
SE researchers have applied opinion mining techniques in various contexts, such as identifying developers’ emotions expressed in
code comments and extracting users’ critics toward mobile apps. Given the large amount of relevant studies available, it can take
considerable time for researchers and developers to figure out which approaches they can adopt in their own studies and what perils
these approaches entail.
We conducted a systematic literature review involving 185 papers. More specifically, we present 1) well-defined categories of opinion
mining-related software development activities, 2) available opinion mining approaches, whether they are evaluated when adopted in
other studies, and how their performance is compared, 3) available datasets for performance evaluation and tool customization, and 4)
concerns or limitations SE researchers might need to take into account when applying/customizing these opinion mining techniques.
The results of our study serve as references to choose suitable opinion mining tools for software development activities, and provide
critical insights for the further development of opinion mining techniques in the SE domain
- …