20 research outputs found
Enhancing Genetic Improvement Mutations Using Large Language Models
Large language models (LLMs) have been successfully applied to software
engineering tasks, including program repair. However, their application in
search-based techniques such as Genetic Improvement (GI) is still largely
unexplored. In this paper, we evaluate the use of LLMs as mutation operators
for GI to improve the search process. We expand the Gin Java GI toolkit to call
OpenAI's API to generate edits for the JCodec tool. We randomly sample the
space of edits using 5 different edit types. We find that the number of patches
passing unit tests is up to 75% higher with LLM-based edits than with standard
Insert edits. Further, we observe that the patches found with LLMs are
generally less diverse compared to standard edits. We ran GI with local search
to find runtime improvements. Although many improving patches are found by
LLM-enhanced GI, the best improving patch was found by standard GI.Comment: Accepted for publication at the Symposium on Search-Based Software
Engineering (SSBSE) 202
Reviewing Template for Artifacts Submitted to ECOOP'24
The Review Guidelines for Research Artifacts Submitted to ECOOP 2024 AE. The template used in the artifact evaluation process of the Artifact Evaluation Track at ECOOP'24
Artifact Submission Template for the Artifact Evaluation Track at ECOOP'24
The template used in the artifact evaluation process of the Artifact Evaluation Track at ECOOP'24
Artifact of GrayC: Greybox Fuzzing of Compilers and Analysers for C
The Artifact of GrayC: Greybox Fuzzing of Compilers and Analysers for C. Achieved Available, Functional, and Reusable Badges at ISSTA-AE track 2023
Enhancing Genetic Improvement Mutations Using Large Language Models
Large language models (LLMs) have been successfully applied to software engineering tasks, including program repair. However, their application in search-based techniques such as Genetic Improvement (GI) is still largely unexplored. In this paper, we evaluate the use of LLMs as mutation operators for GI to improve the search process. We expand the Gin Java GI toolkit to call OpenAI's API to generate edits for the JCodec tool. We randomly sample the space of edits using 5 different edit types. We find that the number of patches passing unit tests is up to 75% higher with LLM-based edits than with standard Insert edits. Further, we observe that the patches found with LLMs are generally less diverse compared to standard edits. We ran GI with local search to find runtime improvements. Although many improving patches are found by LLM-enhanced GI, the best improving patch was found by standard GI