13 research outputs found
Enhancing Genetic Improvement Mutations Using Large Language Models
Large language models (LLMs) have been successfully applied to software
engineering tasks, including program repair. However, their application in
search-based techniques such as Genetic Improvement (GI) is still largely
unexplored. In this paper, we evaluate the use of LLMs as mutation operators
for GI to improve the search process. We expand the Gin Java GI toolkit to call
OpenAI's API to generate edits for the JCodec tool. We randomly sample the
space of edits using 5 different edit types. We find that the number of patches
passing unit tests is up to 75% higher with LLM-based edits than with standard
Insert edits. Further, we observe that the patches found with LLMs are
generally less diverse compared to standard edits. We ran GI with local search
to find runtime improvements. Although many improving patches are found by
LLM-enhanced GI, the best improving patch was found by standard GI.Comment: Accepted for publication at the Symposium on Search-Based Software
Engineering (SSBSE) 202
SearchGEM5: Towards Reliable gem5 with Search Based Software Testing and Large Language Models
Artifact of GrayC: Greybox Fuzzing of Compilers and Analysers for C
The Artifact of GrayC: Greybox Fuzzing of Compilers and Analysers for C. Achieved Available, Functional, and Reusable Badges at ISSTA-AE track 2023
Enhancing Genetic Improvement Mutations Using Large Language Models
Large language models (LLMs) have been successfully applied to software engineering tasks, including program repair. However, their application in search-based techniques such as Genetic Improvement (GI) is still largely unexplored. In this paper, we evaluate the use of LLMs as mutation operators for GI to improve the search process. We expand the Gin Java GI toolkit to call OpenAI's API to generate edits for the JCodec tool. We randomly sample the space of edits using 5 different edit types. We find that the number of patches passing unit tests is up to 75% higher with LLM-based edits than with standard Insert edits. Further, we observe that the patches found with LLMs are generally less diverse compared to standard edits. We ran GI with local search to find runtime improvements. Although many improving patches are found by LLM-enhanced GI, the best improving patch was found by standard GI
Enhancing Genetic Improvement Mutations Using Large Language Models
Large language models (LLMs) have been successfully applied to software engineering tasks, including program repair. However, their application in search-based techniques such as Genetic Improvement (GI) is still largely unexplored. In this paper, we evaluate the use of LLMs as mutation operators for GI to improve the search process. We expand the Gin Java GI toolkit to call OpenAI's API to generate edits for the JCodec tool. We randomly sample the space of edits using 5 different edit types. We find that the number of patches passing unit tests is up to 75% higher with LLM-based edits than with standard Insert edits. Further, we observe that the patches found with LLMs are generally less diverse compared to standard edits. We ran GI with local search to find runtime improvements. Although many improving patches are found by LLM-enhanced GI, the best improving patch was found by standard GI