4,011 research outputs found
Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair
During Automated Program Repair (APR), it can be challenging to synthesize
correct patches for real-world systems in general-purpose programming
languages. Recent Large Language Models (LLMs) have been shown to be helpful
"copilots" in assisting developers with various coding tasks, and have also
been directly applied for patch synthesis. However, most LLMs treat programs as
sequences of tokens, meaning that they are ignorant of the underlying semantics
constraints of the target programming language. This results in plenty of
statically invalid generated patches, impeding the practicality of the
technique. Therefore, we propose Repilot, a framework to further copilot the AI
"copilots" (i.e., LLMs) by synthesizing more valid patches during the repair
process. Our key insight is that many LLMs produce outputs autoregressively
(i.e., token by token), resembling human writing programs, which can be
significantly boosted and guided through a Completion Engine. Repilot
synergistically synthesizes a candidate patch through the interaction between
an LLM and a Completion Engine, which 1) prunes away infeasible tokens
suggested by the LLM and 2) proactively completes the token based on the
suggestions provided by the Completion Engine. Our evaluation on a subset of
the widely-used Defects4j 1.2 and 2.0 datasets shows that Repilot fixes 66 and
50 bugs, respectively, surpassing the best-performing baseline by 14 and 16
bugs fixed. More importantly, Repilot is capable of producing more valid and
correct patches than the base LLM when given the same generation budget
Simplifying Deep-Learning-Based Model for Code Search
To accelerate software development, developers frequently search and reuse
existing code snippets from a large-scale codebase, e.g., GitHub. Over the
years, researchers proposed many information retrieval (IR) based models for
code search, which match keywords in query with code text. But they fail to
connect the semantic gap between query and code. To conquer this challenge, Gu
et al. proposed a deep-learning-based model named DeepCS. It jointly embeds
method code and natural language description into a shared vector space, where
methods related to a natural language query are retrieved according to their
vector similarities. However, DeepCS' working process is complicated and
time-consuming. To overcome this issue, we proposed a simplified model
CodeMatcher that leverages the IR technique but maintains many features in
DeepCS. Generally, CodeMatcher combines query keywords with the original order,
performs a fuzzy search on name and body strings of methods, and returned the
best-matched methods with the longer sequence of used keywords. We verified its
effectiveness on a large-scale codebase with about 41k repositories.
Experimental results showed the simplified model CodeMatcher outperforms DeepCS
by 97% in terms of MRR (a widely used accuracy measure for code search), and it
is over 66 times faster than DeepCS. Besides, comparing with the
state-of-the-art IR-based model CodeHow, CodeMatcher also improves the MRR by
73%. We also observed that: fusing the advantages of IR-based and
deep-learning-based models is promising because they compensate with each other
by nature; improving the quality of method naming helps code search, since
method name plays an important role in connecting query and code
Exploiting Input Sanitization for Regex Denial of Service
Web services use server-side input sanitization to guard against harmful input. Some web services publish their sanitization logic to make their client interface more usable, e.g., allowing clients to debug invalid requests locally. However, this usability practice poses a security risk. Specifically, services may share the regexes they use to sanitize input strings — and regex-based denial of service (ReDoS) is an emerging threat. Although prominent service outages caused by ReDoS have spurred interest in this topic, we know little about the degree to which live web services are vulnerable to ReDoS.
In this paper, we conduct the first black-box study measuring the extent of ReDoS vulnerabilities in live web services. We apply the Consistent Sanitization Assumption: that client-side sanitization logic, including regexes, is consistent with the sanitization logic on the server-side. We identify a service’s regex-based input sanitization in its HTML forms or its API, find vulnerable regexes among these regexes, craft ReDoS probes, and pinpoint vulnerabilities. We analyzed the HTML forms of 1,000 services and the APIs of 475 services. Of these, 355 services publish regexes; 17 services publish unsafe regexes; and 6 services are vulnerable to ReDoS through their APIs (6 domains; 15 subdomains). Both Microsoft and Amazon Web Services patched their web services as a result of our disclosure. Since these vulnerabilities were from API specifications, not HTML forms, we proposed a ReDoS defense for a popular API validation library, and our patch has been merged. To summarize: in client-visible sanitization logic, some web services advertise ReDoS vulnerabilities in plain sight. Our results motivate short-term patches and long-term fundamental solutions
- …