333 research outputs found
Simplifying Deep-Learning-Based Model for Code Search
To accelerate software development, developers frequently search and reuse
existing code snippets from a large-scale codebase, e.g., GitHub. Over the
years, researchers proposed many information retrieval (IR) based models for
code search, which match keywords in query with code text. But they fail to
connect the semantic gap between query and code. To conquer this challenge, Gu
et al. proposed a deep-learning-based model named DeepCS. It jointly embeds
method code and natural language description into a shared vector space, where
methods related to a natural language query are retrieved according to their
vector similarities. However, DeepCS' working process is complicated and
time-consuming. To overcome this issue, we proposed a simplified model
CodeMatcher that leverages the IR technique but maintains many features in
DeepCS. Generally, CodeMatcher combines query keywords with the original order,
performs a fuzzy search on name and body strings of methods, and returned the
best-matched methods with the longer sequence of used keywords. We verified its
effectiveness on a large-scale codebase with about 41k repositories.
Experimental results showed the simplified model CodeMatcher outperforms DeepCS
by 97% in terms of MRR (a widely used accuracy measure for code search), and it
is over 66 times faster than DeepCS. Besides, comparing with the
state-of-the-art IR-based model CodeHow, CodeMatcher also improves the MRR by
73%. We also observed that: fusing the advantages of IR-based and
deep-learning-based models is promising because they compensate with each other
by nature; improving the quality of method naming helps code search, since
method name plays an important role in connecting query and code
Improving Code Example Recommendations on Informal Documentation Using BERT and Query-Aware LSH: A Comparative Study
Our research investigates the recommendation of code examples to aid software
developers, a practice that saves developers significant time by providing
ready-to-use code snippets. The focus of our study is Stack Overflow, a
commonly used resource for coding discussions and solutions, particularly in
the context of the Java programming language.
We applied BERT, a powerful Large Language Model (LLM) that enables us to
transform code examples into numerical vectors by extracting their semantic
information. Once these numerical representations are prepared, we identify
Approximate Nearest Neighbors (ANN) using Locality-Sensitive Hashing (LSH). Our
research employed two variants of LSH: Random Hyperplane-based LSH and
Query-Aware LSH. We rigorously compared these two approaches across four
parameters: HitRate, Mean Reciprocal Rank (MRR), Average Execution Time, and
Relevance.
Our study revealed that the Query-Aware (QA) approach showed superior
performance over the Random Hyperplane-based (RH) method. Specifically, it
exhibited a notable improvement of 20% to 35% in HitRate for query pairs
compared to the RH approach. Furthermore, the QA approach proved significantly
more time-efficient, with its speed in creating hashing tables and assigning
data samples to buckets being at least four times faster. It can return code
examples within milliseconds, whereas the RH approach typically requires
several seconds to recommend code examples. Due to the superior performance of
the QA approach, we tested it against PostFinder and FaCoY, the
state-of-the-art baselines. Our QA method showed comparable efficiency proving
its potential for effective code recommendation
Are Code Examples on an Online Q&A Forum Reliable?
Programmers often consult an online Q&A forum such as Stack Overflow to learn new APIs. This paper presents an empirical study on the prevalence and severity of API misuse on Stack Overflow. To reduce manual assessment effort, we design ExampleCheck, an API usage mining framework that extracts patterns from over 380K Java repositories on GitHub and subsequently reports potential API usage violations in Stack Overflow posts. We analyze 217,818 Stack Overflow posts using ExampleCheck and find that 31% may have potential API usage violations that could produce unexpected behavior such as program crashes and resource leaks. Such API misuse is caused by three main reasons---missing control constructs, missing or incorrect order of API calls, and incorrect guard conditions. Even the posts that are accepted as correct answers or upvoted by other programmers are not necessarily more reliable than other posts in terms of API misuse. This study result calls for a new approach to augment Stack Overflow with alternative API usage details that are not typically shown in curated examples
- …