2,828 research outputs found
Towards Smart Hybrid Fuzzing for Smart Contracts
Smart contracts are Turing-complete programs that are executed across a
blockchain network. Unlike traditional programs, once deployed they cannot be
modified. As smart contracts become more popular and carry more value, they
become more of an interesting target for attackers. In recent years, smart
contracts suffered major exploits, costing millions of dollars, due to
programming errors. As a result, a variety of tools for detecting bugs has been
proposed. However, majority of these tools often yield many false positives due
to over-approximation or poor code coverage due to complex path constraints.
Fuzzing or fuzz testing is a popular and effective software testing technique.
However, traditional fuzzers tend to be more effective towards finding shallow
bugs and less effective in finding bugs that lie deeper in the execution. In
this work, we present CONFUZZIUS, a hybrid fuzzer that combines evolutionary
fuzzing with constraint solving in order to execute more code and find more
bugs in smart contracts. Evolutionary fuzzing is used to exercise shallow parts
of a smart contract, while constraint solving is used to generate inputs which
satisfy complex conditions that prevent the evolutionary fuzzing from exploring
deeper paths. Moreover, we use data dependency analysis to efficiently generate
sequences of transactions, that create specific contract states in which bugs
may be hidden. We evaluate the effectiveness of our fuzzing strategy, by
comparing CONFUZZIUS with state-of-the-art symbolic execution tools and
fuzzers. Our evaluation shows that our hybrid fuzzing approach produces
significantly better results than state-of-the-art symbolic execution tools and
fuzzers
Harvey: A Greybox Fuzzer for Smart Contracts
We present Harvey, an industrial greybox fuzzer for smart contracts, which
are programs managing accounts on a blockchain. Greybox fuzzing is a
lightweight test-generation approach that effectively detects bugs and security
vulnerabilities. However, greybox fuzzers randomly mutate program inputs to
exercise new paths; this makes it challenging to cover code that is guarded by
narrow checks, which are satisfied by no more than a few input values.
Moreover, most real-world smart contracts transition through many different
states during their lifetime, e.g., for every bid in an auction. To explore
these states and thereby detect deep vulnerabilities, a greybox fuzzer would
need to generate sequences of contract transactions, e.g., by creating bids
from multiple users, while at the same time keeping the search space and test
suite tractable. In this experience paper, we explain how Harvey alleviates
both challenges with two key fuzzing techniques and distill the main lessons
learned. First, Harvey extends standard greybox fuzzing with a method for
predicting new inputs that are more likely to cover new paths or reveal
vulnerabilities in smart contracts. Second, it fuzzes transaction sequences in
a targeted and demand-driven way. We have evaluated our approach on 27
real-world contracts. Our experiments show that the underlying techniques
significantly increase Harvey's effectiveness in achieving high coverage and
detecting vulnerabilities, in most cases orders-of-magnitude faster; they also
reveal new insights about contract code.Comment: arXiv admin note: substantial text overlap with arXiv:1807.0787
Targeted Greybox Fuzzing with Static Lookahead Analysis
Automatic test generation typically aims to generate inputs that explore new
paths in the program under test in order to find bugs. Existing work has,
therefore, focused on guiding the exploration toward program parts that are
more likely to contain bugs by using an offline static analysis.
In this paper, we introduce a novel technique for targeted greybox fuzzing
using an online static analysis that guides the fuzzer toward a set of target
locations, for instance, located in recently modified parts of the program.
This is achieved by first semantically analyzing each program path that is
explored by an input in the fuzzer's test suite. The results of this analysis
are then used to control the fuzzer's specialized power schedule, which
determines how often to fuzz inputs from the test suite. We implemented our
technique by extending a state-of-the-art, industrial fuzzer for Ethereum smart
contracts and evaluate its effectiveness on 27 real-world benchmarks. Using an
online analysis is particularly suitable for the domain of smart contracts
since it does not require any code instrumentation---instrumentation to
contracts changes their semantics. Our experiments show that targeted fuzzing
significantly outperforms standard greybox fuzzing for reaching 83% of the
challenging target locations (up to 14x of median speed-up)
Deviant: A Mutation Testing Tool for Solidity Smart Contracts
Blockchain in recent years has exploded in popularity with Ethereum being one of the leading blockchain platforms. Solidity is a widely used scripting language for creating smart contracts in Ethereum applications. Quality assurance in Solidity contracts is of critical importance because bugs or vulnerabilities can lead to a considerable loss of financial assets. However, it is unclear what level of quality assurance is provided in many of these applications.
Mutation testing is the process of intentionally injecting faults into a target program and then running the provided test suite against the various injected faults. Mutation testing is used to evaluate the effectiveness of a test suite, measuring the test suite’s capability of covering certain types of faults. This thesis presents Deviant, the first implementation of a mutation testing tool for Solidity smart contracts. Deviant implements mutation operators that cover the unique features of Solidity according to our constructed fault model, in addition to traditional mutation operators that exist for other programming languages.
Deviant has been applied to five open-source Solidity projects: MetaCoin [30], MultiSigWallet [31], Alice [29], aragonOS [32], and OpenZeppelin [33]. Experimental results show that the provided test suites result in low mutation scores. These results indicate that the provided tests cannot ensure high-level assurance of code quality. Such evaluation results offer important guidelines for Solidity developers to implement more effective tests in order to deliver trustworthy code and reduce the risk of financial loss
EF/CF: High Performance Smart Contract Fuzzing for Exploit Generation
Smart contracts are increasingly being used to manage large numbers of
high-value cryptocurrency accounts. There is a strong demand for automated,
efficient, and comprehensive methods to detect security vulnerabilities in a
given contract. While the literature features a plethora of analysis methods
for smart contracts, the existing proposals do not address the increasing
complexity of contracts. Existing analysis tools suffer from false alarms and
missed bugs in today's smart contracts that are increasingly defined by
complexity and interdependencies. To scale accurate analysis to modern smart
contracts, we introduce EF/CF, a high-performance fuzzer for Ethereum smart
contracts. In contrast to previous work, EF/CF efficiently and accurately
models complex smart contract interactions, such as reentrancy and
cross-contract interactions, at a very high fuzzing throughput rate. To achieve
this, EF/CF transpiles smart contract bytecode into native C++ code, thereby
enabling the reuse of existing, optimized fuzzing toolchains. Furthermore,
EF/CF increases fuzzing efficiency by employing a structure-aware mutation
engine for smart contract transaction sequences and using a contract's ABI to
generate valid transaction inputs. In a comprehensive evaluation, we show that
EF/CF scales better -- without compromising accuracy -- to complex contracts
compared to state-of-the-art approaches, including other fuzzers,
symbolic/concolic execution, and hybrid approaches. Moreover, we show that
EF/CF can automatically generate transaction sequences that exploit reentrancy
bugs to steal Ether.Comment: To be published at Euro S&P 202
Do you still need a manual smart contract audit?
We investigate the feasibility of employing large language models (LLMs) for
conducting the security audit of smart contracts, a traditionally
time-consuming and costly process. Our research focuses on the optimization of
prompt engineering for enhanced security analysis, and we evaluate the
performance and accuracy of LLMs using a benchmark dataset comprising 52
Decentralized Finance (DeFi) smart contracts that have previously been
compromised.
Our findings reveal that, when applied to vulnerable contracts, both GPT-4
and Claude models correctly identify the vulnerability type in 40% of the
cases. However, these models also demonstrate a high false positive rate,
necessitating continued involvement from manual auditors. The LLMs tested
outperform a random model by 20% in terms of F1-score.
To ensure the integrity of our study, we conduct mutation testing on five
newly developed and ostensibly secure smart contracts, into which we manually
insert two and 15 vulnerabilities each. This testing yielded a remarkable
best-case 78.7% true positive rate for the GPT-4-32k model. We tested both,
asking the models to perform a binary classification on whether a contract is
vulnerable, and a non-binary prompt. We also examined the influence of model
temperature variations and context length on the LLM's performance.
Despite the potential for many further enhancements, this work lays the
groundwork for a more efficient and economical approach to smart contract
security audits
- …