On-stack replacement, distilled

On-stack replacement (OSR) is essential technology for adaptive optimization, allowing changes to code actively executing in a managed runtime. The engineering aspects of OSR are well-known among VM architects, with several implementations available to date. However, OSR is yet to be explored as a general means to transfer execution between related program versions, which can pave the road to unprecedented applications that stretch beyond VMs. We aim at filling this gap with a constructive and provably correct OSR framework, allowing a class of general-purpose transformation functions to yield a special-purpose replacement. We describe and evaluate an implementation of our technique in LLVM. As a novel application of OSR, we present a feasibility study on debugging of optimized code, showing how our techniques can be used to fix variables holding incorrect values at breakpoints due to optimizations

Populating the Peephole Optimizer of a Smart Contract Compiler

Developing compiler optimizations, especially for new, rapidly evolving smart contract languages, can be onerous and error-prone, but is especially important for smart contracts, where deployment and execution directly translate to monetary cost and which cannot change once deployed. One common optimization technique is the use of peephole optimizations, replacement rules that are applied using pattern-matching. These rules are normally constructed using human expertise, which is both time-consuming and far from systematic in exploring opportunities for optimization. In this work we propose a pipeline to automatically populate the peephole optimizer of a smart contract compiler. We apply superoptimization to an existing code base to obtain sequences of instructions, which can be replaced by cheaper, observationally equivalent instructions. We then generate peephole optimization rules by extracting the underlying patterns of these optimizations. We provide a case study of our approach and a prototype implementation for bytecode of the Ethereum Virtual Machine, the tool ppltr, which combines the superoptimizer ebso and the rule generator sorg. Then we evaluate our approach by generating and applying nearly 1k peephole optimization rules extracted from 2k optimizations obtained from deployed bytecode

LLVM 내의 Early CSE 를 위한 검증되고 신뢰할 수 있는 컴파일러 프레임워크

학위논문 (석사)-- 서울대학교 대학원 : 공과대학 컴퓨터공학부, 2018. 8. 허충길.Compiler verification is important when obtaining a high level of reliability through software verification. Compiler bugs are crucial for software verification because code that running programs are not source code but execution code. However, many C/C++ mainstream compilers, including GCC and LLVM focus on efficiency rather than reliability. Although testing is an effective method to identify bugs, it does not guarantee a high level of reliability. Various approaches have been proposed to examine compiler internal logics, but as yet none have been very successful. CRELLVM is a compiler framework that validates optimization passes in LLVM to ensure high reliability of LLVM optimizations. It is able to validate major optimizations of LLVM such as Register Promotion and Global Value Numbering. This thesis shows validation of Early CSE optimization in LLVM, using CRELLVM. For the validation, proof generation code which corresponds to Early CSE in LLVM is implemented and the proof checker has been extended. Early CSE is one of the basic optimizations in LLVM that removes the repeated computations by erasing duplicated instructions. Based on 5.40 million lines of C code benchmarks, the experiment result shows there is no mis-compilation for Early CSE, which guarantees a high level of reliability of Early CSE in the benchmarks.Chapter 1 Introduction 1 1.1 CRELLVM Framework . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Proof Checker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Early CSE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.5 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Chapter 2 Early CSE 5 2.1 Early CSE translation example . . . . . . . . . . . . . . . . . . . 5 2.2 Early CSE optimization . . . . . . . . . . . . . . . . . . . . . . . 6 2.3 Block Traversal . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Chapter 3 Proof Generation and Validation for Early CSE 12 3.1 ERHL Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.2 Proof Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.3 Proof Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Chapter 4 Results 21 4.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 4.2 Validation Result . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 4.3 Performance Result . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Chapter 5 Related Works 26 Chapter 6 Conclusion 28 요약 34Maste

CryptOpt: Verified Compilation with Random Program Search for Cryptographic Primitives

Most software domains rely on compilers to translate high-level code to multiple different machine languages, with performance not too much worse than what developers would have the patience to write directly in assembly language. However, cryptography has been an exception, where many performance-critical routines have been written directly in assembly (sometimes through metaprogramming layers). Some past work has shown how to do formal verification of that assembly, and other work has shown how to generate C code automatically along with formal proof, but with consequent performance penalties vs. the best-known assembly. We present CryptOpt, the first compilation pipeline that specializes high-level cryptographic functional programs into assembly code significantly faster than what GCC or Clang produce, with mechanized proof (in Coq) whose final theorem statement mentions little beyond the input functional program and the operational semantics of x86-64 assembly. On the optimization side, we apply randomized search through the space of assembly programs, with repeated automatic benchmarking on target CPUs. On the formal-verification side, we connect to the Fiat Cryptography framework (which translates functional programs into C-like IR code) and extend it with a new formally verified program-equivalence checker, incorporating a modest subset of known features of SMT solvers and symbolic-execution engines. The overall prototype is quite practical, e.g. producing new fastest-known implementations for the relatively new Intel i9 12G, of finite-field arithmetic for both Curve25519 (part of the TLS standard) and the Bitcoin elliptic curve secp256k1

Cost Reduction With Guarantees: Formal Reasoning Applied To Blockchain Technologies

Blockchain technologies are moving fast and their distributed nature as well as their high-stake (financial) applications make it crucial to “get things right”. Moreover, blockchain technologies often come with a high cost for maintaining blockchain infrastructure and for running applications. In this thesis formal reasoning is used for guaranteeing correctness while reducing the cost of (i) maintaining the infrastructure by optimising blockchain protocols, and (ii) running applications by optimising blockchain programs—so called smart contracts. Both have a clear cost measure: for protocols the amount of exchanged messages, and for smart contracts the monetary cost of execution. In the first result for blockchain protocols starting from a proof of correctness for an abstract blockchain consensus protocol using infinitely many messages and infinite state, a refinement proof transfers correctness to a concrete implementation of the protocol reducing the cost to finite resources. In the second result I move from a blockchain to a block graph. This block graph embeds the run of a deterministic byzantine fault tolerant protocol, thereby getting parallelism “for free” and reducing the exchanged messages to the point of omission. For blockchain programs, I optimise programs executed on the Ethereum blockchain. As a first result, I use superoptimisation and encode the search for cheaper, but observationally equivalent, program as a search problem for an automated theorem prover. Since solving this search problem is in itself expensive, my second result is an efficient encoding of the search problem. Finally for reusing found optimisations, my third results gives a framework to generate peephole optimisation rules for a smart contract compiler

Sidekick compilation with xDSL

Traditionally, compiler researchers either conduct experiments within an existing production compiler or develop their own prototype compiler; both options come with trade-offs. On one hand, prototyping in a production compiler can be cumbersome, as they are often optimized for program compilation speed at the expense of software simplicity and development speed. On the other hand, the transition from a prototype compiler to production requires significant engineering work. To bridge this gap, we introduce the concept of sidekick compiler frameworks, an approach that uses multiple frameworks that interoperate with each other by leveraging textual interchange formats and declarative descriptions of abstractions. Each such compiler framework is specialized for specific use cases, such as performance or prototyping. Abstractions are by design shared across frameworks, simplifying the transition from prototyping to production. We demonstrate this idea with xDSL, a sidekick for MLIR focused on prototyping and teaching. xDSL interoperates with MLIR through a shared textual IR and the exchange of IRs through an IR Definition Language. The benefits of sidekick compiler frameworks are evaluated by showing on three use cases how xDSL impacts their development: teaching, DSL compilation, and rewrite system prototyping. We also investigate the trade-offs that xDSL offers, and demonstrate how we simplify the transition between frameworks using the IRDL dialect. With sidekick compilation, we envision a future in which engineers minimize the cost of development by choosing a framework built for their immediate needs, and later transitioning to production with minimal overhead

Compiler fuzzing: how much does it matter?

Despite much recent interest in randomised testing (fuzzing) of compilers, the practical impact of fuzzer-found compiler bugs on real-world applications has barely been assessed. We present the first quantitative and qualitative study of the tangible impact of miscompilation bugs in a mature compiler. We follow a rigorous methodology where the bug impact over the compiled application is evaluated based on (1) whether the bug appears to trigger during compilation; (2) the extent to which generated assembly code changes syntactically due to triggering of the bug; and (3) whether such changes cause regression test suite failures, or whether we can manually find application inputs that trigger execution divergence due to such changes. The study is conducted with respect to the compilation of more than 10 million lines of C/C++ code from 309 Debian packages, using 12% of the historical and now fixed miscompilation bugs found by four state-of-the-art fuzzers in the Clang/LLVM compiler, as well as 18 bugs found by human users compiling real code or as a by-product of formal verification efforts. The results show that almost half of the fuzzer-found bugs propagate to the generated binaries for at least one package, in which case only a very small part of the binary is typically affected, yet causing two failures when running the test suites of all the impacted packages. User-reported and formal verification bugs do not exhibit a higher impact, with a lower rate of triggered bugs and one test failure. The manual analysis of a selection of the syntactic changes caused by some of our bugs (fuzzer-found and non fuzzer-found) in package assembly code, shows that either these changes have no semantic impact or that they would require very specific runtime circumstances to trigger execution divergence