189 research outputs found
Passport: Improving Automated Formal Verification Using Identifiers
Formally verifying system properties is one of the most effective ways of
improving system quality, but its high manual effort requirements often render
it prohibitively expensive. Tools that automate formal verification, by
learning from proof corpora to suggest proofs, have just begun to show their
promise. These tools are effective because of the richness of the data the
proof corpora contain. This richness comes from the stylistic conventions
followed by communities of proof developers, together with the logical systems
beneath proof assistants. However, this richness remains underexploited, with
most work thus far focusing on architecture rather than making the most of the
proof data.
In this paper, we develop Passport, a fully-automated proof-synthesis tool
that systematically explores how to most effectively exploit one aspect of that
proof data: identifiers. Passport enriches a predictive Coq model with three
new encoding mechanisms for identifiers: category vocabulary indexing, subword
sequence modeling, and path elaboration. We compare Passport to three existing
base tools which Passport can enhance: ASTactic, Tac, and Tok. In head-to-head
comparisons, Passport automatically proves 29% more theorems than the
best-performing of these base tools. Combining the three Passport-enhanced
tools automatically proves 38% more theorems than the three base tools
together, without Passport's enhancements. Finally, together, these base tools
and Passport-enhanced tools prove 45% more theorems than the combined base
tools without Passport's enhancements. Overall, our findings suggest that
modeling identifiers can play a significant role in improving proof synthesis,
leading to higher-quality software
Recommended from our members
Automating the Formal Verification of Software
Formally verified correctness is one of the most desirable properties of software systems. Despite great progress made toward verification via interactive proof assistants, such as Coq and Isabelle/HOL, such verification remains one of the most effort-intensive (and often prohibitively difficult) software development activities. Recent work has created tools that automatically synthesize proofs either through reasoning using precomputed facts or using machine learning to model proofs and then perform biased search through the proof space. However, models in existing tools fail to capture the richness present in proofs, such as the information the programmer has access to when writing proofs and the natural language contained within variable names. Furthermore, these prior models do not make use of variations in the learning process and advances in large language models.
In this dissertation, I develop tools to improve proof synthesis and to enable fully automating more verification. I first present TacTok, a proof-synthesis tool that models proofs using both the partial proof written thus far and the semantics of the proof state. I then present Diva, a proof-synthesis tool that controls the learning process to produce a diverse set of models and, due to the unique nature of proof synthesis (the existence of the theorem prover, an oracle that infallibly judges a proof’s correctness), efficiently combines these models to improve the overall proving power. I then present Passport, a proof-synthesis tool that systematically explores different ways of encoding identifiers in proofs to improve synthesis. Finally, I present Baldur, a proof-synthesis tool that uses transformer-based pretrained large language models fine-tuned on proofs to generate and repair whole proofs at once, rather than one step at a time.
This dissertation contributes new ideas for improving automated proof synthesis and empirically demonstrates that the improvement is significant on large benchmarks consisting of open-source software projects
Row and Bounded Polymorphism via Disjoint Polymorphism
Polymorphism and subtyping are important features in mainstream OO languages. The most common way to integrate the two is via ?_{< :} style bounded quantification. A closely related mechanism is row polymorphism, which provides an alternative to subtyping, while still enabling many of the same applications. Yet another approach is to have type systems with intersection types and polymorphism. A recent addition to this design space are calculi with disjoint intersection types and disjoint polymorphism. With all these alternatives it is natural to wonder how they are related.
This paper provides an answer to this question. We show that disjoint polymorphism can recover forms of both row polymorphism and bounded polymorphism, while retaining key desirable properties, such as type-safety and decidability. Furthermore, we identify the extra power of disjoint polymorphism which enables additional features that cannot be easily encoded in calculi with row polymorphism or bounded quantification alone. Ultimately we expect that our work is useful to inform language designers about the expressive power of those common features, and to simplify implementations and metatheory of feature-rich languages with polymorphism and subtyping
αCheck: a mechanized metatheory model-checker
The problem of mechanically formalizing and proving metatheoretic properties
of programming language calculi, type systems, operational semantics, and
related formal systems has received considerable attention recently. However,
the dual problem of searching for errors in such formalizations has attracted
comparatively little attention. In this article, we present Check, a
bounded model-checker for metatheoretic properties of formal systems specified
using nominal logic. In contrast to the current state of the art for metatheory
verification, our approach is fully automatic, does not require expertise in
theorem proving on the part of the user, and produces counterexamples in the
case that a flaw is detected. We present two implementations of this technique,
one based on negation-as-failure and one based on negation elimination, along
with experimental results showing that these techniques are fast enough to be
used interactively to debug systems as they are developed.Comment: Under consideration for publication in Theory and Practice of Logic
Programming (TPLP
Predicate Abstraction for Linked Data Structures
We present Alias Refinement Types (ART), a new approach to the verification
of correctness properties of linked data structures. While there are many
techniques for checking that a heap-manipulating program adheres to its
specification, they often require that the programmer annotate the behavior of
each procedure, for example, in the form of loop invariants and pre- and
post-conditions. Predicate abstraction would be an attractive abstract domain
for performing invariant inference, existing techniques are not able to reason
about the heap with enough precision to verify functional properties of data
structure manipulating programs. In this paper, we propose a technique that
lifts predicate abstraction to the heap by factoring the analysis of data
structures into two orthogonal components: (1) Alias Types, which reason about
the physical shape of heap structures, and (2) Refinement Types, which use
simple predicates from an SMT decidable theory to capture the logical or
semantic properties of the structures. We prove ART sound by translating types
into separation logic assertions, thus translating typing derivations in ART
into separation logic proofs. We evaluate ART by implementing a tool that
performs type inference for an imperative language, and empirically show, using
a suite of data-structure benchmarks, that ART requires only 21% of the
annotations needed by other state-of-the-art verification techniques
- …