27 research outputs found
Learning to Prove with Tactics
We implement a automated tactical prover TacticToe on top of the HOL4
interactive theorem prover. TacticToe learns from human proofs which
mathematical technique is suitable in each proof situation. This knowledge is
then used in a Monte Carlo tree search algorithm to explore promising
tactic-level proof paths. On a single CPU, with a time limit of 60 seconds,
TacticToe proves 66.4 percent of the 7164 theorems in HOL4's standard library,
whereas E prover with auto-schedule solves 34.5 percent. The success rate rises
to 69.0 percent by combining the results of TacticToe and E prover
Learning to Prove Theorems by Learning to Generate Theorems
We consider the task of automated theorem proving, a key AI task. Deep
learning has shown promise for training theorem provers, but there are limited
human-written theorems and proofs available for supervised learning. To address
this limitation, we propose to learn a neural generator that automatically
synthesizes theorems and proofs for the purpose of training a theorem prover.
Experiments on real-world tasks demonstrate that synthetic data from our
approach improves the theorem prover and advances the state of the art of
automated theorem proving in Metamath. Code is available at
https://github.com/princeton-vl/MetaGen
GamePad: A Learning Environment for Theorem Proving
In this paper, we introduce a system called GamePad that can be used to
explore the application of machine learning methods to theorem proving in the
Coq proof assistant. Interactive theorem provers such as Coq enable users to
construct machine-checkable proofs in a step-by-step manner. Hence, they
provide an opportunity to explore theorem proving with human supervision. We
use GamePad to synthesize proofs for a simple algebraic rewrite problem and
train baseline models for a formalization of the Feit-Thompson theorem. We
address position evaluation (i.e., predict the number of proof steps left) and
tactic prediction (i.e., predict the next proof step) tasks, which arise
naturally in tactic-based theorem proving
The Role of Entropy in Guiding a Connection Prover
In this work we study how to learn good algorithms for selecting reasoning
steps in theorem proving. We explore this in the connection tableau calculus
implemented by leanCoP where the partial tableau provides a clean and compact
notion of a state to which a limited number of inferences can be applied. We
start by incorporating a state-of-the-art learning algorithm -- a graph neural
network (GNN) -- into the plCoP theorem prover. Then we use it to observe the
system's behaviour in a reinforcement learning setting, i.e., when learning
inference guidance from successful Monte-Carlo tree searches on many problems.
Despite its better pattern matching capability, the GNN initially performs
worse than a simpler previously used learning algorithm. We observe that the
simpler algorithm is less confident, i.e., its recommendations have higher
entropy. This leads us to explore how the entropy of the inference selection
implemented via the neural network influences the proof search. This is related
to research in human decision-making under uncertainty, and in particular the
probability matching theory. Our main result shows that a proper entropy
regularisation, i.e., training the GNN not to be overconfident, greatly
improves plCoP's performance on a large mathematical corpus
Self-Learned Formula Synthesis in Set Theory
A reinforcement learning algorithm accomplishes the task of synthesizing a
set-theoretical formula that evaluates to given truth values for given
assignments
Hammering Mizar by Learning Clause Guidance
We describe a very large improvement of existing hammer-style proof
automation over large ITP libraries by combining learning and theorem proving.
In particular, we have integrated state-of-the-art machine learners into the E
automated theorem prover, and developed methods that allow learning and
efficient internal guidance of E over the whole Mizar library. The resulting
trained system improves the real-time performance of E on the Mizar library by
70% in a single-strategy setting.Comment: arXiv admin note: substantial text overlap with arXiv:1903.0318
Online Machine Learning Techniques for Coq: A Comparison
We present a comparison of several online machine learning techniques for
tactical learning and proving in the Coq proof assistant. This work builds on
top of Tactician, a plugin for Coq that learns from proofs written by the user
to synthesize new proofs. Learning happens in an online manner, meaning that
Tactician's machine learning model is updated immediately every time the user
performs a step in an interactive proof. This has important advantages compared
to the more studied offline learning systems: (1) it provides the user with a
seamless, interactive experience with Tactician and, (2) it takes advantage of
locality of proof similarity, which means that proofs similar to the current
proof are likely to be found close by. We implement two online methods, namely
approximate k-nearest neighbors based on locality sensitive hashing forests and
random decision forests. Additionally, we conduct experiments with gradient
boosted trees in an offline setting using XGBoost. We compare the relative
performance of Tactician using these three learning methods on Coq's standard
library.Comment: Intelligent Computer Mathematics 14th International Conference, CICM
202
ProofWatch: Watchlist Guidance for Large Theories in E
Watchlist (also hint list) is a mechanism that allows related proofs to guide
a proof search for a new conjecture. This mechanism has been used with the
Otter and Prover9 theorem provers, both for interactive formalizations and for
human-assisted proving of open conjectures in small theories. In this work we
explore the use of watchlists in large theories coming from first-order
translations of large ITP libraries, aiming at improving hammer-style
automation by smarter internal guidance of the ATP systems. In particular, we
(i) design watchlist-based clause evaluation heuristics inside the E ATP
system, and (ii) develop new proof guiding algorithms that load many previous
proofs inside the ATP and focus the proof search using a dynamically updated
notion of proof matching. The methods are evaluated on a large set of problems
coming from the Mizar library, showing significant improvement of E's standard
portfolio of strategies, and also of the previous best set of strategies
invented for Mizar by evolutionary methods.Comment: 19 pages, 10 tables, submitted to ITP 2018 at FLO
Mathematical Reasoning in Latent Space
We design and conduct a simple experiment to study whether neural networks
can perform several steps of approximate reasoning in a fixed dimensional
latent space. The set of rewrites (i.e. transformations) that can be
successfully performed on a statement represents essential semantic features of
the statement. We can compress this information by embedding the formula in a
vector space, such that the vector associated with a statement can be used to
predict whether a statement can be rewritten by other theorems. Predicting the
embedding of a formula generated by some rewrite rule is naturally viewed as
approximate reasoning in the latent space. In order to measure the
effectiveness of this reasoning, we perform approximate deduction sequences in
the latent space and use the resulting embedding to inform the semantic
features of the corresponding formal statement (which is obtained by performing
the corresponding rewrite sequence using real formulas). Our experiments show
that graph neural networks can make non-trivial predictions about the
rewrite-success of statements, even when they propagate predicted latent
representations for several steps. Since our corpus of mathematical formulas
includes a wide variety of mathematical disciplines, this experiment is a
strong indicator for the feasibility of deduction in latent space in general
ENIGMA-NG: Efficient Neural and Gradient-Boosted Inference Guidance for E
We describe an efficient implementation of clause guidance in
saturation-based automated theorem provers extending the ENIGMA approach.
Unlike in the first ENIGMA implementation where fast linear classifier is
trained and used together with manually engineered features, we have started to
experiment with more sophisticated state-of-the-art machine learning methods
such as gradient boosted trees and recursive neural networks. In particular the
latter approach poses challenges in terms of efficiency of clause evaluation,
however, we show that deep integration of the neural evaluation with the ATP
data-structures can largely amortize this cost and lead to competitive
real-time results. Both methods are evaluated on a large dataset of theorem
proving problems and compared with the previous approaches. The resulting
methods improve on the manually designed clause guidance, providing the first
practically convincing application of gradient-boosted and neural clause
guidance in saturation-style automated theorem provers