592 research outputs found
Learning Heuristics for Quantified Boolean Formulas through Deep Reinforcement Learning
We demonstrate how to learn efficient heuristics for automated reasoning
algorithms for quantified Boolean formulas through deep reinforcement learning.
We focus on a backtracking search algorithm, which can already solve formulas
of impressive size - up to hundreds of thousands of variables. The main
challenge is to find a representation of these formulas that lends itself to
making predictions in a scalable way. For a family of challenging problems, we
learned a heuristic that solves significantly more formulas compared to the
existing handwritten heuristics
Can -Learning with Graph Networks Learn a Generalizable Branching Heuristic for a SAT Solver?
We present Graph--SAT, a branching heuristic for a Boolean SAT solver
trained with value-based reinforcement learning (RL) using Graph Neural
Networks for function approximation. Solvers using Graph--SAT are complete
SAT solvers that either provide a satisfying assignment or proof of
unsatisfiability, which is required for many SAT applications. The branching
heuristics commonly used in SAT solvers make poor decisions during their
warm-up period, whereas Graph--SAT is trained to examine the structure of
the particular problem instance to make better decisions early in the search.
Training Graph--SAT is data efficient and does not require elaborate dataset
preparation or feature engineering. We train Graph--SAT using RL interfacing
with MiniSat solver and show that Graph--SAT can reduce the number of
iterations required to solve SAT problems by 2-3X. Furthermore, it generalizes
to unsatisfiable SAT instances, as well as to problems with 5X more variables
than it was trained on. We show that for larger problems, reductions in the
number of iterations lead to wall clock time reductions, the ultimate goal when
designing heuristics. We also show positive zero-shot transfer behavior when
testing Graph--SAT on a task family different from that used for training.
While more work is needed to apply Graph--SAT to reduce wall clock time in
modern SAT solving settings, it is a compelling proof-of-concept showing that
RL equipped with Graph Neural Networks can learn a generalizable branching
heuristic for SAT search.Comment: Camera-ready for NeurIPS 202
Unveiling the Limits of Learned Local Search Heuristics: Are You the Mightiest of the Meek?
In recent years, combining neural networks with local search heuristics has
become popular in the field of combinatorial optimization. Despite its
considerable computational demands, this approach has exhibited promising
outcomes with minimal manual engineering. However, we have identified three
critical limitations in the empirical evaluation of these integration attempts.
Firstly, instances with moderate complexity and weak baselines pose a challenge
in accurately evaluating the effectiveness of learning-based approaches.
Secondly, the absence of an ablation study makes it difficult to quantify and
attribute improvements accurately to the deep learning architecture. Lastly,
the generalization of learned heuristics across diverse distributions remains
underexplored. In this study, we conduct a comprehensive investigation into
these identified limitations. Surprisingly, we demonstrate that a simple
learned heuristic based on Tabu Search surpasses state-of-the-art (SOTA)
learned heuristics in terms of performance and generalizability. Our findings
challenge prevailing assumptions and open up exciting avenues for future
research and innovation in combinatorial optimization
MedleySolver: Online SMT Algorithm Selection
Satisfiability modulo theories (SMT) solvers implement a wide range of optimizations that are often tailored to a particular class of problems, and that differ significantly between solvers. As a result, one solver may solve a query quickly while another might be flummoxed completely. Predicting the performance of a given solver is difficult for users of SMT-driven applications, particularly when the problems they have to solve do not fall neatly into a well-understood category. In this paper, we propose an online algorithm selection framework for SMT called MedleySolver that predicts the relative performances of a set of SMT solvers on a given query, distributes time amongst the solvers, and deploys the solvers in sequence until a solution is obtained. We evaluate MedleySolver against the best available alternative, an offline learning technique, in terms of pure performance and practical usability for a typical SMT user. We find that with no prior training, MedleySolver solves 93.9% of the queries solved by the virtual best solver selector achieving 59.8% of the par-2 score of the most successful individual solver, which solves 87.3%. For comparison, the best available alternative takes longer to train than MedleySolver takes to solve our entire set of 2000 queries
Machine learning for function synthesis
Function synthesis is the process of automatically constructing functions that satisfy a given specification. The space of functions as well as the format of the specifications vary greatly with each area of application. In this thesis, we consider synthesis in the context of satisfiability modulo theories. Within this domain, the goal is to synthesise mathematical expressions that adhere to abstract logical formulas. These types of synthesis problems find many applications in the field of computer-aided verification. One of the main challenges of function synthesis arises from the combinatorial explosion in the number of potential candidates within a certain size. The hypothesis of this thesis is that machine learning methods can be applied to make function synthesis more tractable.
The first contribution of this thesis is a Monte-Carlo based search method for function synthesis. The search algorithm uses machine learned heuristics to guide the search. This is part of a reinforcement learning loop that trains the machine learning models with data generated from previous search attempts. To increase the set of benchmark problems to train and test synthesis methods, we also present a technique for generating synthesis problems from pre-existing satisfiability modulo theories problems. We implement the Monte-Carlo based synthesis algorithm and evaluate it on standard synthesis benchmarks as well as our newly generated benchmarks. An experimental evaluation shows that the learned heuristics greatly improve on the baseline without trained models. Furthermore, the machine learned guidance demonstrates comparable performance to CVC5 and, in some experiments, even surpasses it.
Next, this thesis explores the application of machine learning to more restricted function synthesis domains. We hypothesise that narrowing the scope enables the use of machine learning techniques that are not possible in the general setting. We test this hypothesis by considering the problem of ranking function synthesis. Ranking functions are used in program analysis to prove termination of programs by mapping consecutive program states to decreasing elements of a well-founded set. The second contribution of this dissertation is a novel technique for synthesising ranking functions, using neural networks. The key insight is that instead of synthesising a mathematical expression that represents a ranking function, we can train a neural network to act as a ranking function. Hence, the synthesis procedure is replaced by neural network training. We introduce Neural Termination Analysis as a framework that leverages this. We train neural networks from sampled execution traces of the program we want to prove terminating. We enforce the synthesis specifications of ranking functions using the loss function and network design. After training, we use symbolic reasoning to formally verify that the resulting function is indeed a correct ranking function for the target program. We demonstrate that our method succeeds in synthesising ranking functions for programs that are beyond the reach of state-of-the-art tools. This includes programs with disjunctions and non-linear expressions in the loop guards
- …