1,624 research outputs found
Explaining SAT Solving Using Causal Reasoning
The past three decades have witnessed notable success in designing efficient SAT solvers, with modern solvers capable of solving industrial benchmarks containing millions of variables in just a few seconds. The success of modern SAT solvers owes to the widely-used CDCL algorithm, which lacks comprehensive theoretical investigation. Furthermore, it has been observed that CDCL solvers still struggle to deal with specific classes of benchmarks comprising only hundreds of variables, which contrasts with their widespread use in real-world applications. Consequently, there is an urgent need to uncover the inner workings of these seemingly weak yet powerful black boxes.
In this paper, we present a first step towards this goal by introducing an approach called {CausalSAT}, which employs causal reasoning to gain insights into the functioning of modern SAT solvers. {CausalSAT} initially generates observational data from the execution of SAT solvers and learns a structured graph representing the causal relationships between the components of a SAT solver. Subsequently, given a query such as whether a clause with low literals blocks distance (LBD) has a higher clause utility, {CausalSAT} calculates the causal effect of LBD on clause utility and provides an answer to the question. We use {CausalSAT} to quantitatively verify hypotheses previously regarded as "rules of thumb" or empirical findings, such as the query above or the notion that clauses with high LBD experience a rapid drop in utility over time. Moreover, {CausalSAT} can address previously unexplored questions, like which branching heuristic leads to greater clause utility in order to study the relationship between branching and clause management. Experimental evaluations using practical benchmarks demonstrate that {CausalSAT} effectively fits the data, verifies four "rules of thumb", and provides answers to three questions closely related to implementing modern solvers
A Unified View of Piecewise Linear Neural Network Verification
The success of Deep Learning and its potential use in many safety-critical
applications has motivated research on formal verification of Neural Network
(NN) models. Despite the reputation of learned NN models to behave as black
boxes and the theoretical hardness of proving their properties, researchers
have been successful in verifying some classes of models by exploiting their
piecewise linear structure and taking insights from formal methods such as
Satisifiability Modulo Theory. These methods are however still far from scaling
to realistic neural networks. To facilitate progress on this crucial area, we
make two key contributions. First, we present a unified framework that
encompasses previous methods. This analysis results in the identification of
new methods that combine the strengths of multiple existing approaches,
accomplishing a speedup of two orders of magnitude compared to the previous
state of the art. Second, we propose a new data set of benchmarks which
includes a collection of previously released testcases. We use the benchmark to
provide the first experimental comparison of existing algorithms and identify
the factors impacting the hardness of verification problems.Comment: Updated version of "Piecewise Linear Neural Network verification: A
comparative study
ML + FV = ? A Survey on the Application of Machine Learning to Formal Verification
Formal Verification (FV) and Machine Learning (ML) can seem incompatible due
to their opposite mathematical foundations and their use in real-life problems:
FV mostly relies on discrete mathematics and aims at ensuring correctness; ML
often relies on probabilistic models and consists of learning patterns from
training data. In this paper, we postulate that they are complementary in
practice, and explore how ML helps FV in its classical approaches: static
analysis, model-checking, theorem-proving, and SAT solving. We draw a landscape
of the current practice and catalog some of the most prominent uses of ML
inside FV tools, thus offering a new perspective on FV techniques that can help
researchers and practitioners to better locate the possible synergies. We
discuss lessons learned from our work, point to possible improvements and offer
visions for the future of the domain in the light of the science of software
and systems modeling.Comment: 13 pages, no figures, 3 table
Branch and Bound for Piecewise Linear Neural Network Verification
The success of Deep Learning and its potential use in many safety-critical
applications has motivated research on formal verification of Neural Network
(NN) models. In this context, verification involves proving or disproving that
an NN model satisfies certain input-output properties. Despite the reputation
of learned NN models as black boxes, and the theoretical hardness of proving
useful properties about them, researchers have been successful in verifying
some classes of models by exploiting their piecewise linear structure and
taking insights from formal methods such as Satisifiability Modulo Theory.
However, these methods are still far from scaling to realistic neural networks.
To facilitate progress on this crucial area, we exploit the Mixed Integer
Linear Programming (MIP) formulation of verification to propose a family of
algorithms based on Branch-and-Bound (BaB). We show that our family contains
previous verification methods as special cases. With the help of the BaB
framework, we make three key contributions. Firstly, we identify new methods
that combine the strengths of multiple existing approaches, accomplishing
significant performance improvements over previous state of the art. Secondly,
we introduce an effective branching strategy on ReLU non-linearities. This
branching strategy allows us to efficiently and successfully deal with high
input dimensional problems with convolutional network architecture, on which
previous methods fail frequently. Finally, we propose comprehensive test data
sets and benchmarks which includes a collection of previously released
testcases. We use the data sets to conduct a thorough experimental comparison
of existing and new algorithms and to provide an inclusive analysis of the
factors impacting the hardness of verification problems
- âŠ