129 research outputs found
An Abstraction-Based Framework for Neural Network Verification
Deep neural networks are increasingly being used as controllers for safety-critical systems. Because neural networks are opaque, certifying their correctness is a significant challenge. To address this issue, several neural network verification approaches have recently been proposed. However, these approaches afford limited scalability, and applying them to large networks can be challenging. In this paper, we propose a framework that can enhance neural network verification techniques by using over-approximation to reduce the size of the networkâthus making it more amenable to verification. We perform the approximation such that if the property holds for the smaller (abstract) network, it holds for the original as well. The over-approximation may be too coarse, in which case the underlying verification tool might return a spurious counterexample. Under such conditions, we perform counterexample-guided refinement to adjust the approximation, and then repeat the process. Our approach is orthogonal to, and can be integrated with, many existing verification techniques. For evaluation purposes, we integrate it with the recently proposed Marabou framework, and observe a significant improvement in Marabouâs performance. Our experiments demonstrate the great potential of our approach for verifying larger neural networks
A DPLL(T) Framework for Verifying Deep Neural Networks
Deep Neural Networks (DNNs) have emerged as an effective approach to tackling
real-world problems. However, like human-written software,
automatically-generated DNNs can have bugs and be attacked. This thus attracts
many recent interests in developing effective and scalable DNN verification
techniques and tools. In this work, we introduce a NeuralSAT, a new constraint
solving approach to DNN verification. The design of NeuralSAT follows the
DPLL(T) algorithm used modern SMT solving, which includes (conflict) clause
learning, abstraction, and theory solving, and thus NeuralSAT can be considered
as an SMT framework for DNNs. Preliminary results show that the NeuralSAT
prototype is competitive to the state-of-the-art. We hope, with proper
optimization and engineering, NeuralSAT will carry the power and success of
modern SAT/SMT solvers to DNN verification. NeuralSAT is avaliable from:
https://github.com/dynaroars/neuralsat-solverComment: 27 pages, 8 figures. NeuralSAT is avaliable from:
https://github.com/dynaroars/neuralsat-solve
A Review of Formal Methods applied to Machine Learning
We review state-of-the-art formal methods applied to the emerging field of
the verification of machine learning systems. Formal methods can provide
rigorous correctness guarantees on hardware and software systems. Thanks to the
availability of mature tools, their use is well established in the industry,
and in particular to check safety-critical applications as they undergo a
stringent certification process. As machine learning is becoming more popular,
machine-learned components are now considered for inclusion in critical
systems. This raises the question of their safety and their verification. Yet,
established formal methods are limited to classic, i.e. non machine-learned
software. Applying formal methods to verify systems that include machine
learning has only been considered recently and poses novel challenges in
soundness, precision, and scalability.
We first recall established formal methods and their current use in an
exemplar safety-critical field, avionic software, with a focus on abstract
interpretation based techniques as they provide a high level of scalability.
This provides a golden standard and sets high expectations for machine learning
verification. We then provide a comprehensive and detailed review of the formal
methods developed so far for machine learning, highlighting their strengths and
limitations. The large majority of them verify trained neural networks and
employ either SMT, optimization, or abstract interpretation techniques. We also
discuss methods for support vector machines and decision tree ensembles, as
well as methods targeting training and data preparation, which are critical but
often neglected aspects of machine learning. Finally, we offer perspectives for
future research directions towards the formal verification of machine learning
systems
- âŠ