4,695 research outputs found
Relatively Complete Counterexamples for Higher-Order Programs
In this paper, we study the problem of generating inputs to a higher-order
program causing it to error. We first study the problem in the setting of PCF,
a typed, core functional language and contribute the first relatively complete
method for constructing counterexamples for PCF programs. The method is
relatively complete in the sense of Hoare logic; completeness is reduced to the
completeness of a first-order solver over the base types of PCF. In practice,
this means an SMT solver can be used for the effective, automated generation of
higher-order counterexamples for a large class of programs.
We achieve this result by employing a novel form of symbolic execution for
higher-order programs. The remarkable aspect of this symbolic execution is that
even though symbolic higher-order inputs and values are considered, the path
condition remains a first-order formula. Our handling of symbolic function
application enables the reconstruction of higher-order counterexamples from
this first-order formula.
After establishing our main theoretical results, we sketch how to apply the
approach to untyped, higher-order, stateful languages with first-class
contracts and show how counterexample generation can be used to detect contract
violations in this setting. To validate our approach, we implement a tool
generating counterexamples for erroneous modules written in Racket.Comment: In Proceedings of the 36th annual ACM SIGPLAN conference on
Programming Language Design and Implementation, Portland, Oregon, June 201
Formal Analysis and Redesign of a Neural Network-Based Aircraft Taxiing System with VerifAI
We demonstrate a unified approach to rigorous design of safety-critical
autonomous systems using the VerifAI toolkit for formal analysis of AI-based
systems. VerifAI provides an integrated toolchain for tasks spanning the design
process, including modeling, falsification, debugging, and ML component
retraining. We evaluate all of these applications in an industrial case study
on an experimental autonomous aircraft taxiing system developed by Boeing,
which uses a neural network to track the centerline of a runway. We define
runway scenarios using the Scenic probabilistic programming language, and use
them to drive tests in the X-Plane flight simulator. We first perform
falsification, automatically finding environment conditions causing the system
to violate its specification by deviating significantly from the centerline (or
even leaving the runway entirely). Next, we use counterexample analysis to
identify distinct failure cases, and confirm their root causes with specialized
testing. Finally, we use the results of falsification and debugging to retrain
the network, eliminating several failure cases and improving the overall
performance of the closed-loop system.Comment: Full version of a CAV 2020 pape
Optimization and Abstraction: A Synergistic Approach for Analyzing Neural Network Robustness
In recent years, the notion of local robustness (or robustness for short) has
emerged as a desirable property of deep neural networks. Intuitively,
robustness means that small perturbations to an input do not cause the network
to perform misclassifications. In this paper, we present a novel algorithm for
verifying robustness properties of neural networks. Our method synergistically
combines gradient-based optimization methods for counterexample search with
abstraction-based proof search to obtain a sound and ({\delta}-)complete
decision procedure. Our method also employs a data-driven approach to learn a
verification policy that guides abstract interpretation during proof search. We
have implemented the proposed approach in a tool called Charon and
experimentally evaluated it on hundreds of benchmarks. Our experiments show
that the proposed approach significantly outperforms three state-of-the-art
tools, namely AI^2 , Reluplex, and Reluval
Automatic Verification of Message-Based Device Drivers
We develop a practical solution to the problem of automatic verification of
the interface between device drivers and the OS. Our solution relies on a
combination of improved driver architecture and verification tools. It supports
drivers written in C and can be implemented in any existing OS, which sets it
apart from previous proposals for verification-friendly drivers. Our
Linux-based evaluation shows that this methodology amplifies the power of
existing verification tools in detecting driver bugs, making it possible to
verify properties beyond the reach of traditional techniques.Comment: In Proceedings SSV 2012, arXiv:1211.587
Content and Context: The Contributions of William Van Alstyne to First Amendment Interpretation
Not available
Sciduction: Combining Induction, Deduction, and Structure for Verification and Synthesis
Even with impressive advances in automated formal methods, certain problems
in system verification and synthesis remain challenging. Examples include the
verification of quantitative properties of software involving constraints on
timing and energy consumption, and the automatic synthesis of systems from
specifications. The major challenges include environment modeling,
incompleteness in specifications, and the complexity of underlying decision
problems.
This position paper proposes sciduction, an approach to tackle these
challenges by integrating inductive inference, deductive reasoning, and
structure hypotheses. Deductive reasoning, which leads from general rules or
concepts to conclusions about specific problem instances, includes techniques
such as logical inference and constraint solving. Inductive inference, which
generalizes from specific instances to yield a concept, includes algorithmic
learning from examples. Structure hypotheses are used to define the class of
artifacts, such as invariants or program fragments, generated during
verification or synthesis. Sciduction constrains inductive and deductive
reasoning using structure hypotheses, and actively combines inductive and
deductive reasoning: for instance, deductive techniques generate examples for
learning, and inductive reasoning is used to guide the deductive engines.
We illustrate this approach with three applications: (i) timing analysis of
software; (ii) synthesis of loop-free programs, and (iii) controller synthesis
for hybrid systems. Some future applications are also discussed
- …