78,466 research outputs found
Are There Good Mistakes? A Theoretical Analysis of CEGIS
Counterexample-guided inductive synthesis CEGIS is used to synthesize
programs from a candidate space of programs. The technique is guaranteed to
terminate and synthesize the correct program if the space of candidate programs
is finite. But the technique may or may not terminate with the correct program
if the candidate space of programs is infinite. In this paper, we perform a
theoretical analysis of counterexample-guided inductive synthesis technique. We
investigate whether the set of candidate spaces for which the correct program
can be synthesized using CEGIS depends on the counterexamples used in inductive
synthesis, that is, whether there are good mistakes which would increase the
synthesis power. We investigate whether the use of minimal counterexamples
instead of arbitrary counterexamples expands the set of candidate spaces of
programs for which inductive synthesis can successfully synthesize a correct
program. We consider two kinds of counterexamples: minimal counterexamples and
history bounded counterexamples. The history bounded counterexample used in any
iteration of CEGIS is bounded by the examples used in previous iterations of
inductive synthesis. We examine the relative change in power of inductive
synthesis in both cases. We show that the synthesis technique using minimal
counterexamples MinCEGIS has the same synthesis power as CEGIS but the
synthesis technique using history bounded counterexamples HCEGIS has different
power than that of CEGIS, but none dominates the other.Comment: In Proceedings SYNT 2014, arXiv:1407.493
A Theory of Formal Synthesis via Inductive Learning
Formal synthesis is the process of generating a program satisfying a
high-level formal specification. In recent times, effective formal synthesis
methods have been proposed based on the use of inductive learning. We refer to
this class of methods that learn programs from examples as formal inductive
synthesis. In this paper, we present a theoretical framework for formal
inductive synthesis. We discuss how formal inductive synthesis differs from
traditional machine learning. We then describe oracle-guided inductive
synthesis (OGIS), a framework that captures a family of synthesizers that
operate by iteratively querying an oracle. An instance of OGIS that has had
much practical impact is counterexample-guided inductive synthesis (CEGIS). We
present a theoretical characterization of CEGIS for learning any program that
computes a recursive language. In particular, we analyze the relative power of
CEGIS variants where the types of counterexamples generated by the oracle
varies. We also consider the impact of bounded versus unbounded memory
available to the learning algorithm. In the special case where the universe of
candidate programs is finite, we relate the speed of convergence to the notion
of teaching dimension studied in machine learning theory. Altogether, the
results of the paper take a first step towards a theoretical foundation for the
emerging field of formal inductive synthesis
Sciduction: Combining Induction, Deduction, and Structure for Verification and Synthesis
Even with impressive advances in automated formal methods, certain problems
in system verification and synthesis remain challenging. Examples include the
verification of quantitative properties of software involving constraints on
timing and energy consumption, and the automatic synthesis of systems from
specifications. The major challenges include environment modeling,
incompleteness in specifications, and the complexity of underlying decision
problems.
This position paper proposes sciduction, an approach to tackle these
challenges by integrating inductive inference, deductive reasoning, and
structure hypotheses. Deductive reasoning, which leads from general rules or
concepts to conclusions about specific problem instances, includes techniques
such as logical inference and constraint solving. Inductive inference, which
generalizes from specific instances to yield a concept, includes algorithmic
learning from examples. Structure hypotheses are used to define the class of
artifacts, such as invariants or program fragments, generated during
verification or synthesis. Sciduction constrains inductive and deductive
reasoning using structure hypotheses, and actively combines inductive and
deductive reasoning: for instance, deductive techniques generate examples for
learning, and inductive reasoning is used to guide the deductive engines.
We illustrate this approach with three applications: (i) timing analysis of
software; (ii) synthesis of loop-free programs, and (iii) controller synthesis
for hybrid systems. Some future applications are also discussed
Invariant Synthesis for Incomplete Verification Engines
We propose a framework for synthesizing inductive invariants for incomplete
verification engines, which soundly reduce logical problems in undecidable
theories to decidable theories. Our framework is based on the counter-example
guided inductive synthesis principle (CEGIS) and allows verification engines to
communicate non-provability information to guide invariant synthesis. We show
precisely how the verification engine can compute such non-provability
information and how to build effective learning algorithms when invariants are
expressed as Boolean combinations of a fixed set of predicates. Moreover, we
evaluate our framework in two verification settings, one in which verification
engines need to handle quantified formulas and one in which verification
engines have to reason about heap properties expressed in an expressive but
undecidable separation logic. Our experiments show that our invariant synthesis
framework based on non-provability information can both effectively synthesize
inductive invariants and adequately strengthen contracts across a large suite
of programs
Stepping Stones to Inductive Synthesis of Low-Level Looping Programs
Inductive program synthesis, from input/output examples, can provide an
opportunity to automatically create programs from scratch without presupposing
the algorithmic form of the solution. For induction of general programs with
loops (as opposed to loop-free programs, or synthesis for domain-specific
languages), the state of the art is at the level of introductory programming
assignments. Most problems that require algorithmic subtlety, such as fast
sorting, have remained out of reach without the benefit of significant
problem-specific background knowledge. A key challenge is to identify cues that
are available to guide search towards correct looping programs. We present
MAKESPEARE, a simple delayed-acceptance hillclimbing method that synthesizes
low-level looping programs from input/output examples. During search, delayed
acceptance bypasses small gains to identify significantly-improved stepping
stone programs that tend to generalize and enable further progress. The method
performs well on a set of established benchmarks, and succeeds on the
previously unsolved "Collatz Numbers" program synthesis problem. Additional
benchmarks include the problem of rapidly sorting integer arrays, in which we
observe the emergence of comb sort (a Shell sort variant that is empirically
fast). MAKESPEARE has also synthesized a record-setting program on one of the
puzzles from the TIS-100 assembly language programming game.Comment: AAAI 201
Learning MDL logic programs from noisy data
Many inductive logic programming approaches struggle to learn programs from noisy data. To overcome this limitation, we introduce an approach that learns minimal description length programs from noisy data, including recursive programs. Our experiments on several domains, including drug design, game playing, and program synthesis, show that our approach can outperform existing approaches in terms of predictive accuracies and scale to moderate amounts of noise
Invariant Synthesis for Incomplete Verification Engines
We propose a framework for synthesizing inductive invariants for incomplete verification engines, which soundly reduce logical problems in undecidable theories to decidable theories. Our framework is based on the counter-example guided inductive synthesis principle (CEGIS) and allows verification engines to communicate non-provability information to guide invariant synthesis. We show precisely how the verification engine can compute such non-provability information and how to build effective learning algorithms when invariants are expressed as Boolean combinations of a fixed set of predicates. Moreover, we evaluate our framework in two verification settings, one in which verification engines need to handle quantified formulas and one in which verification engines have to reason about heap properties expressed in an expressive but undecidable separation logic. Our experiments show that our invariant synthesis framework based on non-provability information can both effectively synthesize inductive invariants and adequately strengthen contracts across a large suite of programs
- …