651 research outputs found
Algorithmic Analysis of Qualitative and Quantitative Termination Problems for Affine Probabilistic Programs
In this paper, we consider termination of probabilistic programs with
real-valued variables. The questions concerned are:
1. qualitative ones that ask (i) whether the program terminates with
probability 1 (almost-sure termination) and (ii) whether the expected
termination time is finite (finite termination); 2. quantitative ones that ask
(i) to approximate the expected termination time (expectation problem) and (ii)
to compute a bound B such that the probability to terminate after B steps
decreases exponentially (concentration problem).
To solve these questions, we utilize the notion of ranking supermartingales
which is a powerful approach for proving termination of probabilistic programs.
In detail, we focus on algorithmic synthesis of linear ranking-supermartingales
over affine probabilistic programs (APP's) with both angelic and demonic
non-determinism. An important subclass of APP's is LRAPP which is defined as
the class of all APP's over which a linear ranking-supermartingale exists.
Our main contributions are as follows. Firstly, we show that the membership
problem of LRAPP (i) can be decided in polynomial time for APP's with at most
demonic non-determinism, and (ii) is NP-hard and in PSPACE for APP's with
angelic non-determinism; moreover, the NP-hardness result holds already for
APP's without probability and demonic non-determinism. Secondly, we show that
the concentration problem over LRAPP can be solved in the same complexity as
for the membership problem of LRAPP. Finally, we show that the expectation
problem over LRAPP can be solved in 2EXPTIME and is PSPACE-hard even for APP's
without probability and non-determinism (i.e., deterministic programs). Our
experimental results demonstrate the effectiveness of our approach to answer
the qualitative and quantitative questions over APP's with at most demonic
non-determinism.Comment: 24 pages, full version to the conference paper on POPL 201
Stochastic Invariants for Probabilistic Termination
Termination is one of the basic liveness properties, and we study the
termination problem for probabilistic programs with real-valued variables.
Previous works focused on the qualitative problem that asks whether an input
program terminates with probability~1 (almost-sure termination). A powerful
approach for this qualitative problem is the notion of ranking supermartingales
with respect to a given set of invariants. The quantitative problem
(probabilistic termination) asks for bounds on the termination probability. A
fundamental and conceptual drawback of the existing approaches to address
probabilistic termination is that even though the supermartingales consider the
probabilistic behavior of the programs, the invariants are obtained completely
ignoring the probabilistic aspect.
In this work we address the probabilistic termination problem for
linear-arithmetic probabilistic programs with nondeterminism. We define the
notion of {\em stochastic invariants}, which are constraints along with a
probability bound that the constraints hold. We introduce a concept of {\em
repulsing supermartingales}. First, we show that repulsing supermartingales can
be used to obtain bounds on the probability of the stochastic invariants.
Second, we show the effectiveness of repulsing supermartingales in the
following three ways: (1)~With a combination of ranking and repulsing
supermartingales we can compute lower bounds on the probability of termination;
(2)~repulsing supermartingales provide witnesses for refutation of almost-sure
termination; and (3)~with a combination of ranking and repulsing
supermartingales we can establish persistence properties of probabilistic
programs.
We also present results on related computational problems and an experimental
evaluation of our approach on academic examples.Comment: Full version of a paper published at POPL 2017. 20 page
Proving Expected Sensitivity of Probabilistic Programs with Randomized Variable-Dependent Termination Time
The notion of program sensitivity (aka Lipschitz continuity) specifies that
changes in the program input result in proportional changes to the program
output. For probabilistic programs the notion is naturally extended to expected
sensitivity. A previous approach develops a relational program logic framework
for proving expected sensitivity of probabilistic while loops, where the number
of iterations is fixed and bounded. In this work, we consider probabilistic
while loops where the number of iterations is not fixed, but randomized and
depends on the initial input values. We present a sound approach for proving
expected sensitivity of such programs. Our sound approach is martingale-based
and can be automated through existing martingale-synthesis algorithms.
Furthermore, our approach is compositional for sequential composition of while
loops under a mild side condition. We demonstrate the effectiveness of our
approach on several classical examples from Gambler's Ruin, stochastic hybrid
systems and stochastic gradient descent. We also present experimental results
showing that our automated approach can handle various probabilistic programs
in the literature
Computational Approaches for Stochastic Shortest Path on Succinct MDPs
We consider the stochastic shortest path (SSP) problem for succinct Markov
decision processes (MDPs), where the MDP consists of a set of variables, and a
set of nondeterministic rules that update the variables. First, we show that
several examples from the AI literature can be modeled as succinct MDPs. Then
we present computational approaches for upper and lower bounds for the SSP
problem: (a)~for computing upper bounds, our method is polynomial-time in the
implicit description of the MDP; (b)~for lower bounds, we present a
polynomial-time (in the size of the implicit description) reduction to
quadratic programming. Our approach is applicable even to infinite-state MDPs.
Finally, we present experimental results to demonstrate the effectiveness of
our approach on several classical examples from the AI literature
Finding polynomial loop invariants for probabilistic programs
Quantitative loop invariants are an essential element in the verification of
probabilistic programs. Recently, multivariate Lagrange interpolation has been
applied to synthesizing polynomial invariants. In this paper, we propose an
alternative approach. First, we fix a polynomial template as a candidate of a
loop invariant. Using Stengle's Positivstellensatz and a transformation to a
sum-of-squares problem, we find sufficient conditions on the coefficients.
Then, we solve a semidefinite programming feasibility problem to synthesize the
loop invariants. If the semidefinite program is unfeasible, we backtrack after
increasing the degree of the template. Our approach is semi-complete in the
sense that it will always lead us to a feasible solution if one exists and
numerical errors are small. Experimental results show the efficiency of our
approach.Comment: accompanies an ATVA 2017 submissio
Proving expected sensitivity of probabilistic programs with randomized variable-dependent termination time
The notion of program sensitivity (aka Lipschitz continuity) specifies that changes in the program input result in proportional changes to the program output. For probabilistic programs the notion is naturally extended to expected sensitivity. A previous approach develops a relational program logic framework for proving expected sensitivity of probabilistic while loops, where the number of iterations is fixed and bounded. In this work, we consider probabilistic while loops where the number of iterations is not fixed, but randomized and depends on the initial input values. We present a sound approach for proving expected sensitivity of such programs. Our sound approach is martingale-based and can be automated through existing martingale-synthesis algorithms. Furthermore, our approach is compositional for sequential composition of while loops under a mild side condition. We demonstrate the effectiveness of our approach on several classical examples from Gambler's Ruin, stochastic hybrid systems and stochastic gradient descent. We also present experimental results showing that our automated approach can handle various probabilistic programs in the literature
Ranking and Repulsing Supermartingales for Reachability in Probabilistic Programs
Computing reachability probabilities is a fundamental problem in the analysis
of probabilistic programs. This paper aims at a comprehensive and comparative
account on various martingale-based methods for over- and under-approximating
reachability probabilities. Based on the existing works that stretch across
different communities (formal verification, control theory, etc.), we offer a
unifying account. In particular, we emphasize the role of order-theoretic fixed
points---a classic topic in computer science---in the analysis of probabilistic
programs. This leads us to two new martingale-based techniques, too. We give
rigorous proofs for their soundness and completeness. We also make an
experimental comparison using our implementation of template-based synthesis
algorithms for those martingales
- …