2,157 research outputs found
A New Proof Rule for Almost-Sure Termination
An important question for a probabilistic program is whether the probability
mass of all its diverging runs is zero, that is that it terminates "almost
surely". Proving that can be hard, and this paper presents a new method for
doing so; it is expressed in a program logic, and so applies directly to source
code. The programs may contain both probabilistic- and demonic choice, and the
probabilistic choices may depend on the current state.
As do other researchers, we use variant functions (a.k.a.
"super-martingales") that are real-valued and probabilistically might decrease
on each loop iteration; but our key innovation is that the amount as well as
the probability of the decrease are parametric.
We prove the soundness of the new rule, indicate where its applicability goes
beyond existing rules, and explain its connection to classical results on
denumerable (non-demonic) Markov chains.Comment: V1 to appear in PoPL18. This version collects some existing text into
new example subsection 5.5 and adds a new example 5.6 and makes further
remarks about uncountable branching. The new example 5.6 relates to work on
lexicographic termination methods, also to appear in PoPL18 [Agrawal et al,
2018
Stochastic Invariants for Probabilistic Termination
Termination is one of the basic liveness properties, and we study the
termination problem for probabilistic programs with real-valued variables.
Previous works focused on the qualitative problem that asks whether an input
program terminates with probability~1 (almost-sure termination). A powerful
approach for this qualitative problem is the notion of ranking supermartingales
with respect to a given set of invariants. The quantitative problem
(probabilistic termination) asks for bounds on the termination probability. A
fundamental and conceptual drawback of the existing approaches to address
probabilistic termination is that even though the supermartingales consider the
probabilistic behavior of the programs, the invariants are obtained completely
ignoring the probabilistic aspect.
In this work we address the probabilistic termination problem for
linear-arithmetic probabilistic programs with nondeterminism. We define the
notion of {\em stochastic invariants}, which are constraints along with a
probability bound that the constraints hold. We introduce a concept of {\em
repulsing supermartingales}. First, we show that repulsing supermartingales can
be used to obtain bounds on the probability of the stochastic invariants.
Second, we show the effectiveness of repulsing supermartingales in the
following three ways: (1)~With a combination of ranking and repulsing
supermartingales we can compute lower bounds on the probability of termination;
(2)~repulsing supermartingales provide witnesses for refutation of almost-sure
termination; and (3)~with a combination of ranking and repulsing
supermartingales we can establish persistence properties of probabilistic
programs.
We also present results on related computational problems and an experimental
evaluation of our approach on academic examples.Comment: Full version of a paper published at POPL 2017. 20 page
Supercritical super-Brownian motion with a general branching mechanism and travelling waves
We consider the classical problem of existence, uniqueness and asymptotics of
monotone solutions to the travelling wave equation associated to the parabolic
semi-group equation of a super-Brownian motion with a general branching
mechanism. Whilst we are strongly guided by the probabilistic reasoning of
Kyprianou (2004) for branching Brownian motion, the current paper offers a
number of new insights. Our analysis incorporates the role of Seneta-Heyde
norming which, in the current setting, draws on classical work of Grey (1974).
We give a pathwise explanation of Evans' immortal particle picture (the spine
decomposition) which uses the Dynkin-Kuznetsov N-measure as a key ingredient.
Moreover, in the spirit of Neveu's stopping lines we make repeated use of
Dynkin's exit measures. Additional complications arise from the general nature
of the branching mechanism. As a consequence of the analysis we also offer an
exact X(log X)^2 moment dichotomy for the almost sure convergence of the
so-called derivative martingale at its critical parameter to a non-trivial
limit. This differs to the case of branching Brownian motion and branching
random walk where a moment `gap' appears in the necessary and sufficient
conditions.Comment: 34 page
Computational Approaches for Stochastic Shortest Path on Succinct MDPs
We consider the stochastic shortest path (SSP) problem for succinct Markov
decision processes (MDPs), where the MDP consists of a set of variables, and a
set of nondeterministic rules that update the variables. First, we show that
several examples from the AI literature can be modeled as succinct MDPs. Then
we present computational approaches for upper and lower bounds for the SSP
problem: (a)~for computing upper bounds, our method is polynomial-time in the
implicit description of the MDP; (b)~for lower bounds, we present a
polynomial-time (in the size of the implicit description) reduction to
quadratic programming. Our approach is applicable even to infinite-state MDPs.
Finally, we present experimental results to demonstrate the effectiveness of
our approach on several classical examples from the AI literature
Algorithmic Analysis of Qualitative and Quantitative Termination Problems for Affine Probabilistic Programs
In this paper, we consider termination of probabilistic programs with
real-valued variables. The questions concerned are:
1. qualitative ones that ask (i) whether the program terminates with
probability 1 (almost-sure termination) and (ii) whether the expected
termination time is finite (finite termination); 2. quantitative ones that ask
(i) to approximate the expected termination time (expectation problem) and (ii)
to compute a bound B such that the probability to terminate after B steps
decreases exponentially (concentration problem).
To solve these questions, we utilize the notion of ranking supermartingales
which is a powerful approach for proving termination of probabilistic programs.
In detail, we focus on algorithmic synthesis of linear ranking-supermartingales
over affine probabilistic programs (APP's) with both angelic and demonic
non-determinism. An important subclass of APP's is LRAPP which is defined as
the class of all APP's over which a linear ranking-supermartingale exists.
Our main contributions are as follows. Firstly, we show that the membership
problem of LRAPP (i) can be decided in polynomial time for APP's with at most
demonic non-determinism, and (ii) is NP-hard and in PSPACE for APP's with
angelic non-determinism; moreover, the NP-hardness result holds already for
APP's without probability and demonic non-determinism. Secondly, we show that
the concentration problem over LRAPP can be solved in the same complexity as
for the membership problem of LRAPP. Finally, we show that the expectation
problem over LRAPP can be solved in 2EXPTIME and is PSPACE-hard even for APP's
without probability and non-determinism (i.e., deterministic programs). Our
experimental results demonstrate the effectiveness of our approach to answer
the qualitative and quantitative questions over APP's with at most demonic
non-determinism.Comment: 24 pages, full version to the conference paper on POPL 201
How long, O Bayesian network, will I sample thee? A program analysis perspective on expected sampling times
Bayesian networks (BNs) are probabilistic graphical models for describing
complex joint probability distributions. The main problem for BNs is inference:
Determine the probability of an event given observed evidence. Since exact
inference is often infeasible for large BNs, popular approximate inference
methods rely on sampling.
We study the problem of determining the expected time to obtain a single
valid sample from a BN. To this end, we translate the BN together with
observations into a probabilistic program. We provide proof rules that yield
the exact expected runtime of this program in a fully automated fashion. We
implemented our approach and successfully analyzed various real-world BNs taken
from the Bayesian network repository
- …