Decisive Markov Chains

We consider qualitative and quantitative verification problems for infinite-state Markov chains. We call a Markov chain decisive w.r.t. a given set of target states F if it almost certainly eventually reaches either F or a state from which F can no longer be reached. While all finite Markov chains are trivially decisive (for every set F), this also holds for many classes of infinite Markov chains. Infinite Markov chains which contain a finite attractor are decisive w.r.t. every set F. In particular, this holds for probabilistic lossy channel systems (PLCS). Furthermore, all globally coarse Markov chains are decisive. This class includes probabilistic vector addition systems (PVASS) and probabilistic noisy Turing machines (PNTM). We consider both safety and liveness problems for decisive Markov chains, i.e., the probabilities that a given set of states F is eventually reached or reached infinitely often, respectively. 1. We express the qualitative problems in abstract terms for decisive Markov chains, and show an almost complete picture of its decidability for PLCS, PVASS and PNTM. 2. We also show that the path enumeration algorithm of Iyer and Narasimha terminates for decisive Markov chains and can thus be used to solve the approximate quantitative safety problem. A modified variant of this algorithm solves the approximate quantitative liveness problem. 3. Finally, we show that the exact probability of (repeatedly) reaching F cannot be effectively expressed (in a uniform way) in Tarski-algebra for either PLCS, PVASS or (P)NTM.Comment: 32 pages, 0 figure

Eager Markov Chains

Abstract. We consider infinite-state discrete Markov chains which are eager: the probability of avoiding a defined set of final states for more thanÒsteps is bounded by some exponentially decreasing function�(Ò). We prove that eager Markov chains include those induced by Probabilistic Lossy Channel Systems, Probabilistic Vector Addition Systems with States, and Noisy Turing Machines, and that the bounding function�(Ò) can be effectively constructed for them. Furthermore, we study the problem of computing the expected reward (or cost) of runs until reaching the final states, where rewards are assigned to individual runs by computable reward functions. For eager Markov chains, an effective path exploration scheme, based on forward reachability analysis, can be used to approximate the expected reward up-to an arbitrarily small error.

Does Church-Turing thesis apply outside computer science?

We analyze whether Church-Turing thesis can be applied to mathematical and physical systems. We find the factors that allow to a class of systems to reach a Turing or a super-Turing computational power. We illustrate our general statements by some more concrete theorems on hybrid and stochastic systems

Robust computations with dynamical systems

In this paper we discuss the computational power of Lipschitz dynamical systems which are robust to in nitesimal perturbations. Whereas the study in [1] was done only for not-so-natural systems from a classical mathematical point of view (discontinuous di erential equation systems, discontinuous piecewise a ne maps, or perturbed Turing machines), we prove that the results presented there can be generalized to Lipschitz and computable dynamical systems. In other words, we prove that the perturbed reachability problem (i.e. the reachability problem for systems which are subjected to in nitesimal perturbations) is co-recursively enumerable for this kind of systems. Using this result we show that if robustness to in nitesimal perturbations is also required, the reachability problem becomes decidable. This result can be interpreted in the following manner: undecidability of veri cation doesn't hold for Lipschitz, computable and robust systems. We also show that the perturbed reachability problem is co-r.e. complete even for C1-systems

Computation with perturbed dynamical systems

This paper analyzes the computational power of dynamical systems robust to infinitesimal perturbations. Previous work on the subject has delved on very specific types of systems. Here we obtain results for broader classes of dynamical systems (including those systems defined by Lipschitz/analytic functions). In particular we show that systems robust to infinitesimal perturbations only recognize recursive languages. We also show the converse direction: every recursive language can be robustly recognized by a computable system. By other words we show that robustness is equivalent to decidability. (C) 2013 Elsevier Inc. All rights reserved.INRIA program "Equipe Associee" ComputR; Fundacao para a Ciencia e a Tecnologia; EU FEDER POCTI/POCI via SQIG - Instituto de Telecomunicacoes through the FCT project [PEst-OE/EEI/LA0008/2011]info:eu-repo/semantics/publishedVersio

Stochastic Games with Lossy Channels

We consider turn-based stochastic games on infinite graphs induced by game probabilistic lossy channel systems (GPLCS), the game version of probabilistic lossy channel systems (PLCS). We study games with Büchi (repeated reachability) objectives and almost-sure winning conditions. These games are pure memoryless determined and, under the assumption that the target set is regular, a symbolic representation of the set of winning states for each player can be effectively constructed. Thus, turn-based stochastic games on GPLCS are decidable. This generalizes the decidability result for PLCS-induced Markov decision processes in [10]

A Survey on Continuous Time Computations

We provide an overview of theories of continuous time computation. These theories allow us to understand both the hardness of questions related to continuous time dynamical systems and the computational power of continuous time analog models. We survey the existing models, summarizing results, and point to relevant references in the literature

Sensitive finite state computations using a distributed network with a noisy network attractor

This is the author accepted manuscript. The final version is available from IEEE via the DOI in this record.We exhibit a class of smooth continuous-state neural-inspired networks composed of simple nonlinear elements that can be made to function as a finite state computational machine. We give an explicit construction of arbitrary finitestate virtual machines in the spatio-temporal dynamics of the network. The dynamics of the functional network can be completely characterised as a “noisy network attractor” in phase space operating in either an “excitable” or a “free-running” regime, respectively corresponding to excitable or heteroclinic connections between states. The regime depends on the sign of an “excitability parameter”. Viewing the network as a nonlinear stochastic differential equation where deterministic (signal) and/or stochastic (noise) input are applied to any element, we explore the influence of signal to noise ratio on the error rate of the computations. The free-running regime is extremely sensitive to inputs: arbitrarily small amplitude perturbations can be used to perform computations with the system as long as the input dominates the noise. We find a counter-intuitive regime where increasing noise amplitude can lead to more, rather than less, accurate computation. We suggest that noisy network attractors will be useful for understanding neural networks that reliably and sensitively perform finite-state computations in a noisy environment.PA gratefully acknowledges the financial support of the EPSRC via grant EP/N014391/1. CMP acknowledges travel funding from the University of Auckland and support from the London Mathematical Laboratory