Maximal Cost-Bounded Reachability Probability on Continuous-Time Markov Decision Processes

In this paper, we consider multi-dimensional maximal cost-bounded reachability probability over continuous-time Markov decision processes (CTMDPs). Our major contributions are as follows. Firstly, we derive an integral characterization which states that the maximal cost-bounded reachability probability function is the least fixed point of a system of integral equations. Secondly, we prove that the maximal cost-bounded reachability probability can be attained by a measurable deterministic cost-positional scheduler. Thirdly, we provide a numerical approximation algorithm for maximal cost-bounded reachability probability. We present these results under the setting of both early and late schedulers

Approximating Acceptance Probabilities of CTMC-Paths on Multi-Clock Deterministic Timed Automata

We consider the problem of approximating the probability mass of the set of timed paths under a continuous-time Markov chain (CTMC) that are accepted by a deterministic timed automaton (DTA). As opposed to several existing works on this topic, we consider DTA with multiple clocks. Our key contribution is an algorithm to approximate these probabilities using finite difference methods. An error bound is provided which indicates the approximation error. The stepping stones towards this result include rigorous proofs for the measurability of the set of accepted paths and the integral-equation system characterizing the acceptance probability, and a differential characterization for the acceptance probability

Computational Approaches for Stochastic Shortest Path on Succinct MDPs

We consider the stochastic shortest path (SSP) problem for succinct Markov decision processes (MDPs), where the MDP consists of a set of variables, and a set of nondeterministic rules that update the variables. First, we show that several examples from the AI literature can be modeled as succinct MDPs. Then we present computational approaches for upper and lower bounds for the SSP problem: (a)~for computing upper bounds, our method is polynomial-time in the implicit description of the MDP; (b)~for lower bounds, we present a polynomial-time (in the size of the implicit description) reduction to quadratic programming. Our approach is applicable even to infinite-state MDPs. Finally, we present experimental results to demonstrate the effectiveness of our approach on several classical examples from the AI literature

Proving Expected Sensitivity of Probabilistic Programs with Randomized Variable-Dependent Termination Time

The notion of program sensitivity (aka Lipschitz continuity) specifies that changes in the program input result in proportional changes to the program output. For probabilistic programs the notion is naturally extended to expected sensitivity. A previous approach develops a relational program logic framework for proving expected sensitivity of probabilistic while loops, where the number of iterations is fixed and bounded. In this work, we consider probabilistic while loops where the number of iterations is not fixed, but randomized and depends on the initial input values. We present a sound approach for proving expected sensitivity of such programs. Our sound approach is martingale-based and can be automated through existing martingale-synthesis algorithms. Furthermore, our approach is compositional for sequential composition of while loops under a mild side condition. We demonstrate the effectiveness of our approach on several classical examples from Gambler's Ruin, stochastic hybrid systems and stochastic gradient descent. We also present experimental results showing that our automated approach can handle various probabilistic programs in the literature

Modal analysis of the certain membrane disc coupling

The membrane disc coupling has the ability to transmit higher power under high-speed rotation. It also has the ability of compensation angular and radial displacement. The finite element model of the coupling was established and vibrational characteristics were analyzed. The modal experiments were carried out with hammering method, the modal parameters were obtained and the correct of the simulations was verified under the same constraints with the FEM model. The results showed that simulating results agreed well with that of the experiments

Automated Tail Bound Analysis for Probabilistic Recurrence Relations

Probabilistic recurrence relations (PRRs) are a standard formalism for describing the runtime of a randomized algorithm. Given a PRR and a time limit $\kappa$, we consider the classical concept of tail probability $\Pr[T \ge \kappa]$, i.e., the probability that the randomized runtime $T$ of the PRR exceeds the time limit $\kappa$. Our focus is the formal analysis of tail bounds that aims at finding a tight asymptotic upper bound $u \geq \Pr[T\ge\kappa]$ in the time limit $\kappa$. To address this problem, the classical and most well-known approach is the cookbook method by Karp (JACM 1994), while other approaches are mostly limited to deriving tail bounds of specific PRRs via involved custom analysis. In this work, we propose a novel approach for deriving exponentially-decreasing tail bounds (a common type of tail bounds) for PRRs whose preprocessing time and random passed sizes observe discrete or (piecewise) uniform distribution and whose recursive call is either a single procedure call or a divide-and-conquer. We first establish a theoretical approach via Markov's inequality, and then instantiate the theoretical approach with a template-based algorithmic approach via a refined treatment of exponentiation. Experimental evaluation shows that our algorithmic approach is capable of deriving tail bounds that are (i) asymptotically tighter than Karp's method, (ii) match the best-known manually-derived asymptotic tail bound for QuickSelect, and (iii) is only slightly worse (with a $\log\log n$ factor) than the manually-proven optimal asymptotic tail bound for QuickSort. Moreover, our algorithmic approach handles all examples (including realistic PRRs such as QuickSort, QuickSelect, DiameterComputation, etc.) in less than 0.1 seconds, showing that our approach is efficient in practice.Comment: 46 pages, 15 figure

Modular Verification for Almost-Sure Termination of Probabilistic Programs

International audienceIn this work, we consider the almost-sure termination problem for probabilistic programs that asks whether agiven probabilistic program terminates with probability 1. Scalable approaches for program analysis oftenrely on modularity as their theoretical basis. In non-probabilistic programs, the classical variant rule (V-rule)of Floyd-Hoare logic provides the foundation for modular analysis. Extension of this rule to almost-suretermination of probabilistic programs is quite tricky, and a probabilistic variant was proposed in [Fioriti andHermanns 2015]. While the proposed probabilistic variant cautiously addresses the key issue of integrability,we show that the proposed modular rule is still not sound for almost-sure termination of probabilistic programs.Besides establishing unsoundness of the previous rule, our contributions are as follows: First, we present asound modular rule for almost-sure termination of probabilistic programs. Our approach is based on a novelnotion of descent supermartingales. Second, for algorithmic approaches, we consider descent supermartingalesthat are linear and show that they can be synthesized in polynomial time. Finally, we present experimentalresults on a variety of benchmarks and several natural examples that model various types of nested whileloops in probabilistic programs and demonstrate that our approach is able to efficiently prove their almost-suretermination property