Real-Reward Testing for Probabilistic Processes (Extended Abstract)

We introduce a notion of real-valued reward testing for probabilistic processes by extending the traditional nonnegative-reward testing with negative rewards. In this richer testing framework, the may and must preorders turn out to be inverses. We show that for convergent processes with finitely many states and transitions, but not in the presence of divergence, the real-reward must-testing preorder coincides with the nonnegative-reward must-testing preorder. To prove this coincidence we characterise the usual resolution-based testing in terms of the weak transitions of processes, without having to involve policies, adversaries, schedulers, resolutions, or similar structures that are external to the process under investigation. This requires establishing the continuity of our function for calculating testing outcomes.Comment: In Proceedings QAPL 2011, arXiv:1107.074

Is social decision making for close others consistent across domains and within individuals?

Humans make decisions across a variety of social contexts. Though social decision-making research has blossomed in recent decades, surprisingly little is known about whether social decision-making preferences are consistent across different domains. We conducted an exploratory study in which participants made choices about 2 types of close others: parents and friends. To elicit decision making preferences, we pit the interests in parents and friends against one another. To assess the consistency of preferences for close others, decision making was assessed in three domains-risk taking, probabilistic learning, and self-other similarity judgments. We reasoned that if social decision-making preferences are consistent across domains, participants ought to exhibit the same preference in all three domains (i.e., a parent preference, based on prior work), and individual differences in preference magnitude ought to be conserved across domains within individuals. A combination of computational modeling, random coefficient regression, and traditional statistical tests revealed a robust parent-over-friend preference in the risk taking and probabilistic learning domains but not the self-other similarity domain. Preferences for parent-over-friend in the risk-taking domain were strongly associated with similar preferences in the probabilistic learning domain but not the self-other similarity domain. These results suggest that distinct and dissociable value-based and social-cognitive computations underlie social decision making. (PsycInfo Database Record (c) 2020 APA, all rights reserved)

Learning Markov Decision Processes for Model Checking

Constructing an accurate system model for formal model verification can be both resource demanding and time-consuming. To alleviate this shortcoming, algorithms have been proposed for automatically learning system models based on observed system behaviors. In this paper we extend the algorithm on learning probabilistic automata to reactive systems, where the observed system behavior is in the form of alternating sequences of inputs and outputs. We propose an algorithm for automatically learning a deterministic labeled Markov decision process model from the observed behavior of a reactive system. The proposed learning algorithm is adapted from algorithms for learning deterministic probabilistic finite automata, and extended to include both probabilistic and nondeterministic transitions. The algorithm is empirically analyzed and evaluated by learning system models of slot machines. The evaluation is performed by analyzing the probabilistic linear temporal logic properties of the system as well as by analyzing the schedulers, in particular the optimal schedulers, induced by the learned models.Comment: In Proceedings QFM 2012, arXiv:1212.345

Strategy Synthesis for Autonomous Agents Using PRISM

We present probabilistic models for autonomous agent search and retrieve missions derived from Simulink models for an Unmanned Aerial Vehicle (UAV) and show how probabilistic model checking and the probabilistic model checker PRISM can be used for optimal controller generation. We introduce a sequence of scenarios relevant to UAVs and other autonomous agents such as underwater and ground vehicles. For each scenario we demonstrate how it can be modelled using the PRISM language, give model checking statistics and present the synthesised optimal controllers. We conclude with a discussion of the limitations when using probabilistic model checking and PRISM in this context and what steps can be taken to overcome them. In addition, we consider how the controllers can be returned to the UAV and adapted for use on larger search areas

Smart Sampling for Lightweight Verification of Markov Decision Processes

Markov decision processes (MDP) are useful to model optimisation problems in concurrent systems. To verify MDPs with efficient Monte Carlo techniques requires that their nondeterminism be resolved by a scheduler. Recent work has introduced the elements of lightweight techniques to sample directly from scheduler space, but finding optimal schedulers by simple sampling may be inefficient. Here we describe "smart" sampling algorithms that can make substantial improvements in performance.Comment: IEEE conference style, 11 pages, 5 algorithms, 11 figures, 1 tabl