Value Iteration for Long-run Average Reward in Markov Decision Processes

Markov decision processes (MDPs) are standard models for probabilistic systems with non-deterministic behaviours. Long-run average rewards provide a mathematically elegant formalism for expressing long term performance. Value iteration (VI) is one of the simplest and most efficient algorithmic approaches to MDPs with other properties, such as reachability objectives. Unfortunately, a naive extension of VI does not work for MDPs with long-run average rewards, as there is no known stopping criterion. In this work our contributions are threefold. (1) We refute a conjecture related to stopping criteria for MDPs with long-run average rewards. (2) We present two practical algorithms for MDPs with long-run average rewards based on VI. First, we show that a combination of applying VI locally for each maximal end-component (MEC) and VI for reachability objectives can provide approximation guarantees. Second, extending the above approach with a simulation-guided on-demand variant of VI, we present an anytime algorithm that is able to deal with very large models. (3) Finally, we present experimental results showing that our methods significantly outperform the standard approaches on several benchmarks

Optimized preparation of quantum states by conditional measurements

We introduce a general strategy for preparation of arbitrary quantum states via optimal control of repeated conditional measurements. The effectiveness of this strategy in generating finite Fock-state superpositions with a high level of confidence from experimentally accessible coherent states is demonstrated for the simple and well known Jaynes-Cummings model dynamics

Ranking and Repulsing Supermartingales for Reachability in Probabilistic Programs

Computing reachability probabilities is a fundamental problem in the analysis of probabilistic programs. This paper aims at a comprehensive and comparative account on various martingale-based methods for over- and under-approximating reachability probabilities. Based on the existing works that stretch across different communities (formal verification, control theory, etc.), we offer a unifying account. In particular, we emphasize the role of order-theoretic fixed points---a classic topic in computer science---in the analysis of probabilistic programs. This leads us to two new martingale-based techniques, too. We give rigorous proofs for their soundness and completeness. We also make an experimental comparison using our implementation of template-based synthesis algorithms for those martingales

Gravity and Light: Combining Gravitational Wave and Electromagnetic Observations in the 2020s

As of today, we have directly detected exactly one source in both gravitational waves (GWs) and electromagnetic (EM) radiation, the binary neutron star merger GW170817, its associated gamma-ray burst GRB170817A, and the subsequent kilonova SSS17a/AT 2017gfo. Within ten years, we will detect hundreds of events, including new classes of events such as neutron-star-black-hole mergers, core-collapse supernovae, and almost certainly something completely unexpected. As we build this sample, we will explore exotic astrophysical topics ranging from nucleosynthesis, stellar evolution, general relativity, high-energy astrophysics, nuclear matter, to cosmology. The discovery potential is extraordinary, and investments in this area will yield major scientific breakthroughs. Here we outline some of the most exciting scientific questions that can be answered by combining GW and EM observations

Stochastic Invariants for Probabilistic Termination

Termination is one of the basic liveness properties, and we study the termination problem for probabilistic programs with real-valued variables. Previous works focused on the qualitative problem that asks whether an input program terminates with probability~1 (almost-sure termination). A powerful approach for this qualitative problem is the notion of ranking supermartingales with respect to a given set of invariants. The quantitative problem (probabilistic termination) asks for bounds on the termination probability. A fundamental and conceptual drawback of the existing approaches to address probabilistic termination is that even though the supermartingales consider the probabilistic behavior of the programs, the invariants are obtained completely ignoring the probabilistic aspect. In this work we address the probabilistic termination problem for linear-arithmetic probabilistic programs with nondeterminism. We define the notion of {\em stochastic invariants}, which are constraints along with a probability bound that the constraints hold. We introduce a concept of {\em repulsing supermartingales}. First, we show that repulsing supermartingales can be used to obtain bounds on the probability of the stochastic invariants. Second, we show the effectiveness of repulsing supermartingales in the following three ways: (1)~With a combination of ranking and repulsing supermartingales we can compute lower bounds on the probability of termination; (2)~repulsing supermartingales provide witnesses for refutation of almost-sure termination; and (3)~with a combination of ranking and repulsing supermartingales we can establish persistence properties of probabilistic programs. We also present results on related computational problems and an experimental evaluation of our approach on academic examples.Comment: Full version of a paper published at POPL 2017. 20 page

A Demographic Analysis of the Tithing Behaviour of 2562 Seventh-day Adventists in Northern New South Wales, Australia

An analysis of tithe receipted in Seventh-day Adventist Churches in Northern New South Wales compared to census data reveals that there is a decided difference between the tithing behaviour of the under-fifty-year olds and those over fifty. It is suggested that this explains why there has been an approximately 40% drop in NNSW tithe compared to incomes over the last 25 years

On the relation between Differential Privacy and Quantitative Information Flow

Differential privacy is a notion that has emerged in the community of statistical databases, as a response to the problem of protecting the privacy of the database's participants when performing statistical queries. The idea is that a randomized query satisfies differential privacy if the likelihood of obtaining a certain answer for a database $x$ is not too different from the likelihood of obtaining the same answer on adjacent databases, i.e. databases which differ from $x$ for only one individual. Information flow is an area of Security concerned with the problem of controlling the leakage of confidential information in programs and protocols. Nowadays, one of the most established approaches to quantify and to reason about leakage is based on the R\'enyi min entropy version of information theory. In this paper, we analyze critically the notion of differential privacy in light of the conceptual framework provided by the R\'enyi min information theory. We show that there is a close relation between differential privacy and leakage, due to the graph symmetries induced by the adjacency relation. Furthermore, we consider the utility of the randomized answer, which measures its expected degree of accuracy. We focus on certain kinds of utility functions called "binary", which have a close correspondence with the R\'enyi min mutual information. Again, it turns out that there can be a tight correspondence between differential privacy and utility, depending on the symmetries induced by the adjacency relation and by the query. Depending on these symmetries we can also build an optimal-utility randomization mechanism while preserving the required level of differential privacy. Our main contribution is a study of the kind of structures that can be induced by the adjacency relation and the query, and how to use them to derive bounds on the leakage and achieve the optimal utility

Avoidant Coping Style to High Imminence Threat Is Linked to Higher Anxiety-Like Behavior.

Human studies with self-reported measures have suggested a link between an avoidant coping style and high anxiety. Here, using the common marmoset as a model, we characterize the latent factors underlying behavioral responses of these monkeys towards low and high imminence threat and investigate if a predominantly avoidant behavioral response to high imminence threat is associated with greater anxiety-like behavior in a context of low imminence threat. Exploratory factor analysis (EFA) of the human intruder test of low imminence threat revealed a single factor in which a combination of active vigilance and avoidance responses underpinned anxiety-like behavior. In contrast, two negatively-associated factors were revealed in the model snake test reflecting active and avoidant coping to high imminence threat. Subsequent analysis showed that animals with a predominantly avoidant coping style on the model snake test displayed higher anxiety-like behavior on the human intruder test, findings consistent with those described in humans. Together they illustrate the richness of the behavioral repertoire displayed by marmosets in low and high imminence threatening contexts and the additional insight that factor analysis can provide by identifying the latent factors underlying these complex behavioral datasets. They also highlight the translational value of this approach when studying the neural circuits underlying complex anxiety-like states in this primate model

Women candidates and councillors in Scottish local government, 1974-2012

While significant attention has been paid to the levels of representation of women in both the Westminster Parliament and the Scottish Parliament, much less considered has been given to the position within local government. This article addresses that deficit for Scotland. It shows that for twenty-five years following the reorganisation of local government in Scotland in 1974 there was a slow but relatively steady increase in the numbers of female candidates and councillors, although more recently this appears to have since plateaued somewhat, together with a similar increase in the number of women councillors taking up more senior roles in Scotland’s councils. The article analyses the representation of women in Scottish local government over the period from 1974 to 2012 against the backdrop of significant change in Scotland, including a further restructuring of local government and the introduction of the Single Transferable Vote for council elections, the creation of the Scottish Parliament, the rise of the SNP and the decline of the Conservative Party