Mean-payoff Automaton Expressions

Quantitative languages are an extension of boolean languages that assign to each word a real number. Mean-payoff automata are finite automata with numerical weights on transitions that assign to each infinite path the long-run average of the transition weights. When the mode of branching of the automaton is deterministic, nondeterministic, or alternating, the corresponding class of quantitative languages is not robust as it is not closed under the pointwise operations of max, min, sum, and numerical complement. Nondeterministic and alternating mean-payoff automata are not decidable either, as the quantitative generalization of the problems of universality and language inclusion is undecidable. We introduce a new class of quantitative languages, defined by mean-payoff automaton expressions, which is robust and decidable: it is closed under the four pointwise operations, and we show that all decision problems are decidable for this class. Mean-payoff automaton expressions subsume deterministic mean-payoff automata, and we show that they have expressive power incomparable to nondeterministic and alternating mean-payoff automata. We also present for the first time an algorithm to compute distance between two quantitative languages, and in our case the quantitative languages are given as mean-payoff automaton expressions

Spatial patterns and scale freedom in a Prisoner's Dilemma cellular automata with Pavlovian strategies

A cellular automaton in which cells represent agents playing the Prisoner's Dilemma (PD) game following the simple "win-stay, loose-shift" strategy is studied. Individuals with binary behavior, such as they can either cooperate (C) or defect (D), play repeatedly with their neighbors (Von Neumann's and Moore's neighborhoods). Their utilities in each round of the game are given by a rescaled payoff matrix described by a single parameter Tau, which measures the ratio of 'temptation to defect' to 'reward for cooperation'. Depending on the region of the parameter space Tau, the system self-organizes - after a transient - into dynamical equilibrium states characterized by different definite fractions of C agents (2 states for the Von Neumann neighborhood and 4 for Moore neighborhood). For some ranges of Tau the cluster size distributions, the power spectrums P(f) and the perimeter-area curves follow power-law scalings. Percolation below threshold is also found for D agent clusters. We also analyze the asynchronous dynamics version of this model and compare results.Comment: Accepted for publication in JSTA

Comparator automata in quantitative verification

The notion of comparison between system runs is fundamental in formal verification. This concept is implicitly present in the verification of qualitative systems, and is more pronounced in the verification of quantitative systems. In this work, we identify a novel mode of comparison in quantitative systems: the online comparison of the aggregate values of two sequences of quantitative weights. This notion is embodied by {\em comparator automata} ({\em comparators}, in short), a new class of automata that read two infinite sequences of weights synchronously and relate their aggregate values. We show that {aggregate functions} that can be represented with B\"uchi automaton result in comparators that are finite-state and accept by the B\"uchi condition as well. Such {\em $\omega$-regular comparators} further lead to generic algorithms for a number of well-studied problems, including the quantitative inclusion and winning strategies in quantitative graph games with incomplete information, as well as related non-decision problems, such as obtaining a finite representation of all counterexamples in the quantitative inclusion problem. We study comparators for two aggregate functions: discounted-sum and limit-average. We prove that the discounted-sum comparator is $\omega$-regular iff the discount-factor is an integer. Not every aggregate function, however, has an $\omega$-regular comparator. Specifically, we show that the language of sequence-pairs for which limit-average aggregates exist is neither $\omega$-regular nor $\omega$-context-free. Given this result, we introduce the notion of {\em prefix-average} as a relaxation of limit-average aggregation, and show that it admits $\omega$-context-free comparators

On the Role of External Constraints in a Spatially Extended Evolutionary Prisoner's Dilemma Game

We study the emergency of mutual cooperation in evolutionary prisoner's dilemma games when the players are located on a square lattice. The players can choose one of the three strategies: cooperation (C), defection (D) or "tit for tat" (T), and their total payoffs come from games with the nearest neighbors. During the random sequential updates the players adopt one of their neighboring strategies if the chosen neighbor has higher payoff. We compare the effect of two types of external constraints added to the Darwinian evolutionary processes. In both cases the strategy of a randomly chosen player is replaced with probability P by another strategy. In the first case, the strategy is replaced by a randomly chosen one among the two others, while in the second case the new strategy is always C. Using generalized mean-field approximations and Monte Carlo simulations the strategy concentrations are evaluated in the stationary state for different strength of external constraints characterized by the probability P.Comment: 19 pages, 10 figure

Analysis of reinforcement learning strategies for predation in a mimic-model prey environment

In this paper we propose a mathematical learning model for a stochastic automaton simulating the behaviour of a predator operating in a random environment occupied by two types of prey: palatable mimics and unpalatable models. Specifically, a well known linear reinforcement learning algorithm is used to update the probabilities of the two actions, eat prey or ignore prey, at every random encounter. Each action elicits a probabilistic response from the environment that can be either favorable or unfavourable. We analyse both fixed and varying stochastic responses for the system. The basic approach of mimicry is defined and a short review of relevant previous approaches in the literature is given. Finally, the conditions for continuous predator performance improvement are explicitly formulated and precise definitions of predatory efficiency and mimicry efficiency are also provided