Strategy Logic with Imperfect Information

We introduce an extension of Strategy Logic for the imperfect-information setting, called SLii, and study its model-checking problem. As this logic naturally captures multi-player games with imperfect information, the problem turns out to be undecidable. We introduce a syntactical class of "hierarchical instances" for which, intuitively, as one goes down the syntactic tree of the formula, strategy quantifications are concerned with finer observations of the model. We prove that model-checking SLii restricted to hierarchical instances is decidable. This result, because it allows for complex patterns of existential and universal quantification on strategies, greatly generalises previous ones, such as decidability of multi-player games with imperfect information and hierarchical observations, and decidability of distributed synthesis for hierarchical systems. To establish the decidability result, we introduce and study QCTL*ii, an extension of QCTL* (itself an extension of CTL* with second-order quantification over atomic propositions) by parameterising its quantifiers with observations. The simple syntax of QCTL* ii allows us to provide a conceptually neat reduction of SLii to QCTL*ii that separates concerns, allowing one to forget about strategies and players and focus solely on second-order quantification. While the model-checking problem of QCTL*ii is, in general, undecidable, we identify a syntactic fragment of hierarchical formulas and prove, using an automata-theoretic approach, that it is decidable. The decidability result for SLii follows since the reduction maps hierarchical instances of SLii to hierarchical formulas of QCTL*ii

CoInDiVinE: Parallel Distributed Model Checker for Component-Based Systems

CoInDiVinE is a tool for parallel distributed model checking of interactions among components in hierarchical component-based systems. The tool extends the DiVinE framework with a new input language (component-interaction automata) and a property specification logic (CI-LTL). As the language differs from the input language of DiVinE, our tool employs a new state space generation algorithm that also supports partial order reduction. Experiments indicate that the tool has good scaling properties when run in parallel setting.Comment: In Proceedings PDMC 2011, arXiv:1111.006

Automata guided hierarchical reinforcement learning for zero-shot skill composition

An obstacle that prevents the wide adoption of (deep) reinforcement learning (RL) in control systems is its need for a large amount of interactions with the environment in order to master a skill. The learned skill usually generalizes poorly across domains and re-training is often necessary when presented with a new task. We present a framework that combines methods in formal methods with hierarchical reinforcement learning (HRL). The set of techniques we provide allows for convenient specification of tasks with complex logic, learn hierarchical policies (meta-controller and low-level controllers) with well-defined intrinsic rewards using any RL methods and is able to construct new skills from existing ones without additional learning. We evaluate the proposed methods in a simple grid world simulation as well as simulation on a Baxter robot

Bonabeau hierarchy models revisited

What basic processes generate hierarchy in a collective? The Bonabeau model provides us a simple mechanism based on randomness which develops self-organization through both winner/looser effects and relaxation process. A phase transition between egalitarian and hierarchic states has been found both analytically and numerically in previous works. In this paper we present a different approach: by means of a discrete scheme we develop a mean field approximation that not only reproduces the phase transition but also allows us to characterize the complexity of hierarchic phase. In the same philosophy, we study a new version of the Bonabeau model, developed by Stauffer et al. Several previous works described numerically the presence of a similar phase transition in this later version. We find surprising results in this model that can be interpreted properly as the non-existence of phase transition in this version of Bonabeau model, but a changing in fixed point structure