14,769 research outputs found
Strategy Synthesis for Autonomous Agents Using PRISM
We present probabilistic models for autonomous agent search and retrieve missions derived from Simulink models for an Unmanned Aerial Vehicle (UAV) and show how probabilistic model checking and the probabilistic model checker PRISM can be used for optimal controller generation. We introduce a sequence of scenarios relevant to UAVs and other autonomous agents such as underwater and ground vehicles. For each scenario we demonstrate how it can be modelled using the PRISM language, give model checking statistics and present the synthesised optimal controllers. We conclude with a discussion of the limitations when using probabilistic model checking and PRISM in this context and what steps can be taken to overcome them. In addition, we consider how the controllers can be returned to the UAV and adapted for use on larger search areas
Probabilistic model checking for strategic equilibria-based decision making:advances and challenges
Game-theoretic concepts have been extensively studied in economics to provide insight into competitive behaviour and strategic decision making. As computing systems increasingly involve concurrently acting autonomous agents, game-theoretic approaches are becoming widespread in computer science as a faithful modelling abstraction. These techniques can be used to reason about the competitive or collaborative behaviour of multiple rational agents with distinct goals or objectives. This paper provides an overview of recent advances in developing a modelling, verification and strategy synthesis framework for concurrent stochastic games implemented in the probabilistic model checker PRISM-games. This is based on a temporal logic that supports finite- and infinite-horizon temporal properties in both a zero-sum and nonzero-sum setting, the latter using Nash and correlated equilibria with respect to two optimality criteria, social welfare and social fairness. We summarise the key concepts, logics and algorithms and the currently available tool support. Future challenges and recent progress in adapting the framework and algorithmic solutions to continuous environments and neural networks are also outlined
Collaborative models for autonomous systems controller synthesis
Funding: UK EPSRC grants EP/N508792/1, EP/N007565 and EC/P51133X/1.We show how detailed simulation models and abstract Markov models can be developed collaboratively to generate and implement effective controllers for autonomous agent search and retrieve missions. We introduce a concrete simulation model of an Unmanned Aerial Vehicle (UAV). We then show how the probabilistic model checker PRISM is used for optimal strategy synthesis for a sequence of scenarios relevant to UAVs and potentially other autonomous agent systems. For each scenario we demonstrate how it can be modelled using PRISM, give model checking statistics and present the synthesised optimal strategies. We then show how our strategies can be returned to the controller for the simulation model and provide experimental results to demonstrate the effectiveness of one such strategy. Finally we explain how our models can be adapted, using symmetry, for use on larger search areas, and demonstrate the feasibility of this approach.Publisher PDFPeer reviewe
Efficient Energy Distribution in a Smart Grid using Multi-Player Games
Algorithms and models based on game theory have nowadays become prominent
techniques for the design of digital controllers for critical systems. Indeed,
such techniques enable automatic synthesis: given a model of the environment
and a property that the controller must enforce, those techniques automatically
produce a correct controller, when it exists. In the present paper, we consider
a class of concurrent, weighted, multi-player games that are well-suited to
model and study the interactions of several agents who are competing for some
measurable resources like energy. We prove that a subclass of those games
always admit a Nash equilibrium, i.e. a situation in which all players play in
such a way that they have no incentive to deviate. Moreover, the strategies
yielding those Nash equilibria have a special structure: when one of the agents
deviate from the equilibrium, all the others form a coalition that will enforce
a retaliation mechanism that punishes the deviant agent. We apply those results
to a real-life case study in which several smart houses that produce their own
energy with solar panels, and can share this energy among them in micro-grid,
must distribute the use of this energy along the day in order to avoid
consuming electricity that must be bought from the global grid. We demonstrate
that our theory allows one to synthesise an efficient controller for these
houses: using penalties to be paid in the utility bill as an incentive, we
force the houses to follow a pre-computed schedule that maximises the
proportion of the locally produced energy that is consumed.Comment: In Proceedings Cassting'16/SynCoP'16, arXiv:1608.0017
Safety-Aware Apprenticeship Learning
Apprenticeship learning (AL) is a kind of Learning from Demonstration
techniques where the reward function of a Markov Decision Process (MDP) is
unknown to the learning agent and the agent has to derive a good policy by
observing an expert's demonstrations. In this paper, we study the problem of
how to make AL algorithms inherently safe while still meeting its learning
objective. We consider a setting where the unknown reward function is assumed
to be a linear combination of a set of state features, and the safety property
is specified in Probabilistic Computation Tree Logic (PCTL). By embedding
probabilistic model checking inside AL, we propose a novel
counterexample-guided approach that can ensure safety while retaining
performance of the learnt policy. We demonstrate the effectiveness of our
approach on several challenging AL scenarios where safety is essential.Comment: Accepted by International Conference on Computer Aided Verification
(CAV) 201
Towards Assume-Guarantee Profiles for Autonomous Vehicles
Rules or specifications for autonomous vehicles are currently formulated on a case-by-case basis, and put together in a rather ad-hoc fashion. As a step towards eliminating this practice, we propose a systematic procedure for generating a set of supervisory specifications for self-driving cars that are 1) associated with a distributed assume-guarantee structure and 2) characterizable by the notion of consistency and completeness. Besides helping autonomous vehicles make better decisions on the road, the assume-guarantee contract structure also helps address the notion of blame when undesirable events occur. We give several game-theoretic examples to demonstrate applicability of our framework
- …