2,805 research outputs found
Stochastic Shortest Path with Energy Constraints in POMDPs
We consider partially observable Markov decision processes (POMDPs) with a
set of target states and positive integer costs associated with every
transition. The traditional optimization objective (stochastic shortest path)
asks to minimize the expected total cost until the target set is reached. We
extend the traditional framework of POMDPs to model energy consumption, which
represents a hard constraint. The energy levels may increase and decrease with
transitions, and the hard constraint requires that the energy level must remain
positive in all steps till the target is reached. First, we present a novel
algorithm for solving POMDPs with energy levels, developing on existing POMDP
solvers and using RTDP as its main method. Our second contribution is related
to policy representation. For larger POMDP instances the policies computed by
existing solvers are too large to be understandable. We present an automated
procedure based on machine learning techniques that automatically extracts
important decisions of the policy allowing us to compute succinct human
readable policies. Finally, we show experimentally that our algorithm performs
well and computes succinct policies on a number of POMDP instances from the
literature that were naturally enhanced with energy levels.Comment: Technical report accompanying a paper published in proceedings of
AAMAS 201
ARBEITSBEREICH WISSENSBASIERTE SYSTEME TEAM PROGRAMMING IN GOLOG UNDER PARTIAL OBSERVABILITY
Abstract. We present and explore the agent programming language TEAMGOLOG, which is a novel approach to programming a team of cooperative agents under partial observability. Every agent is associated with a partial control program in Golog, which is completed by the TEAMGOLOG interpreter in an optimal way by assuming a decision-theoretic semantics. The approach is based on the key concepts of a synchronization state and a communication state, which allow the agents to passively resp. actively coordinate their behavior, while keeping their belief states, observations, and activities invisible to the other agents. We show the practical usefulness of the TEAMGOLOG approach in a rescue simulated domain. We describe the algorithms behind the TEAMGOLOG interpreter and provide a prototype implementation. We also show through experimental results that the TEAMGOLOG approach outperforms a standard greedy one in the rescue simulated domain
Severity-sensitive norm-governed multi-agent planning
This research was funded by Selex ES. The software developed during this research, including the norm analysis and planning algorithms, the simulator and harbour protection scenario used during evaluation is freely available from doi:10.5258/SOTON/D0139Peer reviewedPublisher PD
Reasoning about LTL Synthesis over finite and infinite games
In the last few years, research formal methods for the analysis and the verification of properties of systems has increased greatly. A meaningful contribution in this area has been given by algorithmic methods developed in the context of synthesis. The basic idea is simple and appealing: instead of developing a system and verifying that it satisfies its specification, we look for an automated procedure that, given the specification returns a system that is correct by construction. Synthesis of reactive systems is one of the most popular variants of this problem, in which we want to synthesize a system characterized by an ongoing interaction with the environment. In this setting, large effort has been devoted to analyze specifications given as formulas of linear temporal logic, i.e., LTL synthesis.
Traditional approaches to LTL synthesis rely on transforming the LTL specification into parity deterministic automata, and then to parity games, for which a so-called winning region is computed. Computing such an automaton is, in the worst-case, double-exponential in the size of the LTL formula, and this becomes a computational bottleneck in using the synthesis process in practice.
The first part of this thesis is devoted to improve the solution of parity games as they are used in solving LTL synthesis, trying to give efficient techniques, in terms of running time and space consumption, for solving parity games. We start with the study and the implementation of an automata-theoretic technique to solve parity games. More precisely, we consider an algorithm introduced by Kupferman and Vardi that solves a parity game by solving the emptiness problem of a corresponding alternating parity automaton. Our empirical evaluation demonstrates that this algorithm outperforms other algorithms when the game has a small number of priorities relative to the size of the game. In many concrete applications, we do indeed end up with parity games
where the number of priorities is relatively small. This makes the new algorithm quite useful in practice.
We then provide a broad investigation of the symbolic approach for solving parity games. Specifically, we implement in a fresh tool, called SPGSolver, four symbolic algorithms to solve parity games and compare their performances to the corresponding explicit versions for different classes of games. By means of benchmarks, we show that for random games, even for constrained random games, explicit algorithms actually perform better than symbolic algorithms. The situation changes, however, for structured games, where symbolic algorithms seem to have the advantage. This suggests that when evaluating algorithms for parity-game solving, it would be useful to have real benchmarks and not only random benchmarks, as the common practice has been.
LTL synthesis has been largely investigated also in artificial intelligence, and specifically in
automated planning. Indeed, LTL synthesis corresponds to fully observable nondeterministic planning in which the domain is given compactly and the goal is an LTL formula, that in turn is related to two-player games with LTL goals. Finding a strategy for these games means to synthesize a plan for the planning problem. The last part of this thesis is then dedicated to investigate LTL synthesis under this different view. In particular, we study a generalized form of planning under partial observability, in which we have multiple, possibly infinitely many, planning domains with the same actions and observations, and goals expressed over observations, which are possibly temporally extended. By building on work on two-player games with imperfect information in the Formal Methods literature, we devise a general technique, generalizing the belief-state construction, to remove partial observability. This reduces the planning problem to a game of perfect information with a tight correspondence between plans and strategies. Then we instantiate the technique and solve some generalized planning problems
Conformant Planning via Symbolic Model Checking
We tackle the problem of planning in nondeterministic domains, by presenting
a new approach to conformant planning. Conformant planning is the problem of
finding a sequence of actions that is guaranteed to achieve the goal despite
the nondeterminism of the domain. Our approach is based on the representation
of the planning domain as a finite state automaton. We use Symbolic Model
Checking techniques, in particular Binary Decision Diagrams, to compactly
represent and efficiently search the automaton. In this paper we make the
following contributions. First, we present a general planning algorithm for
conformant planning, which applies to fully nondeterministic domains, with
uncertainty in the initial condition and in action effects. The algorithm is
based on a breadth-first, backward search, and returns conformant plans of
minimal length, if a solution to the planning problem exists, otherwise it
terminates concluding that the problem admits no conformant solution. Second,
we provide a symbolic representation of the search space based on Binary
Decision Diagrams (BDDs), which is the basis for search techniques derived from
symbolic model checking. The symbolic representation makes it possible to
analyze potentially large sets of states and transitions in a single
computation step, thus providing for an efficient implementation. Third, we
present CMBP (Conformant Model Based Planner), an efficient implementation of
the data structures and algorithm described above, directly based on BDD
manipulations, which allows for a compact representation of the search layers
and an efficient implementation of the search steps. Finally, we present an
experimental comparison of our approach with the state-of-the-art conformant
planners CGP, QBFPLAN and GPT. Our analysis includes all the planning problems
from the distribution packages of these systems, plus other problems defined to
stress a number of specific factors. Our approach appears to be the most
effective: CMBP is strictly more expressive than QBFPLAN and CGP and, in all
the problems where a comparison is possible, CMBP outperforms its competitors,
sometimes by orders of magnitude
Verification and Control of Partially Observable Probabilistic Real-Time Systems
We propose automated techniques for the verification and control of
probabilistic real-time systems that are only partially observable. To formally
model such systems, we define an extension of probabilistic timed automata in
which local states are partially visible to an observer or controller. We give
a probabilistic temporal logic that can express a range of quantitative
properties of these models, relating to the probability of an event's
occurrence or the expected value of a reward measure. We then propose
techniques to either verify that such a property holds or to synthesise a
controller for the model which makes it true. Our approach is based on an
integer discretisation of the model's dense-time behaviour and a grid-based
abstraction of the uncountable belief space induced by partial observability.
The latter is necessarily approximate since the underlying problem is
undecidable, however we show how both lower and upper bounds on numerical
results can be generated. We illustrate the effectiveness of the approach by
implementing it in the PRISM model checker and applying it to several case
studies, from the domains of computer security and task scheduling
- …