183,635 research outputs found
Benchmarks for Parity Games (extended version)
We propose a benchmark suite for parity games that includes all benchmarks
that have been used in the literature, and make it available online. We give an
overview of the parity games, including a description of how they have been
generated. We also describe structural properties of parity games, and using
these properties we show that our benchmarks are representative. With this work
we provide a starting point for further experimentation with parity games.Comment: The corresponding tool and benchmarks are available from
https://github.com/jkeiren/paritygame-generator. This is an extended version
of the paper that has been accepted for FSEN 201
Multiscale Markov Decision Problems: Compression, Solution, and Transfer Learning
Many problems in sequential decision making and stochastic control often have
natural multiscale structure: sub-tasks are assembled together to accomplish
complex goals. Systematically inferring and leveraging hierarchical structure,
particularly beyond a single level of abstraction, has remained a longstanding
challenge. We describe a fast multiscale procedure for repeatedly compressing,
or homogenizing, Markov decision processes (MDPs), wherein a hierarchy of
sub-problems at different scales is automatically determined. Coarsened MDPs
are themselves independent, deterministic MDPs, and may be solved using
existing algorithms. The multiscale representation delivered by this procedure
decouples sub-tasks from each other and can lead to substantial improvements in
convergence rates both locally within sub-problems and globally across
sub-problems, yielding significant computational savings. A second fundamental
aspect of this work is that these multiscale decompositions yield new transfer
opportunities across different problems, where solutions of sub-tasks at
different levels of the hierarchy may be amenable to transfer to new problems.
Localized transfer of policies and potential operators at arbitrary scales is
emphasized. Finally, we demonstrate compression and transfer in a collection of
illustrative domains, including examples involving discrete and continuous
statespaces.Comment: 86 pages, 15 figure
- …