3,337 research outputs found
Lost in Abstraction: Monotonicity in Multi-Threaded Programs (Extended Technical Report)
Monotonicity in concurrent systems stipulates that, in any global state,
extant system actions remain executable when new processes are added to the
state. This concept is not only natural and common in multi-threaded software,
but also useful: if every thread's memory is finite, monotonicity often
guarantees the decidability of safety property verification even when the
number of running threads is unknown. In this paper, we show that the act of
obtaining finite-data thread abstractions for model checking can be at odds
with monotonicity: Predicate-abstracting certain widely used monotone software
results in non-monotone multi-threaded Boolean programs - the monotonicity is
lost in the abstraction. As a result, well-established sound and complete
safety checking algorithms become inapplicable; in fact, safety checking turns
out to be undecidable for the obtained class of unbounded-thread Boolean
programs. We demonstrate how the abstract programs can be modified into
monotone ones, without affecting safety properties of the non-monotone
abstraction. This significantly improves earlier approaches of enforcing
monotonicity via overapproximations
Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition
This paper presents the MAXQ approach to hierarchical reinforcement learning
based on decomposing the target Markov decision process (MDP) into a hierarchy
of smaller MDPs and decomposing the value function of the target MDP into an
additive combination of the value functions of the smaller MDPs. The paper
defines the MAXQ hierarchy, proves formal results on its representational
power, and establishes five conditions for the safe use of state abstractions.
The paper presents an online model-free learning algorithm, MAXQ-Q, and proves
that it converges wih probability 1 to a kind of locally-optimal policy known
as a recursively optimal policy, even in the presence of the five kinds of
state abstraction. The paper evaluates the MAXQ representation and MAXQ-Q
through a series of experiments in three domains and shows experimentally that
MAXQ-Q (with state abstractions) converges to a recursively optimal policy much
faster than flat Q learning. The fact that MAXQ learns a representation of the
value function has an important benefit: it makes it possible to compute and
execute an improved, non-hierarchical policy via a procedure similar to the
policy improvement step of policy iteration. The paper demonstrates the
effectiveness of this non-hierarchical execution experimentally. Finally, the
paper concludes with a comparison to related work and a discussion of the
design tradeoffs in hierarchical reinforcement learning.Comment: 63 pages, 15 figure
Logic Programming Applications: What Are the Abstractions and Implementations?
This article presents an overview of applications of logic programming,
classifying them based on the abstractions and implementations of logic
languages that support the applications. The three key abstractions are join,
recursion, and constraint. Their essential implementations are for-loops, fixed
points, and backtracking, respectively. The corresponding kinds of applications
are database queries, inductive analysis, and combinatorial search,
respectively. We also discuss language extensions and programming paradigms,
summarize example application problems by application areas, and touch on
example systems that support variants of the abstractions with different
implementations
Sequentializing Parameterized Programs
We exhibit assertion-preserving (reachability preserving) transformations
from parameterized concurrent shared-memory programs, under a k-round
scheduling of processes, to sequential programs. The salient feature of the
sequential program is that it tracks the local variables of only one thread at
any point, and uses only O(k) copies of shared variables (it does not use extra
counters, not even one counter to keep track of the number of threads).
Sequentialization is achieved using the concept of a linear interface that
captures the effect an unbounded block of processes have on the shared state in
a k-round schedule. Our transformation utilizes linear interfaces to
sequentialize the program, and to ensure the sequential program explores only
reachable states and preserves local invariants.Comment: In Proceedings FIT 2012, arXiv:1207.348
- …