63,512 research outputs found
Lost in Abstraction: Monotonicity in Multi-Threaded Programs (Extended Technical Report)
Monotonicity in concurrent systems stipulates that, in any global state,
extant system actions remain executable when new processes are added to the
state. This concept is not only natural and common in multi-threaded software,
but also useful: if every thread's memory is finite, monotonicity often
guarantees the decidability of safety property verification even when the
number of running threads is unknown. In this paper, we show that the act of
obtaining finite-data thread abstractions for model checking can be at odds
with monotonicity: Predicate-abstracting certain widely used monotone software
results in non-monotone multi-threaded Boolean programs - the monotonicity is
lost in the abstraction. As a result, well-established sound and complete
safety checking algorithms become inapplicable; in fact, safety checking turns
out to be undecidable for the obtained class of unbounded-thread Boolean
programs. We demonstrate how the abstract programs can be modified into
monotone ones, without affecting safety properties of the non-monotone
abstraction. This significantly improves earlier approaches of enforcing
monotonicity via overapproximations
A finite simulation method in a non-deterministic call-by-need calculus with letrec, constructors and case
The paper proposes a variation of simulation for checking and proving contextual equivalence in a non-deterministic call-by-need lambda-calculus with constructors, case, seq, and a letrec with cyclic dependencies. It also proposes a novel method to prove its correctness. The calculus' semantics is based on a small-step rewrite semantics and on may-convergence. The cyclic nature of letrec bindings, as well as non-determinism, makes known approaches to prove that simulation implies contextual equivalence, such as Howe's proof technique, inapplicable in this setting. The basic technique for the simulation as well as the correctness proof is called pre-evaluation, which computes a set of answers for every closed expression. If simulation succeeds in finite computation depth, then it is guaranteed to show contextual preorder of expressions
Soft constraint abstraction based on semiring homomorphism
The semiring-based constraint satisfaction problems (semiring CSPs), proposed
by Bistarelli, Montanari and Rossi \cite{BMR97}, is a very general framework of
soft constraints. In this paper we propose an abstraction scheme for soft
constraints that uses semiring homomorphism. To find optimal solutions of the
concrete problem, the idea is, first working in the abstract problem and
finding its optimal solutions, then using them to solve the concrete problem.
In particular, we show that a mapping preserves optimal solutions if and only
if it is an order-reflecting semiring homomorphism. Moreover, for a semiring
homomorphism and a problem over , if is optimal in
, then there is an optimal solution of such that
has the same value as in .Comment: 18 pages, 1 figur
Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition
This paper presents the MAXQ approach to hierarchical reinforcement learning
based on decomposing the target Markov decision process (MDP) into a hierarchy
of smaller MDPs and decomposing the value function of the target MDP into an
additive combination of the value functions of the smaller MDPs. The paper
defines the MAXQ hierarchy, proves formal results on its representational
power, and establishes five conditions for the safe use of state abstractions.
The paper presents an online model-free learning algorithm, MAXQ-Q, and proves
that it converges wih probability 1 to a kind of locally-optimal policy known
as a recursively optimal policy, even in the presence of the five kinds of
state abstraction. The paper evaluates the MAXQ representation and MAXQ-Q
through a series of experiments in three domains and shows experimentally that
MAXQ-Q (with state abstractions) converges to a recursively optimal policy much
faster than flat Q learning. The fact that MAXQ learns a representation of the
value function has an important benefit: it makes it possible to compute and
execute an improved, non-hierarchical policy via a procedure similar to the
policy improvement step of policy iteration. The paper demonstrates the
effectiveness of this non-hierarchical execution experimentally. Finally, the
paper concludes with a comparison to related work and a discussion of the
design tradeoffs in hierarchical reinforcement learning.Comment: 63 pages, 15 figure
A Notion of Dynamic Interface for Depth-Bounded Object-Oriented Packages
Programmers using software components have to follow protocols that specify
when it is legal to call particular methods with particular arguments. For
example, one cannot use an iterator over a set once the set has been changed
directly or through another iterator. We formalize the notion of dynamic
package interfaces (DPI), which generalize state-machine interfaces for single
objects, and give an algorithm to statically compute a sound abstraction of a
DPI. States of a DPI represent (unbounded) sets of heap configurations and
edges represent the effects of method calls on the heap. We introduce a novel
heap abstract domain based on depth-bounded systems to deal with potentially
unboundedly many objects and the references among them. We have implemented our
algorithm and show that it is effective in computing representations of common
patterns of package usage, such as relationships between viewer and label,
container and iterator, and JDBC statements and cursors
- âŚ