5,402 research outputs found
Sample Efficient Policy Search for Optimal Stopping Domains
Optimal stopping problems consider the question of deciding when to stop an
observation-generating process in order to maximize a return. We examine the
problem of simultaneously learning and planning in such domains, when data is
collected directly from the environment. We propose GFSE, a simple and flexible
model-free policy search method that reuses data for sample efficiency by
leveraging problem structure. We bound the sample complexity of our approach to
guarantee uniform convergence of policy value estimates, tightening existing
PAC bounds to achieve logarithmic dependence on horizon length for our setting.
We also examine the benefit of our method against prevalent model-based and
model-free approaches on 3 domains taken from diverse fields.Comment: To appear in IJCAI-201
Array operators using multiple dispatch: a design methodology for array implementations in dynamic languages
Arrays are such a rich and fundamental data type that they tend to be built
into a language, either in the compiler or in a large low-level library.
Defining this functionality at the user level instead provides greater
flexibility for application domains not envisioned by the language designer.
Only a few languages, such as C++ and Haskell, provide the necessary power to
define -dimensional arrays, but these systems rely on compile-time
abstraction, sacrificing some flexibility. In contrast, dynamic languages make
it straightforward for the user to define any behavior they might want, but at
the possible expense of performance.
As part of the Julia language project, we have developed an approach that
yields a novel trade-off between flexibility and compile-time analysis. The
core abstraction we use is multiple dispatch. We have come to believe that
while multiple dispatch has not been especially popular in most kinds of
programming, technical computing is its killer application. By expressing key
functions such as array indexing using multi-method signatures, a surprising
range of behaviors can be obtained, in a way that is both relatively easy to
write and amenable to compiler analysis. The compact factoring of concerns
provided by these methods makes it easier for user-defined types to behave
consistently with types in the standard library.Comment: 6 pages, 2 figures, workshop paper for the ARRAY '14 workshop, June
11, 2014, Edinburgh, United Kingdo
Efficient Open World Reasoning for Planning
We consider the problem of reasoning and planning with incomplete knowledge
and deterministic actions. We introduce a knowledge representation scheme
called PSIPLAN that can effectively represent incompleteness of an agent's
knowledge while allowing for sound, complete and tractable entailment in
domains where the set of all objects is either unknown or infinite. We present
a procedure for state update resulting from taking an action in PSIPLAN that is
correct, complete and has only polynomial complexity. State update is performed
without considering the set of all possible worlds corresponding to the
knowledge state. As a result, planning with PSIPLAN is done without direct
manipulation of possible worlds. PSIPLAN representation underlies the PSIPOP
planning algorithm that handles quantified goals with or without exceptions
that no other domain independent planner has been shown to achieve. PSIPLAN has
been implemented in Common Lisp and used in an application on planning in a
collaborative interface.Comment: 39 pages, 13 figures. to appear in Logical Methods in Computer
Scienc
- …