115 research outputs found
Adaptive Envelope MDPs for Relational Equivalence-based Planning
We describe a method to use structured representations of the environmentâs dynamics to constrain and speed up the planning process. Given a problem domain described in a probabilistic logical description language, we develop an anytime technique that incrementally improves on an initial, partial policy. This partial solution is found by ï¬rst reducing the number of predicates needed to represent a relaxed version of the problem to a minimum, and then dynamically partitioning the action space into a set of equivalence classes with respect to this minimal representation. Our approach uses the envelope MDP framework, which creates a Markov decision process out of a subset of the full state space as de- termined by the initial partial solution. This strategy permits an agent to begin acting within a restricted part of the full state space and to expand its envelope judiciously as resources permit
Computing action equivalences for planning under time-constraints
In order for autonomous artificial decision-makers to solverealistic tasks, they need to deal with the dual problems of searching throughlarge state and action spaces under time pressure.We study the problem of planning in domains with lots of objects. Structuredrepresentations of action can help provide guidance when the number of actionchoices and size of the state space is large.We show how structured representations ofaction effects can help us partition the action space in to a smallerset of approximate equivalence classes. Then, the pared-downaction space can be used to identify a useful subset of the state space in whichto search for a solution. As computational resources permit, we thenallow ourselves to elaborate the original solution. This kind of analysisallows us to collapse the action space and permits faster planning in muchlarger domains than before
Relevance Grounding for Planning in Relational Domains
Abstract. Probabilistic relational models are an efficient way to learn and represent the dynamics in realistic environments consisting of many objects. Autonomous intelligent agents that ground this representation for all objects need to plan in exponentially large state spaces and large sets of stochastic actions. A key insight for computational efficiency is that successful planning typically involves only a small subset of relevant objects. In this paper, we introduce a probabilistic model to represent planning with subsets of objects and provide a definition of object relevance. Our definition is sufficient to prove consistency between repeated planning in partially grounded models restricted to relevant objects and planning in the fully grounded model. We propose an algorithm that exploits object relevance to plan efficiently in complex domains. Empirical results in a simulated 3D blocksworld with an articulated manipulator and realistic physics prove the effectiveness of our approach.
FluCaP: A Heuristic Search Planner for First-Order MDPs
We present a heuristic search algorithm for solving first-order Markov
Decision Processes (FOMDPs). Our approach combines first-order state
abstraction that avoids evaluating states individually, and heuristic search
that avoids evaluating all states. Firstly, in contrast to existing systems,
which start with propositionalizing the FOMDP and then perform state
abstraction on its propositionalized version we apply state abstraction
directly on the FOMDP avoiding propositionalization. This kind of abstraction
is referred to as first-order state abstraction. Secondly, guided by an
admissible heuristic, the search is restricted to those states that are
reachable from the initial state. We demonstrate the usefulness of the above
techniques for solving FOMDPs with a system, referred to as FluCaP (formerly,
FCPlanner), that entered the probabilistic track of the 2004 International
Planning Competition (IPC2004) and demonstrated an advantage over other
planners on the problems represented in first-order terms
- …