Gerevini, Alfonso

Saetti, Alessandro

Vallati, Mauro

English

University of Huddersfield Repository

University of Huddersfield RepositoryGerevini, Alfonso, Saetti, Alessandro and Vallati, MauroOptimal SAT-based Planning with Macro-actions and Learned HorizonsOriginal CitationGerevini, Alfonso, Saetti, Alessandro and Vallati, Mauro (2010) Optimal SAT-based Planning with Macro-actions and Learned Horizons. In: The 28th Workshop of the UK Planning and Scheduling Special Interest Group (PLANSIG), 1st - 3rd December 2010, Brescia, Italy.This version is available at http://eprints.hud.ac.uk/15368/The University Repository is a digital collection of the research output of theUniversity, available on Open Access. Copyright and Moral Rights for the itemson this site are retained by the individual author and/or other copyright owners.Users may access full items free of charge; copies of full text items generallycan be reproduced, displayed or performed and given to third parties in anyformat or medium for personal research or study, educational or not-for-profitpurposes without prior permission or charge, provided:• The authors, title and full bibliographic details is credited in any copy;• A hyperlink and/or URL is included for the original metadata page; and• The content is not changed in any way.For more information, including our policy and submission procedure, pleasecontact the Repository Team at: E.mailbox@hud.ac.uk.http://eprints.hud.ac.uk/Optimal SAT-based Planning with Macro-actions and Learned HorizonsAlfonso E. Gerevini and Alessandro Saetti and Mauro VallatiDipartimento di Ingegneria dell’InformazioneUniversità degli Studi di BresciaVia Branze 38, 25123 Brescia, Italy{gerevini,saetti,mauro.vallati}@ing.unibs.itIntroductionPlanning as propositional satisfiability (SAT) is a powerfulapproach for computing optimal plans in terms of Graph-plan plan length. SatPlan (Kautz and Selman 1992) is oneof the most popular and efficient planning system adoptingthis approach. First, it computes a lower bound k of theoptimal plan length. Then, using k as the planning hori-zon, i.e., a fixed time step after which actions cannot be exe-cuted, it translates the planning problem into a SAT problemΠ, which is then solved by an existing SAT solver. If Π issatisfiable, then the assignment to propositional fluents sat-isfying the SAT problem is translated into a plan of actionsthat is a solution of the original planning problem. Other-wise (Π is unsatisfiable), the process is repeated using anincreased value of k. A critical weakness of the approach isthat often the initial value of k is much less than the optimalplan length, and hence many unsolvable SAT problems canbe generated and processed.The performance of a planning system is typically af-fected by the structure of the search space, which depends onthe considered planning domain. In many domains, the plan-ning performance can be improved by deriving and exploit-ing some knowledge about the domain structure that is notexplicitly encoded in the input domain formalization. Wellknown examples of such knowledge are macro actions (e.g.,(Botea et al. 2005; Newton et al. 2007)). A macro-action isa sequence of domain actions that can be planned at one timelike a single action. Using macro-actions the planning pro-cess is often faster, but the length of the computed solutionplan can be worse than optimal.In this paper, we propose a SAT-based optimal planner,called MacroSatPlan, which exploits two types of knowl-edge learned for a given domain to speedup the SAT solving:(i) a predictive model based on some problem features esti-mating the optimal plan length, and (ii) useful sets of learnedmacro-actions.A preliminary experimental analysis shows that thelearned knowledge is useful for speeding up the computa-tion of the optimal solution of the planning problem.Architecture of MacroSatPlanThe architecture of MacroSatPlan, sketched in Figure 2,consists of three main modules, briefly described below.Solution planDomain andtest problemtraining probsDomain andLearningPre-processingMacrosSatisfiable orPlanningUnsatisfiableSAT Encoding pirPlan lengthpio pirMiniSAT modified to handle macro actionsMacro-FF SatPlan WEKA FFFFSAT encoderFigure 1: A sketch of MacroSatPlan’s architecture. pio andpir indicate an optimal plan and a relaxed plan, respectively.Learning module. This module consists of four main com-ponents: the 2006 version of SatPlan (Kautz, Selman, andHoffmann 2006), a modified version of planner FF (Hoff-mann and Nebel 2001), Macro-FF (Botea et al. 2005), andthe machine learning toolWEKA (Witten and Frank 2005).FF is a forward search planner which exploits GraphPlanfor computing a relaxed plan pi achieving the problem goalsfrom the successor states. The number of actions in pi is anestimate of the distance from the successor states to the goalstates. We use a revised version of FF that only computesthe length of the relaxed plan derived from the initial stateof the planning problem, without computing a solution for it.WEKA is a well-known tool for learning predictive mod-els. Given a set of training problems for domain D, WEKAis used to identify a predictive model of the optimal planlength for a given problem in domain D from (i) the lengthof the optimal plan computed by SatPlan, (ii) some pre-identified features of the planning problem, and (iii) thelength of the relaxed plan computed by FF. The optimalplan length predicted byWEKA is subsequently used by theSAT encoder of the planning problem as the initial valueof the planning horizon. It is important to note that, differ-ently from SatPlan, in our approach the initial horizon canbe higher than the optimal plan length.Macro-FF (Botea et al. 2005) is a planning system com-puting macro-actions and using them for solving planningproblems. The method implemented into theMacro-FF sys-tem computes the macros by analyzing the solutions of a setof training problem instances, so that the macros that appearfrequently and that reduce the required search effort signifi-cantly are preferred. The computed macros are subsequentlyused by a modified version of SAT-solverMiniSAT (Eèn andSörensson 2003) during the planning phase.Preprocessing module. This module consists of two exist-ing systems: the modified version of FF and the SAT en-coder used by SatPlan to compile a planning problem into aSAT-problem.The compilation of the original planning problem (with-out macros) is done using as horizon the predicted planlength value. It is important to note that if the domaindefining the input planning problem were extended with themacro-actions computed by the learning module, the planderived from the solution of the SAT solver could be sub-optimal.The input of the planning module includes the SAT en-coding, together with the computed macro-actions and therelaxed plan computed by FF for achieving the goals fromthe initial state of the planning problem.Planning module. This module consists of a variant ofMiniSAT that can exploit the macro-actions computed bythe learning module to guide the search during SAT solving.At each search step, the original version of MiniSAT se-lects an unassigned variable, and sets it to the false value.Then, this decision is in turn propagated by unit propagation.As soon as a clause becomes unary under the current assign-ment, the remaining literal in the clause is set to true andthis decision is propagated, possibly reducing other clausesto unary clauses. The propagation process continues until nomore information can be propagated. If a conflict is encoun-tered (all literals of a clause are false), a “conflict clause”is constructed and added to the SAT problem. The deci-sions made are retracted by backtracking until the conflictclause becomes unary; this unary clause is propagated, andthe search process continues.The modified version of MiniSAT assigns true to the se-lected variable, instead of false, preferring unassigned vari-ables belonging to the most promising macro-actions. Theseare macro actions involving actions in the relaxed plan pre-viously computed by FF. Ties are broken using some sec-ondary criteria, such as the ratio between the number of vari-ables with true value and the size of the macro, and the timestep of the first action forming the macro.If the SAT encoding of the planning problem with pre-dicted horizon t is initially solvable, the process is repeatedwith horizon t−1, and so on, until a horizon q < t for whichthe SAT encoding is unsolvable has been identified. Other-wise, the process is repeated with an increased value of t,and it terminates when a solvable SAT encoding is gener-ated. The optimal solution is derived from the last computedassignment satisfying the SAT problem. 0.1 1 10 100 1000 0  5  10  15  20  25  30FerryCPU-time (seconds)MacroSatPlan (optimal solutions)MacroSatPlan (with optimality proof)SatPlanFigure 2: CPU times of SatPlan andMacroSatPlan for find-ing an optimal solution and proving its optimality for do-main Ferry. On the x-axis, we have the problem namessimplified by numbers; on the y-axis, the CPU times (loga-rithmic scale).Preliminary Experimental resultsFigure 2 shows the CPU time required by MacroSatPlanto find the optimal solution and proving its optimality afterhaving found it with respect to the time required by SatPlan,using 30 instances of domain Ferry. Concerning find-ing and proving optimal solutions,MacroSatPlan is slightlyfaster than SatPlan. On the other hand, the CPU time usedto generate the optimal solution (without proving its opti-mality) is much better than the CPU time of SatPlan.Conclusions and Future WorkWe have briefly described MacroSatPlan, a planner basedon the propositional satisfiability approach using macro-actions and learned horizons. A preliminary experimen-tal study shows that the learned knowledge can be usefulfor generating optimal solutions much more quickly thanSatPlan. Future work includes additional experiments aboutthe relative impact of the macro-actions and the predictivemodel of the optimal plan length on the performance ofthe planning system, and the integration of Wizard (New-ton et al. 2007) as an alternative system for learning macro-actions.ReferencesA. Botea, M. Enzenberger, M. Müller and J. Schaeffer. 2005.Macro-FF: Improving AI Planning with Automatically LearnedMacro-Operators. JAIR, 24:581–621.Eèn, N.; Sörensson, N. 2003. An Extensible SAT-solver In Proc.of SAT-03.Hoffmann, J., and Nebel, B. 2001. The FF planning system: fastplan generation through heuristic search. JAIR, 14:253–302.Kautz, H.; Selman, B. 1992. Planning as satisfiability. In Proc.of ECAI-92.Kautz, H.; Selman, B.; and Hoffmann, J. 2006. Satplan: Planningas satisfiability. In Abstracts of the 5th Int. Planning Competition.Newton, M. A. H.; Levine, J.; Fox, M.; and Long, D. 2007.Learning macro-actions for arbitrary planners and domains. InProc. of ICAPS-07.Witten, I. H., and Frank, E. 2005. Data Mining: Practical Ma-chine Learning Tools and Techniques (Second Edition).

An Extensible SAT-solver

Learning macro-actions for arbitrary planners and domains.

Optimal SAT-based Planning with Macro-actions and Learned Horizons Original Citation Gerevini, Alfonso, Saetti, Alessandro and Vallati,

Planning as satisfiability.

Satplan: Planning as satisfiability.

The FF planning system: fast plan generation through heuristic search.

Optimal SAT-based Planning with Macro-actions and Learned Horizons

Optimal SAT-based Planning with Macro-actions and Learned Horizons

Abstract

Similar works

Full text

Available Versions

University of Huddersfield Repository