Policy-Space Search: Equivalences, Improvements, and Compression

Fully-observable non-deterministic (FOND) planning is at the core of artificial intelligence planning with uncertainty. It models uncertainty through actions with non-deterministic effects. A* with Non-Determinism (AND*) (Messa and Pereira, 2023) is a FOND planner that generalizes A* (Hart et al., 1968) for FOND planning. It searches for a solution policy by performing an explicit heuristic search on the policy space of the FOND task. In this paper, we study and improve the performance of the policy-space search performed by AND*. We present a polynomial-time procedure that constructs a solution policy given just the set of states that should be mapped. This procedure, together with a better understanding of the structure of FOND policies, allows us to present three concepts of equivalences between policies. We use policy equivalences to prune part of the policy search space, making AND* substantially more effective in solving FOND tasks. We also study the impact of taking into account structural state-space symmetries to strengthen the detection of equivalence policies and the impact of performing the search with satisficing techniques. We apply a recent technique from the group theory literature to better compute structural state-space symmetries. Finally, we present a solution compressor that, given a policy defined over complete states, finds a policy that unambiguously represents it using the minimum number of partial states. AND* with the introduced techniques generates, on average, two orders of magnitude fewer policies to solve FOND tasks. These techniques allow explicit policy-space search to be competitive in terms of both coverage and solution compactness with other state-of-the-art FOND planners

Iterative Depth-First Search for Fully Observable Non-Deterministic Planning

Fully Observable Non-Deterministic (FOND) planning models uncertainty through actions with non-deterministic effects. Existing FOND planning algorithms are effective and employ a wide range of techniques. However, most of the existing algorithms are not robust for dealing with both non-determinism and task size. In this paper, we develop a novel iterative depth-first search algorithm that solves FOND planning tasks and produces strong cyclic policies. Our algorithm is explicitly designed for FOND planning, addressing more directly the non-deterministic aspect of FOND planning, and it also exploits the benefits of heuristic functions to make the algorithm more effective during the iterative searching process. We compare our proposed algorithm to well-known FOND planners, and show that it has robust performance over several distinct types of FOND domains considering different metrics

PEA∗+IDA∗ : um algoritmo híbrido de memória limitada melhorado

It is well-known that the search algorithms A∗ and Iterative Deepening A∗ (IDA∗ ) can fail to solve state-space tasks optimally due to time and memory limits. The former typically fails in memory-restricted scenarios and the latter in time-restricted scenarios. Therefore, several algorithms were proposed to solve state-space tasks optimally using less memory than A∗ and less time than IDA∗ , such as A∗+IDA∗ , a hybrid memory-restricted algo rithm that combines A∗ and IDA∗ . In this work, we present a hybrid memory-restricted algorithm that combines Partial Expansion A∗ (PEA∗ ) and IDA∗ . This new algorithm has two phases, the same structure as the A∗+IDA∗ algorithm. The first phase of PEA∗+IDA∗ runs PEA∗ until it reaches a memory limit, and the second phase runs IDA∗ without du plicate detection on each node of the Open of PEA∗ . First, we present a model that shows how PEA∗+IDA∗ can perform better than A∗+IDA∗ although pure PEA∗ usually makes more expansions than pure A∗ . Later, we perform an experimental evaluation using three memory limits and show that compared to A∗+IDA∗ on classical planning domains, PEA∗+IDA∗ has higher coverage and expands fewer nodes. Finally, we experimentally analyze both algorithms and show that having higher F-limits and better priority-queue composition given by PEA∗ have a considerable impact on the performance of the algo rithms.É bem conhecido que os algoritmos de busca A∗ e Aprofundamento Iterativo A∗ (IDA∗ em inglês) podem falhar em resolver otimamente tarefas de busca em espaços de estado de vido a limites de tempo e memória. O primeiro tipicamente falha em cenários de memória limitada e o segundo em cenários de tempo limitado. Portanto, diversos algoritmos foram propostos para resolver otimamente tarefas de busca em espaços de estado usando menos memória que A∗ e menos tempo que IDA∗ , como por exemplo A∗+IDA∗ , um algoritmo híbrido de memória limitada que combina A∗ e IDA∗ . Nesse artigo, nós apresentamos um algoritmo híbrido de memória limita que combina o A∗ de Expansões Parciais (PEA∗ em inglês) com IDA∗ . Este novo algoritmo possui duas fases, mesma estrutura que o algo ritmo A∗+IDA∗ . A primeira fase do PEA∗+IDA∗ roda PEA∗ até o limite de memória ser alcançado, e a segunda fase roda IDA∗ , sem detecção de duplicatas, em cada nó da Open do PEA∗ . Primeiramente nós apresentamos um modelo que mostra como PEA∗+IDA∗ pode performar melhor que A∗+IDA∗ apesar do PEA∗ puro normalmente fazer mais ex pansões que o A∗ puro. Depois nós apresentamos uma avaliação experimental usando três limites de memória e mostramos que comparado ao A∗+IDA∗ , em domínios de planeja mento clássico, PEA∗+IDA∗ tem uma cobertura maior e expande menos nós. Por fim nós analisamos experimentalmente ambos algoritmos e mostramos que ter um F-limite maior e ter a fila de prioridades com melhor composição por conta do PEA∗ causa um impacto considerável na performance dos algoritmos

PEA*+IDA*: An Improved Hybrid Memory-Restricted Algorithm

It is well-known that the search algorithms A* and Iterative Deepening A* (IDA*) can fail to solve state-space tasks optimally due to time and memory limits. The former typically fails in memory-restricted scenarios and the latter in time-restricted scenarios. Therefore, several algorithms were proposed to solve state-space tasks optimally using less memory than A* and less time than IDA*, such as A*+IDA*, a hybrid memory-restricted algorithm that combines A* and IDA*. In this paper, we present a hybrid memory-restricted algorithm that combines Partial Expansion A* (PEA*) and IDA*. This new algorithm has two phases, the same structure as the A*+IDA* algorithm. The first phase of PEA*+IDA* runs PEA* until it reaches a memory limit, and the second phase runs IDA* without duplicate detection on each node of PEA*'s Open. First, we present a model that shows how PEA*+IDA* can perform better than A*+IDA* although pure PEA* usually makes more expansions than pure A*. Later, we perform an experimental evaluation using three memory limits and show that, compared to A*+IDA* on classical planning domains, PEA*+IDA* has higher coverage and expands fewer nodes. Finally, we experimentally analyze both algorithms and show that having higher F-limits and better priority-queue composition given by PEA* have a considerable impact on the performance of the algorithms

A Best-First Search Algorithm for FOND Planning and Heuristic Functions to Optimize Decompressed Solution Size

In this work, we study fully-observable non-deterministic (FOND) planning, which models uncertainty through actions with non-deterministic effects. We present a best-first heuristic search algorithm called AND* that searches the policy-space of the FOND task to find a solution policy. We generalize the concepts of optimality, admissibility, and goal-awareness for FOND. Using these new concepts, we formalize the concept of heuristic functions that can guide a policy-space search. We analyze different aspects of the general structure of FOND solutions to introduce and characterize a set of FOND heuristics that estimate how far a policy is from becoming a solution. One of these heuristics applies a novel insight. Guided by them AND* returns only solutions with the minimal possible number of mapped states. We systematically study these FOND heuristics theoretically and empirically. We observe that our best heuristic makes AND* much more effective than the straightforward heuristics. We believe that our work allows a better understanding of how to design algorithms and heuristics to solve FOND tasks

Iterative Depth-First Search for FOND Planning

Fully Observable Non-Deterministic (FOND) planning models uncertainty through actions with non-deterministic effects. Existing FOND planning algorithms are effective and employ a wide range of techniques. However, most of the existing algorithms are not robust for dealing with both non-determinism and task size. In this paper, we develop a novel iterative depth-first search algorithm that solves FOND planning tasks and produces strong cyclic policies. Our algorithm is explicitly designed for FOND planning, addressing more directly the non-deterministic aspect of FOND planning, and it also exploits the benefits of heuristic functions to make the algorithm more effective during the iterative searching process. We compare our proposed algorithm to well-known FOND planners, and show that it has robust performance over several distinct types of FOND domains considering different metrics