Surrogate Search As a Way to Combat Harmful Effects of Ill-behaved Evaluation Functions

Recently, several researchers have found that cost-based satisficing search with A* often runs into problems. Although some "work arounds" have been proposed to ameliorate the problem, there has been little concerted effort to pinpoint its origin. In this paper, we argue that the origins of this problem can be traced back to the fact that most planners that try to optimize cost also use cost-based evaluation functions (i.e., f(n) is a cost estimate). We show that cost-based evaluation functions become ill-behaved whenever there is a wide variance in action costs; something that is all too common in planning domains. The general solution to this malady is what we call a surrogatesearch, where a surrogate evaluation function that doesn't directly track the cost objective, and is resistant to cost-variance, is used. We will discuss some compelling choices for surrogate evaluation functions that are based on size rather that cost. Of particular practical interest is a cost-sensitive version of size-based evaluation function -- where the heuristic estimates the size of cheap paths, as it provides attractive quality vs. speed tradeoffsComment: arXiv admin note: substantial text overlap with arXiv:1103.368

Adaptive search techniques in AI planning and heuristic search

State-space search is a common approach to solve problems appearing in artificial intelligence and other subfields of computer science. In such problems, an agent must find a sequence of actions leading from an initial state to a goal state. However, the state spaces of practical applications are often too large to explore exhaustively. Hence, heuristic functions that estimate the distance to a goal state (such as straight-line distance for navigation tasks) are used to guide the search more effectively. Heuristic search is typically viewed as a static process. The heuristic function is assumed to be unchanged throughout the search, and its resulting values are directly used for guidance without applying any further reasoning to them. Yet critical aspects of the task may only be discovered during the search, e.g., regions of the state space where the heuristic does not yield reliable values. Our work here aims to make this process more dynamic, allowing the search to adapt to such observations. One form of adaptation that we consider is online refinement of the heuristic function. We design search algorithms that detect weaknesses in the heuristic, and address them with targeted refinement operations. If the heuristic converges to perfect estimates, this results in a secondary method of progress, causing search algorithms that are otherwise incomplete to eventually find a solution. We also consider settings that inherently require adaptation: In online replanning, a plan that is being executed must be amended for changes in the environment. Similarly, in real-time search, an agent must act under strict time constraints with limited information. The search algorithms we introduce in this work share a common pattern of online adaptation, allowing them to effectively react to challenges encountered during the search. We evaluate our contributions on a wide range of standard benchmarks. Our results show that the flexibility of these algorithms makes them more robust than traditional approaches, and they often yield substantial improvements over current state-of-the-art planners.Die Zustandsraumsuche ist ein oft verwendeter Ansatz um verschiedene Probleme zu lösen, die in der Künstlichen Intelligenz und anderen Bereichen der Informatik auftreten. Dabei muss ein Akteur eine Folge von Aktionen finden, die einen Pfad von einem Startzustand zu einem Zielzustand bilden. Die Zustandsräume von praktischen Anwendungen sind häufig zu groß um sie vollständig zu durchsuchen. Aus diesem Grund leitet man die Suche mit Heuristiken, die die Distanz zu einem Zielzustand abschätzen; zum Beispiel lässt sich die Luftliniendistanz als Heuristik für Navigationsprobleme einsetzen. Heuristische Suche wird typischerweise als statischer Prozess angesehen. Man nimmt an, dass die Heuristik während der Suche eine unveränderte Funktion ist, und die resultierenden Werte werden direkt zur Leitung der Suche benutzt ohne weitere Logik darauf anzuwenden. Jedoch könnten kritische Aspekte des Problems erst im Laufe der Suche erkannt werden, wie zum Beispiel Bereiche des Zustandsraums in denen die Heuristik keine verlässlichen Abschätzungen liefert. In dieser Arbeit wird der Suchprozess dynamischer gestaltet und der Suche ermöglicht sich solchen Beobachtungen anzupassen. Eine Art dieser Anpassung ist die Onlineverbesserung der Heuristik. Es werden Suchalgorithmen entwickelt, die Schwächen in der Heuristik erkennen und mit gezielten Verbesserungsoperationen beheben. Wenn die Heuristik zu perfekten Werten konvergiert ergibt sich daraus eine zusätzliche Form von Fortschritt, wodurch auch Suchalgorithmen, die sonst unvollständig sind, garantiert irgendwann eine Lösung finden werden. Es werden auch Szenarien betrachtet, die schon von sich aus Anpassung erfordern: In der Onlineumplanung muss ein Plan, der gerade ausgeführt wird, auf Änderungen in der Umgebung angepasst werden. Ähnlich dazu muss sich ein Akteur in der Echtzeitsuche unter strengen Zeitauflagen und mit eingeschränkten Informationen bewegen. Die Suchalgorithmen, die in dieser Arbeit eingeführt werden, folgen einem gemeinsamen Muster von Onlineanpassung, was ihnen ermöglicht effektiv auf Herausforderungen zu reagieren die im Verlauf der Suche aufkommen. Diese Ansätze werden auf einer breiten Reihe von Benchmarks ausgewertet. Die Ergebnisse zeigen, dass die Flexibilität dieser Algorithmen zu erhöhter Zuverlässigkeit im Vergleich zu traditionellen Ansätzen führt, und es werden oft deutliche Verbesserungen gegenüber modernen Planungssystemen erzielt.DFG grant 389792660 as part of TRR 248 – CPEC (see https://perspicuous-computing.science), and DFG grant HO 2169/5-1, "Critically Constrained Planning via Partial Delete Relaxation

最良優先探索のための探索非局在化手法

学位の種別: 課程博士審査委員会委員 : （主査）東京大学教授 Fukunaga Alex, 東京大学教授 山口 和紀, 東京大学准教授 田中 哲朗, 東京大学准教授 金子 知適, 東京大学准教授 森畑 明昌University of Tokyo(東京大学

Accelerating Heuristic Search for AI Planning

AI Planning is an important research field. Heuristic search is the most commonly used method in solving planning problems. Despite recent advances in improving the quality of heuristics and devising better search strategies, the high computational cost of heuristic search remains a barrier that severely limits its application to real world problems. In this dissertation, we propose theories, algorithms and systems to accelerate heuristic search for AI planning. We make four major contributions in this dissertation. First, we propose a state-space reduction method called Stratified Planning to accelerate heuristic search. Stratified Planning can be combined with any heuristic search to prune redundant paths in state space, without sacrificing the optimality and completeness of search algorithms. Second, we propose a general theory for partial order reduction in planning. The proposed theory unifies previous reduction algorithms for planning, and ushers in new partial order reduction algorithms that can further accelerate heuristic search by pruning more nodes in state space than previously proposed algorithms. Third, we study the local structure of state space and propose using random walks to accelerate plateau exploration for heuristic search. We also implement two state-of-the-art planners that perform competitively in the Seventh International Planning Competition. Last, we utilize cloud computing to further accelerate search for planning. We propose a portfolio stochastic search algorithm that takes advantage of the cloud. We also implement a cloud-based planning system to which users can submit planning tasks and make full use of the computational resources provided by the cloud. We push the state of the art in AI planning by developing theories and algorithms that can accelerate heuristic search for planning. We implement state-of-the-art planning systems that have strong speed and quality performance

Search Progress and Potentially Expanded States in Greedy Best-First Search

A classical result in optimal search shows that A* with an admissible and consistent heuristic expands every state whose f-value is below the optimal solution cost and no state whose f-value is above the optimal solution cost. For satisficing search algorithms, a similarly clear understanding is currently lacking. We examine the search behavior of greedy best-first search (GBFS) in order to make progress towards such an understanding. We introduce the concept of high-water mark benches, which separate the search space into areas that are searched by a GBFS algorithm in sequence. High-water mark benches allow us to exactly determine the set of states that are expanded by at least one GBFS tie-breaking strategy and give us a clearer understanding of search progress

Fast Scheduling of Robot Teams Performing Tasks With Temporospatial Constraints

The application of robotics to traditionally manual manufacturing processes requires careful coordination between human and robotic agents in order to support safe and efficient coordinated work. Tasks must be allocated to agents and sequenced according to temporal and spatial constraints. Also, systems must be capable of responding on-the-fly to disturbances and people working in close physical proximity to robots. In this paper, we present a centralized algorithm, named 'Tercio,' that handles tightly intercoupled temporal and spatial constraints. Our key innovation is a fast, satisficing multi-agent task sequencer inspired by real-time processor scheduling techniques and adapted to leverage a hierarchical problem structure. We use this sequencer in conjunction with a mixed-integer linear program solver and empirically demonstrate the ability to generate near-optimal schedules for real-world problems an order of magnitude larger than those reported in prior art. Finally, we demonstrate the use of our algorithm in a multirobot hardware testbed

Automated Hierarchical, Forward-Chaining Temporal Planner for Planetary Robots Exploring Unknown Environments

The transition of mobile robots from a controlled environment towards the real-world represents a major leap in terms of complexity coming primarily from three different factors: partial observability, nondeterminism and dynamic events. To cope with them, robots must achieve some intelligence behaviours to be cost and operationally effective. Two particularly interesting examples of highly complex robotic scenarios are Mars rover missions and the Darpa Robotic Challenge (DRC). In spite of the important differences they present in terms of constraints and requirements, they both have adopted certain level of autonomy to overcome some specific problems. For instance, Mars rovers have been endowed with multiple systems to enable autonomous payload operations and consequently increase science return. In the case of DRC, most teams have autonomous footstep planning or arm trajectory calculation. Even though some specific problems can be addressed with dedicated tools, the general problem remains unsolved: to deploy on-board a reliable reasoning system able to operate robots without human intervention even in complex environments. This is precisely the goal of an automated mission planner. The scientific community has provided plenty of planners able to provide very fast solutions for classical problems, typically characterized by the lack of time and resources representation. Moreover, there are also a handful of applied planners with higher levels of expressiveness at the price of lowest performance. However, a fast, expressive and robust planner has never been used in complex robotic missions. These three properties represent the main drivers for the outcomes of the thesis. To bridge the gap between classical and applied planning, a novel formalism named Hierarchical TimeLine Networks (HTLN) combining Timeline and HTN planning has been proposed. HTLN has been implemented on a mission planner named QuijoteExpress, the first forward-chaining timeline planner to the best of our knowledge. The main idea is to benefit from the great performance of forward-chaining search to resolve temporal problems on the state-space. In addition, QuijoteExpress includes search enhancements such as parallel planning by division of the problem in sub-problems or advanced heuristics management. Regarding expressiveness, the planner incorporates HTN techniques that allow to define hierarchical models and solutions. Finally, plan robustness in uncertain scenarios has been addressed by means of sufficient plans that allow to leave parts of valid plans undefined. To test the planner, a novel lightweight, timeline and ROS-based executive named SanchoExpress has been designed to translate the plans into actions understandable by the different robot subsystems. The entire approach has been tested in two realistic and complementary domains. A cooperative multirover Mars mission and an urban search and rescue mission. The results were extremely positive and opens new promising ways in the field of automated planning applied to robotics

Land Use Change and Economic Opportunity in Amazonia: An Agent-based Model

Economic changes such as rising açaí prices and the availability of off-farm employment are transforming the landscape of the Amazonian várzea, subject to decision-making at the farming household level. Land use change results from complex human-environment interactions which can be addressed by an agent-based model. An agent-based model is a simulation model composed of autonomous interacting entities known as agents, built from the bottom-up. Coupled with cellular automata, which forms the agents’ environment, agent-based models are becoming an important tool of land use science, complementing traditional methods of induction and deduction. The decision-making methods employed by agent-based models in recent years have included optimization, imitation, heuristics, classifier systems and genetic algorithms, among others, but multiple methods have rarely been comparatively analyzed. A modular agent-based model is designed to allow the researcher to substitute alternative decision-making methods. For a smallholder farming community in Marajó Island near Ponta de Pedras, Pará, Brazil, 21 households are simulated over a 40-year period. In three major scenarios of increasing complexity, these households first face an environment where goods sell at a constant price throughout the simulated period and there are no outside employment opportunities. This is followed by a scenario of variable prices based on empirical data. The third scenario combines variable prices with limited employment opportunities, creating multi-sited households as members emigrate. In each scenario, populations of optimizing agents and heuristic agents are analyzed in parallel. While optimizing agents allocate land cells to maximize revenue using linear programming, fast and frugal heuristic agents use decision trees to quickly pare down feasible solutions and probabilistically select between alternatives weighted by expected revenue. Using distributed computing, the model is run through several parameter sweeps and results are recorded to a cenral database. Land use trajectories and sensitivity analyses highlight the relative biases of each decision-making method and illustrate cases where alternative methods lead to significantly divergent outcomes. A hybrid approach is recommended, employing alternative decision-making methods in parallel to illustrate inefficiencies exogenous and endogenous to the decision-maker, or allowing agents to select among multiple methods to mitigate bias and best represent their real-world analogues

Best-subset Selection for Complex Systems using Agent-based Simulation

It is difficult to analyze and determine strategies to control complex systems due to their inherent complexity. The complex interactions among elements make it difficult to develop and test decision makers' intuition of how the system will behave under different policies. Computer models are often used to simulate the system and to observe both direct and indirect effects of alternative interventions. However, many decision makers are unwilling to concede complete control to a computer model because of the abstractions in the model, and the other factors that cannot be modeled, such as physical, human, social and organizational relationship constraints. This dissertation develops an agent-based simulation (ABS) model to analyze a complex system and its policy alternatives, and contributes a best-subset selection (BSS) procedure that provides a group of good performing alternatives to which decision makers can then apply their subject and context knowledge in making a final decision for implementation. As a specific example of a complex system, a mass casualty incident (MCI) response system was simulated using an ABS model consisting of three interrelated sub-systems. The model was then validated by a series of sensitivity analysis experiments. The model provides a good test bed to evaluate various evacuation policies. In order to find the best policy that minimizes the overall mortality, two ranking-and-selection (R&S) procedures from the literature (Rinott (1978) and Kim and Nelson (2001)) were implemented and compared. Then a new best-subset selection (BSS) procedure was developed to efficiently select a statistically guaranteed best-subset containing all alternatives that are close enough to the best one for a pre-specified probability. Extensive numerical experiments were organized to prove the effectiveness and demonstrate the performance of the BSS procedure. The BSS procedure was then implemented in conjunction with the MCI ABS model to select the best evacuation policies. The experimental results demonstrate the feasibility and effectiveness of our agent-based optimization methodology for complex system policy evaluation and selection