DSHOP: Distributed simple hierarchical ordered planner.

Planning has been an important subject in the area of Artificial Intelligence (AI) for over three decades. Planning is the problem of seeking a series of actions (that is, a plan) that will accomplish a desired goal. Most planning approaches rely on a single processor or a single-agent paradigm. Unfortunately, in a complex world, a single agent may not be sufficient to optimally solve the problem. Distributed Planning is a sub-field of Distributed AI that involves multi-agents working together to solve large planning problems. Distribution may speed up the traditional planning system through parallelism. Hierarchical Task Network (HTN) planning is an AI planning methodology that creates plans by task decomposition. SHOP (Simple Hierarchical Ordered Planner) is a domain-independent HTN planning system designed by Dana Nau et al. that plans for tasks in the same order that they will later be executed. This thesis aims at designing and implementing a distributed version of SHOP (that is, DSHOP) and running it on a high performance distributed system called SHARCNET. The implementation is based upon Message Passing Interface (MPI), that is, a library of functions used to achieve parallelism via message-passing. We investigate two approaches to share work between processors: state-copying and state-recomputation. We implemented a state-copying based DSHOP system (DSHOPC), and a state-recomputation based DSHOP system (DSHOPR). We compared these two implementations of DSHOP with the Java version of SHOP on a set of randomly generated artificial domains. A set of experimental results has been used to evaluate the performance of the DSHOP algorithm.Dept. of Computer Science. Paper copy at Leddy Library: Theses & Major Papers - Basement, West Bldg. / Call Number: Thesis2004 .L83. Source: Masters Abstracts International, Volume: 43-01, page: 0240. Advisers: Scott Goodwin; Froduald Kabanza. Thesis (M.Sc.)--University of Windsor (Canada), 2004

Agent-based transportation planning compared with scheduling heuristics

Here we consider the problem of dynamically assigning vehicles to transportation orders that have di¤erent time windows and should be handled in real time. We introduce a new agent-based system for the planning and scheduling of these transportation networks. Intelligent vehicle agents schedule their own routes. They interact with job agents, who strive for minimum transportation costs, using a Vickrey auction for each incoming order. We use simulation to compare the on-time delivery percentage and the vehicle utilization of an agent-based planning system to a traditional system based on OR heuristics (look-ahead rules, serial scheduling). Numerical experiments show that a properly designed multi-agent system may perform as good as or even better than traditional methods

Generating Long-term Trajectories Using Deep Hierarchical Networks

We study the problem of modeling spatiotemporal trajectories over long time horizons using expert demonstrations. For instance, in sports, agents often choose action sequences with long-term goals in mind, such as achieving a certain strategic position. Conventional policy learning approaches, such as those based on Markov decision processes, generally fail at learning cohesive long-term behavior in such high-dimensional state spaces, and are only effective when myopic modeling lead to the desired behavior. The key difficulty is that conventional approaches are "shallow" models that only learn a single state-action policy. We instead propose a hierarchical policy class that automatically reasons about both long-term and short-term goals, which we instantiate as a hierarchical neural network. We showcase our approach in a case study on learning to imitate demonstrated basketball trajectories, and show that it generates significantly more realistic trajectories compared to non-hierarchical baselines as judged by professional sports analysts.Comment: Published in NIPS 201

Comparison of agent-based scheduling to look-ahead heuristics for real-time transportation problems

We consider the real-time scheduling of full truckload transportation orders with time windows that arrive during schedule execution. Because a fast scheduling method is required, look-ahead heuristics are traditionally used to solve these kinds of problems. As an alternative, we introduce an agent-based approach where intelligent vehicle agents schedule their own routes. They interact with job agents, who strive for minimum transportation costs, using a Vickrey auction for each incoming order. This approach offers several advantages: it is fast, requires relatively little information and facilitates easy schedule adjustments in reaction to information updates. We compare the agent-based approach to more traditional hierarchical heuristics in an extensive simulation experiment. We find that a properly designed multiagent approach performs as good as or even better than traditional methods. Particularly, the multi-agent approach yields less empty miles and a more stable service level

Constructing Abstraction Hierarchies Using a Skill-Symbol Loop

We describe a framework for building abstraction hierarchies whereby an agent alternates skill- and representation-acquisition phases to construct a sequence of increasingly abstract Markov decision processes. Our formulation builds on recent results showing that the appropriate abstract representation of a problem is specified by the agent's skills. We describe how such a hierarchy can be used for fast planning, and illustrate the construction of an appropriate hierarchy for the Taxi domain

CoRide: Joint Order Dispatching and Fleet Management for Multi-Scale Ride-Hailing Platforms

How to optimally dispatch orders to vehicles and how to tradeoff between immediate and future returns are fundamental questions for a typical ride-hailing platform. We model ride-hailing as a large-scale parallel ranking problem and study the joint decision-making task of order dispatching and fleet management in online ride-hailing platforms. This task brings unique challenges in the following four aspects. First, to facilitate a huge number of vehicles to act and learn efficiently and robustly, we treat each region cell as an agent and build a multi-agent reinforcement learning framework. Second, to coordinate the agents from different regions to achieve long-term benefits, we leverage the geographical hierarchy of the region grids to perform hierarchical reinforcement learning. Third, to deal with the heterogeneous and variant action space for joint order dispatching and fleet management, we design the action as the ranking weight vector to rank and select the specific order or the fleet management destination in a unified formulation. Fourth, to achieve the multi-scale ride-hailing platform, we conduct the decision-making process in a hierarchical way where a multi-head attention mechanism is utilized to incorporate the impacts of neighbor agents and capture the key agent in each scale. The whole novel framework is named as CoRide. Extensive experiments based on multiple cities real-world data as well as analytic synthetic data demonstrate that CoRide provides superior performance in terms of platform revenue and user experience in the task of city-wide hybrid order dispatching and fleet management over strong baselines.Comment: CIKM 201