The Complexity of Planning Problems With Simple Causal Graphs

We present three new complexity results for classes of planning problems with simple causal graphs. First, we describe a polynomial-time algorithm that uses macros to generate plans for the class 3S of planning problems with binary state variables and acyclic causal graphs. This implies that plan generation may be tractable even when a planning problem has an exponentially long minimal solution. We also prove that the problem of plan existence for planning problems with multi-valued variables and chain causal graphs is NP-hard. Finally, we show that plan existence for planning problems with binary state variables and polytree causal graphs is NP-complete

Hierarchical Linearly-Solvable Markov Decision Problems

We present a hierarchical reinforcement learning framework that formulates each task in the hierarchy as a special type of Markov decision process for which the Bellman equation is linear and has analytical solution. Problems of this type, called linearly-solvable MDPs (LMDPs) have interesting properties that can be exploited in a hierarchical setting, such as efficient learning of the optimal value function or task compositionality. The proposed hierarchical approach can also be seen as a novel alternative to solving LMDPs with large state spaces. We derive a hierarchical version of the so-called Z-learning algorithm that learns different tasks simultaneously and show empirically that it significantly outperforms the state-of-the-art learning methods in two classical hierarchical reinforcement learning domains: the taxi domain and an autonomous guided vehicle task.Comment: 11 pages, 6 figures, 26th International Conference on Automated Planning and Schedulin

Deep Policies for Width-Based Planning in Pixel Domains

Width-based planning has demonstrated great success in recent years due to its ability to scale independently of the size of the state space. For example, Bandres et al. (2018) introduced a rollout version of the Iterated Width algorithm whose performance compares well with humans and learning methods in the pixel setting of the Atari games suite. In this setting, planning is done on-line using the "screen" states and selecting actions by looking ahead into the future. However, this algorithm is purely exploratory and does not leverage past reward information. Furthermore, it requires the state to be factored into features that need to be pre-defined for the particular task, e.g., the B-PROST pixel features. In this work, we extend width-based planning by incorporating an explicit policy in the action selection mechanism. Our method, called $\pi$-IW, interleaves width-based planning and policy learning using the state-actions visited by the planner. The policy estimate takes the form of a neural network and is in turn used to guide the planning step, thus reinforcing promising paths. Surprisingly, we observe that the representation learned by the neural network can be used as a feature space for the width-based planner without degrading its performance, thus removing the requirement of pre-defined features for the planner. We compare $\pi$-IW with previous width-based methods and with AlphaZero, a method that also interleaves planning and learning, in simple environments, and show that $\pi$-IW has superior performance. We also show that $\pi$-IW algorithm outperforms previous width-based methods in the pixel setting of Atari games suite.Comment: In Proceedings of the 29th International Conference on Automated Planning and Scheduling (ICAPS 2019). arXiv admin note: text overlap with arXiv:1806.0589

The Influence of k-Dependence on the Complexity of Planning

A planning problem is k-dependent if each action has at most k pre-conditions on variables unaffected by the action. This concept is well-founded since k is a constant for all but a few of the standard planning domains, and is known to have implications for tractability. In this paper, we present several new complexity results for P(k), the class of k-dependent planning problems with binary variables and polytree causal graphs. The problem of plan generation for P(k) is equivalent to determining how many times each variable can change. Using this fact, we present a polytime plan generation algorithm for P(2) and P(3). For constant k> 3, we introduce and use the notion of a cover to find conditions under which plan generation for P(k) is polynomial

A seventh-Order Accurate and Stable Algorithm for the Computation of Stress Inside Cracked Rectangular Domains

A seventh-order accurate and extremely stable algorithm for the rapid computation of stress fields inside cracked rectangular domains is presented. The algorithm is seventh-order accurate since it incorporates basis functions, taking the asymptotic shape of the stress fields close to crack tips and corners into account at least up to order six. The algorithm is stable since it is based on a Fredholm integral equation of the second kind. The particular form of the integral equation represents the solution as the limit of a function which is analytic inside the domain. This allows for an efficient implementation. In an example, involving 112 discretization points on an elastic square with a center crack, values of normalized stress intensity factors and T-stress with a relative error of 10−6 are computed in seconds on a workstation. More points reduce the relative error down to 10−15, where it saturates in double precision arithmetic. A large-scale setup with up to 1024 cracks in an elastic square is also studied, using up to 740,000 discretization points. The algorithm is intended as a basic building block in general-purpose solvers for fracture mechanics. It can also be used as a substitute for benchmark tables

On the computation of stress fields on polygonal domains with V-notches

The interior stress problem is solved numerically for a single-edge notched specimen under uniaxial load. The algorithm is based on a modification of a Fredholm second-kind integral equation with compact operators due to Muskhelishvili. Several singular basis functions for each of the seven corners in the geometry enable high uniform resolution of the stress field with a modest number of discretization points. As a consequence, notch stress intensity factors can be computed directly from the solution. This is an improvement over other procedures where the stress field is not resolved in the corners and where notch stress intensity factors are computed in a roundabout way via a path-independent integral. Numerical examples illustrate the superior stability and economy of the new scheme

On the accuracy of benchmark tables and graphical results in the applied mechanics literature

Converged normalized stress intensity factors for a matrix crack interacting with an elastic cylinder are presented. The new results differ from previously published results in several examples. The need for better error analysis in computational fracture mechanics is emphasized