Analysis of Generalized Artificial Intelligence Potential through Reinforcement and Deep Reinforcement Learning Approaches

Artificial Intelligence is the next competitive domain; the first nation to develop human level artificial intelligence will have an impact similar to the development of the atomic bomb. To maintain the security of the United States and her people, the Department of Defense has funded research into the development of artificial intelligence and its applications. This research uses reinforcement learning and deep reinforcement learning methods as proxies for current and future artificial intelligence agents and to assess potential issues in development. Agent performance were compared across two games and one excursion: Cargo Loading, Tower of Hanoi, and Knapsack Problem, respectively. Deep reinforcement learning agents were observed to handle a wider range of problems, but behave inferior to specialized reinforcement learning algorithms

On Rearrangement of Items Stored in Stacks

There are $n \ge 2$ stacks, each filled with $d$ items, and one empty stack. Every stack has capacity $d > 0$. A robot arm, in one stack operation (step), may pop one item from the top of a non-empty stack and subsequently push it onto a stack not at capacity. In a {\em labeled} problem, all $nd$ items are distinguishable and are initially randomly scattered in the $n$ stacks. The items must be rearranged using pop-and-pushs so that in the end, the $k^{\rm th}$ stack holds items $(k-1)d +1, \ldots, kd$, in that order, from the top to the bottom for all $1 \le k \le n$. In an {\em unlabeled} problem, the $nd$ items are of $n$ types of $d$ each. The goal is to rearrange items so that items of type $k$ are located in the $k^{\rm th}$ stack for all $1 \le k \le n$. In carrying out the rearrangement, a natural question is to find the least number of required pop-and-pushes. Our main contributions are: (1) an algorithm for restoring the order of $n^2$ items stored in an $n \times n$ table using only $2n$ column and row permutations, and its generalization, and (2) an algorithm with a guaranteed upper bound of $O(nd)$ steps for solving both versions of the stack rearrangement problem when $d \le \lceil cn \rceil$ for arbitrary fixed positive number $c$. In terms of the required number of steps, the labeled and unlabeled version have lower bounds $\Omega(nd + nd{\frac{\log d}{\log n}})$ and $\Omega(nd)$, respectively

Loopless Gray Code Enumeration and the Tower of Bucharest

We give new algorithms for generating all n-tuples over an alphabet of m letters, changing only one letter at a time (Gray codes). These algorithms are based on the connection with variations of the Towers of Hanoi game. Our algorithms are loopless, in the sense that the next change can be determined in a constant number of steps, and they can be implemented in hardware. We also give another family of loopless algorithms that is based on the idea of working ahead and saving the work in a buffer

On the restricted Hanoi Graphs

Consider the restricted Hanoi graphs which correspond to the variants of the famous Tower of Hanoi problem with multiple pegs where moves of the discs are restricted throughout the arcs of a movement digraph whose vertices represent the pegs of the puzzle and an arc from vertex $p$ to vertex $q$ exists if and only if moves from peg $p$ to peg $q$ are allowed. In this paper, we gave some notes on how to construct the restricted Hanoi graphs as well as some combinatorial results on the number of arcs in these graphs.Comment: 8 pages, 2 figure

Boosting Multi-Core Reachability Performance with Shared Hash Tables

This paper focuses on data structures for multi-core reachability, which is a key component in model checking algorithms and other verification methods. A cornerstone of an efficient solution is the storage of visited states. In related work, static partitioning of the state space was combined with thread-local storage and resulted in reasonable speedups, but left open whether improvements are possible. In this paper, we present a scaling solution for shared state storage which is based on a lockless hash table implementation. The solution is specifically designed for the cache architecture of modern CPUs. Because model checking algorithms impose loose requirements on the hash table operations, their design can be streamlined substantially compared to related work on lockless hash tables. Still, an implementation of the hash table presented here has dozens of sensitive performance parameters (bucket size, cache line size, data layout, probing sequence, etc.). We analyzed their impact and compared the resulting speedups with related tools. Our implementation outperforms two state-of-the-art multi-core model checkers (SPIN and DiVinE) by a substantial margin, while placing fewer constraints on the load balancing and search algorithms.Comment: preliminary repor

Front-to-End Bidirectional Heuristic Search with Near-Optimal Node Expansions

It is well-known that any admissible unidirectional heuristic search algorithm must expand all states whose $f$-value is smaller than the optimal solution cost when using a consistent heuristic. Such states are called "surely expanded" (s.e.). A recent study characterized s.e. pairs of states for bidirectional search with consistent heuristics: if a pair of states is s.e. then at least one of the two states must be expanded. This paper derives a lower bound, VC, on the minimum number of expansions required to cover all s.e. pairs, and present a new admissible front-to-end bidirectional heuristic search algorithm, Near-Optimal Bidirectional Search (NBS), that is guaranteed to do no more than 2VC expansions. We further prove that no admissible front-to-end algorithm has a worst case better than 2VC. Experimental results show that NBS competes with or outperforms existing bidirectional search algorithms, and often outperforms A* as well.Comment: Accepted to IJCAI 2017. Camera ready version with new timing result