Monte Carlo Tree Search with Heuristic Evaluations using Implicit Minimax Backups

Monte Carlo Tree Search (MCTS) has improved the performance of game engines in domains such as Go, Hex, and general game playing. MCTS has been shown to outperform classic alpha-beta search in games where good heuristic evaluations are difficult to obtain. In recent years, combining ideas from traditional minimax search in MCTS has been shown to be advantageous in some domains, such as Lines of Action, Amazons, and Breakthrough. In this paper, we propose a new way to use heuristic evaluations to guide the MCTS search by storing the two sources of information, estimated win rates and heuristic evaluations, separately. Rather than using the heuristic evaluations to replace the playouts, our technique backs them up implicitly during the MCTS simulations. These minimax values are then used to guide future simulations. We show that using implicit minimax backups leads to stronger play performance in Kalah, Breakthrough, and Lines of Action.Comment: 24 pages, 7 figures, 9 tables, expanded version of paper presented at IEEE Conference on Computational Intelligence and Games (CIG) 2014 conferenc

Queues and risk models with simultaneous arrivals

We focus on a particular connection between queueing and risk models in a multi-dimensional setting. We first consider the joint workload process in a queueing model with parallel queues and simultaneous arrivals at the queues. For the case that the service times are ordered (from largest in the first queue to smallest in the last queue) we obtain the Laplace-Stieltjes transform of the joint stationary workload distribution. Using a multivariate duality argument between queueing and risk models, this also gives the Laplace transform of the survival probability of all books in a multivariate risk model with simultaneous claim arrivals and the same ordering between claim sizes. Other features of the paper include a stochastic decomposition result for the workload vector, and an outline how the two-dimensional risk model with a general two-dimensional claim size distribution (hence without ordering of claim sizes) is related to a known Riemann boundary value problem

Positive and negative streamers in ambient air: measuring diameter, velocity and dissipated energy

Positive and negative streamers are studied in ambient air at 1 bar; they emerge from a needle electrode placed 40 mm above a planar electrode. The amplitudes of the applied voltage pulses range from 5 to 96 kV; most pulses have rise times of 30 ns or shorter. Diameters, velocities and energies of the streamers are measured. Two regimes are identified; a low voltage regime where only positive streamers appear and a high voltage regime where both positive and negative streamers exist. Below 5 kV, no streamers emerge. In the range from 5 to 40 kV, positive streamers form, while the negative discharges only form a glowing cloud at the electrode tip, but no streamers. For 5 to 20 kV, diameters and velocities of the positive streamers have the minimal values of d=0.2 mm and v \approx 10^5 m/s. For 20 to 40 kV, their diameters increase by a factor 6 while the voltage increases only by a factor 2. Above the transition value of 40 kV, streamers of both polarities form; they strongly resemble each other, though the positive ones propagate further; their diameters continue to increase with applied voltage. For 96 kV, positive streamers attain diameters of 3 mm and velocities of 4*10^6 m/s, negative streamers are about 20 % slower and thinner. An empirical fit formula for the relation between velocity v and diameter d is v=0.5 d^2/(mm ns) for both polarities. Streamers of both polarities dissipate energies of the order of several mJ per streamer while crossing the gap.Comment: 20 pages, 9 figures, accepted for J. Phys.

Adaptive General Search Framework for Games and Beyond

The research field of Artificial General Intelligence (AGI) is concerned with the creation of adaptive programs that can autonomously address tasks of a different nature. Search and planning have been identified as core capabilities of AGI, and have been successful in many scenarios that require sequential decision-making. However, many search algorithms are developed for specific problems and exploit domain-specific knowledge, which makes them not applicable to perform different tasks autonomously. Although some domain-independent search algorithms have been proposed, a programmer still has to make decisions on their design, setup and enhancements. Thus, the performance is limited by the programmer's decisions, which are usually biased. This paper proposes to develop a framework that, in line with the goals of AGI, autonomously addresses a wide variety of search tasks, adapting automatically to each new, unknown task. To achieve this, we propose to encode search algorithms in a formal language and combine algorithm portfolios with automatic algorithm generation. In addition, we see games as the ideal test bed for the framework, because they can model a wide variety of complex problems. Finally, we believe that this research will have an impact not only on the AGI research field, but also on the game industry and on real-world problems

Ludii -- The Ludemic General Game System

While current General Game Playing (GGP) systems facilitate useful research in Artificial Intelligence (AI) for game-playing, they are often somewhat specialised and computationally inefficient. In this paper, we describe the "ludemic" general game system Ludii, which has the potential to provide an efficient tool for AI researchers as well as game designers, historians, educators and practitioners in related fields. Ludii defines games as structures of ludemes -- high-level, easily understandable game concepts -- which allows for concise and human-understandable game descriptions. We formally describe Ludii and outline its main benefits: generality, extensibility, understandability and efficiency. Experimentally, Ludii outperforms one of the most efficient Game Description Language (GDL) reasoners, based on a propositional network, in all games available in the Tiltyard GGP repository. Moreover, Ludii is also competitive in terms of performance with the more recently proposed Regular Boardgames (RBG) system, and has various advantages in qualitative aspects such as generality.Comment: Accepted at ECAI 202

Heavy traffic analysis of roving server networks

This paper studies the heavy-traffic (HT) behaviour of queueing networks with a single roving server. External customers arrive at the queues according to independent renewal processes and after completing service, a customer either leaves the system or is routed to another queue. This type of customer routing in queueing networks arises very naturally in many application areas (in production systems, computer- and communication networks, maintenance, etc.). In these networks, the single most important characteristic of the system performance is oftentimes the path time, i.e. the total time spent in the system by an arbitrary customer traversing a specific path. The current paper presents the first HT asymptotic for the path-time distribution in queueing networks with a roving server under general renewal arrivals. In particular, we provide a strong conjecture for the system's behaviour under HT extending the conjecture of Coffman et al. [E.G. Coffman Jr., A.A. Puhalskii, M.I. Reiman 1995 and 1998] to the roving server setting of the current paper. By combining this result with novel light-traffic asymptotics we derive an approximation of the mean path-time for arbitrary values of the load and renewal arrivals. This approximation is not only highly accurate for a wide range of parameter settings, but is also exact in various limiting cases

A Real-Time Picking and Sorting System in E-Commerce Distribution Centers

Order fulfillment is the most expensive and critical operation for companies engaged in e-commerce. E-commerce distribution centers must rapidly organize the picking and sorting processes during and after the transaction has taken place, with the ongoing need to create greater responsiveness to customers. Sorting brings a relatively large setup time, which cannot be well admitted by existing polling models. We build a new stochastic polling model to describe and analyze such systems, and provide approximate explicit expressions for the complete distribution of order line waiting time for polling-based order picking systems and test their accuracy. These expressions lend themselves for operations and design operations, including deciding between “pick-and-sort” or “sort-while-pick” processes, and warehouse performance evaluation

Inception and propagation of positive streamers in high-purity nitrogen: effects of the voltage rise-rate

Controlling streamer morphology is important for numerous applications. Up to now, the effect of the voltage rise rate was only studied across a wide range. Here we show that even slight variations in the voltage rise can have significant effects. We have studied positive streamer discharges in a 16 cm point-plane gap in high-purity nitrogen 6.0, created by 25 kV pulses with a duration of 130 ns. The voltage rise varies by a rise rate from 1.9 kV/ns to 2.7 kV/ns and by the first peak voltage of 22 to 28 kV. A structural link is found between smaller discharges with a larger inception cloud caused by a faster rising voltage. This relation is explained by the greater stability of the inception cloud due to a faster voltage rise, causing a delay in the destabilisation. Time-resolved measurements show that the inception cloud propagates slower than an earlier destabilised, more filamentary discharge. This explains that the discharge with a faster rising voltage pulse ends up to be shorter. Furthermore, the effect of remaining background ionisation in a pulse sequence has been studied, showing that channel thickness and branching rate are locally affected, depending on the covered volume of the previous discharge.Comment: 16 pages, 9 figure