Learning to Play Othello with N-Tuple Systems

This paper investigates the use of n-tuple systems as position value functions for the game of Othello. The architecture is described, and then evaluated for use with temporal difference learning. Performance is compared with previously de-veloped weighted piece counters and multi-layer perceptrons. The n-tuple system is able to defeat the best performing of these after just five hundred games of self-play learning. The conclusion is that n-tuple networks learn faster and better than the other more conventional approaches

Temporal difference learning with interpolated table value functions

This paper introduces a novel function approximation architecture especially well suited to temporal difference learning. The architecture is based on using sets of interpolated table look-up functions. These offer rapid and stable learning, and are efficient when the number of inputs is small. An empirical investigation is conducted to test their performance on a supervised learning task, and on themountain car problem, a standard reinforcement learning benchmark. In each case, the interpolated table functions offer competitive performance. ©2009 IEEE

Investigating learning rates for evolution and temporal difference learning

Evidently, any learning algorithm can only learn on the basis of the information given to it. This paper presents a first attempt to place an upper bound on the information rates attainable with standard co-evolution and with TDL. The upper bound for TDL is shown to be much higher than for coevolution. Under commonly used settings for learning to play Othello for example, TDL may have an upper bound that is hundreds or even thousands of times higher than that of coevolution. To test how well these bounds correlate with actual learning rates, a simple two-player game called Treasure Hunt. is developed. While the upper bounds cannot be used to predict the number of games required to learn the optimal policy, they do correctly predict the rank order of the number of games required by each algorithm. © 2008 IEEE

Minimizing the excitation of parasitic modes of vibration in slender power ultrasonic devices

The design of slender power ultrasonic devices can often be challenging due to the excitation of parasitic modes of vibration during operation. The excitation of these modes is known to manifest from behaviors such as modal coupling which if not controlled or designed out of the system can, under operational conditions, lead to poor device performance and device failure. However, a report published by the authors has indicted that the excitation of these modes of vibration could be minimized through device design, specifically careful location of the piezoceramic stack. This paper illustrates that it is possible, through piezoceramic stack position, to minimize modal coupling between a parasitic mode and the tuned longitudinal mode of vibration for slender ultrasonic devices

Forcing neurocontrollers to exploit sensory symmetry through hard-wired modularity in the game of Cellz

Several attempts have been made in the past to construct encoding schemes that allow modularity to emerge in evolving systems, but success is limited. We believe that in order to create successful and scalable encodings for emerging modularity, we first need to explore the benefits of different types of modularity by hard-wiring these into evolvable systems. In this paper we explore different ways of exploiting sensory symmetry inherent in the agent in the simple game Cellz by evolving symmetrically identical modules. It is concluded that significant increases in both speed of evolution and final fitness can be achieved relative to monolithic controllers. Furthermore, we show that a simple function approximation task that exhibits sensory symmetry can be used as a quick approximate measure of the utility of an encoding scheme for the more complex game-playing task

Ms Pac-Man versus Ghost Team CEC 2011 competition

Games provide an ideal test bed for computational intelligence and significant progress has been made in recent years, most notably in games such as Go, where the level of play is now competitive with expert human play on smaller boards. Recently, a significantly more complex class of games has received increasing attention: real-time video games. These games pose many new challenges, including strict time constraints, simultaneous moves and open-endedness. Unlike in traditional board games, computational play is generally unable to compete with human players. One driving force in improving the overall performance of artificial intelligence players are game competitions where practitioners may evaluate and compare their methods against those submitted by others and possibly human players as well. In this paper we introduce a new competition based on the popular arcade video game Ms Pac-Man: Ms Pac-Man versus Ghost Team. The competition, to be held at the Congress on Evolutionary Computation 2011 for the first time, allows participants to develop controllers for either the Ms Pac-Man agent or for the Ghost Team and unlike previous Ms Pac-Man competitions that relied on screen capture, the players now interface directly with the game engine. In this paper we introduce the competition, including a review of previous work as well as a discussion of several aspects regarding the setting up of the game competition itself. © 2011 IEEE

Evolving controllers for simulated car racing

This paper describes the evolution of controllers for racing a simulated radio-controlled car around a track, modelled on a real physical track. Five different controller architectures were compared, based on neural networks, force fields and action sequences. The controllers use either egocentric (first person), Newtonian (third person) or no information about the state of the car (open-loop controller). The only controller that is able to evolve good racing behaviour is based on a neural network acting on egocentric inputs

Decorated Feynman Categories

In [KW14], the new concept of Feynman categories was introduced to simplify the discussion of operad--like objects. In this present paper, we demonstrate the usefulness of this approach, by introducing the concept of decorated Feynman categories. The procedure takes a Feynman category $\mathfrak F$ and a functor $\mathcal O$ to a monoidal category to produce a new Feynman category ${\mathfrak F}_{dec {\mathcal O}}$. This in one swat explains the existence of non--sigma operads, non--sigma cyclic operads, and the non--sigma--modular operads of Markl as well as all the usual candidates simply from the category $\mathfrak G$, which is a full subcategory of the category of graphs of [BM08]. Moreover, we explain the appearance of terminal objects noted in [Mar15]. We can then easily extend this for instance to the dihedral case. Furthermore, we obtain graph complexes and all other known operadic type notions from decorating and restricting the basic Feynman category $\mathfrak G$ of aggregates of corollas. We additionally show that the construction is functorial. There are further geometric and number theoretic applications, which will follow in a separate preprint.Comment: Updated version from 6/8/16 to appear in J. of Noncommutative Geometr