Submitted to the University of London for the Degree of Doctor of Philosophy in Computer Scienc

Smith, Martin C.

English

[co-authored with Beal, D.F.]

[co-authored with Beal, D.F.] (1998a) Temporal Coherence and Prediction Decay in Temporal Difference Learning.

[co-authored with Beal, D.F.] (1998b) First Results from using Temporal Difference Learning in Shogi. In

[co-authored with Beal, D.F.] (1999a) Temporal Difference Learning Applied to Game Playing and the Results of Application to Shogi.

[co-authored with Beal, D.F.] (1999b) Temporal Coherence and Prediction Decay in TD Learning.

[co-authored with Beal, D.F.] (1999c) Temporal Difference Learning for Heuristic Search and Game Playing.

1001 Winning Chess Sacrifices and Combinations.

A Minimax Algorithm Better than Alpha-Beta?

A New Hashing Method with Application for Game Playing.

A Partial Analysis on Minimaxing Game Trees with Random Leaf Values.

A Reinforcement Learning Approach to JobShop Scheduling.

A Review of Game-Tree Pruning,

A Short History of Computer Chess.

A Taxonomy of Concepts for Evaluating Chess Strength.

A Test Suite for Chess Programs.

A Unified Theory of Heuristic Evaluation Functions and its Application to

Adapting bias by gradient descent: an incremental version of delta-bar-delta.

Adaptive Pattern Oriented Chess.

Adaptive Switching Circuits.

Advances in Man-Machine Play.

An analysis of alpha-beta pruning.

Analytical Mean Squared Error Curves for Temporal Difference Learning.

Appendix A A.1 Test Positions used in the Transposition Table Experiments Below are the positions used in the transposition table experiments of section 4.2. All positions occurred in games from the 10th VSB Tournament,

Brute Force in Chess and Science.

Chess 4.5 - The Northwestern University Chess Program.

Chess as the Drosophila of AI.

Chunking for Experience.

co-authored with Beal,

Computer Chess and Search.

Confidently Selecting a Search Heuristic.

Digital Computers Applied to Games.

Evaluation Tuning for Computer Chess: Linear Discriminant Methods.

Experiments with the Null Move.

Experiments with the Null-Move Heuristic.

Extension Heuristics.

First Results from using Temporal Difference Learning in Shogi. In

Games: The Next Challenge.

Gnu Shogi v1.2p03. Available from many sources, including ftp://ftp.uni.passau.de/pub/local/shogi Palay,

High-Performance Job-Shop Scheduling with a TimeDelay TD(l) Network.

How Computers Play Chess.

Hybrid Heuristic Search.

Increased Rates of Convergence Through Learning Rate Adaptation.

KnightCap: A chess program that learns by combining TD(l) with game-tree search.

Learning from Delayed Rewards.

Learning internal representation by error propagation.

Learning piece values using temporal differences.

Learning to Predict by the Methods of Temporal Differences.

Machine Learning in Computer Chess: The Next Generation.

Mating Sequences in the Quiescence Search.

Measuring the Performance Potential of Chess Programs.

METAGAME: A New Challenge for Games and Learning. In

Multiple Probes of Transposition Tables.

Natural Developments in Game Research: From Chess to Shogi to Go.

Neurogammon wins Computer Olympiad.

Null Move and Deep Search: Selective Search Heuristics for Obtuse Chess Programs.

On Step-Size and Bias in Temporal-Difference Learning.

On-Line Step Size Adaptation.

Online Local Gain Adaptation for Multi-layer Perceptrons.

Open Theoretical Questions in Reinforcement Learning.

Passages on the Life of a Philosopher.

Practical Chess Endings,

Practical Issues in Temporal Difference Learning.

Programming a Computer to Play Chess.

Quantification of Search Extension Benefits.

Random Evaluations in Chess.

Reinforcement Learning for Dynamic Channel Allocation in Cellular Telephone Systems.

Reinforcement learning: a survey.

Relative Efficiency of Alph-Beta Implementations.

Replacement Schemes and Two-Level Tables.

Replacement Schemes for Transposition Tables.

Retrograde Analysis of Certain Endgames.

Scheduling and Rescheduling with Iterative Repair.

Selective Search versus Brute Force.

Shogi for Beginners.

Shogi: Japan’s Game of Strategy. Charles E. Tuttle Company.

Singular Extensions: adding selectivity to brute force searching.

Some Innovations Introduced by Hitech.

Some studies in machine learning using the game of checkers.

TD-Gammon, a Self-Teaching Backgammon Program, achieves Master Level Play.

Temporal Coherence and Prediction Decay in TD Learning.

Temporal Coherence and Prediction Decay in Temporal Difference Learning.

Temporal Difference Learning and TD-Gammon.

Temporal Difference Learning Applied to Game Playing and the Results of Application to Shogi.

Temporal Difference Learning for Heuristic Search and Game Playing.

Temporal difference learning of position evaluators in the game of go.

The Art of Computer Programming.

The B* Tree Search Algorithm: A Best First Proof Procedure.

The Convergence of TD(l) for General l.

The Encyclopaedia of Chess Variants.

The History Heuristic.

The Human Side of Chess.

The Kasparov – Deep Blue Games.

The SEX Algorithm in Computer Chess.

Win at Chess.

YSS: About the Data Structures and the Algorithm. Published on the WWW at http://plaza15.mbn.or.jp/~yss

Temoral Difference Learning in Complex Domains