A Methodology for the Development of Recurrent Networks for Sequence Processing Tasks

Artificial neural networks are increasingly being used for dealing with real world applications. Many of these (e.g. speech recognition) are based on an ability to perform sequence processing. A class of artificial neural networks, known as recurrent networks, have architectures which incorporate feedback connections. This in turn allows the development of a memory mechanism to allow sequence processing to occur. A large number of recurrent network models have been developed, together with modifications of existing architectures and learning rules. However there has been comparatively little effort made to compare the performance of these models relative to each other. Such comparative studies would show differences in performance between networks and allow an examination of what features of a network give rise to desirable behaviours such as faster learning and superior generalisation ability. This thesis describes the results of a number of existing comparative studies and the results of new research. Three different recurrent networks, both in their original form and with modifications, are tested with four different sequence processing tasks. The results of this research clearly show that recurrent networks vary widely in terms of their performance and lead to a methodology based on the following conclusions: </br

Learning to play 3x3 games: neural networks as bounded-rational players

"We present a neural network methodology for learning game-playing rules in general. Existing research suggests learning to find a Nash equilibrium in a new game is too difficult a task for a neural network, but says little about what it will do instead. We observe that a neural network trained to find Nash equilibria in a known subset of games will use self-taught rules developed endogenously when facing new games. These rules are close to payoff dominance and its best response. Our findings are consistent with existing experimental results, both in terms of subject's methodology and success rates." [author's abstract