Learning the Past Tense of English Verbs: Connectionism Fights Back

The ability to learn the past tense of English verbs has become a benchmark test for cognitive modelling. In a recent paper, Ling (1994) presented a detailed head-to-head comparison of the generalization abilities of a particular Artificial Neural Network (ANN) model and a general purpose Symbolic Pattern Associator (SPA). The conclusion was that `the SPA generalizes the past tense of unseen verbs better than ANN models by a wide margin'. In this paper we show that this conclusion was based on comparisons with an uncharacteristically poorly performing ANN. A different ANN model is presented which not only out-performs the existing ANN models by a wide margin but also out-performs the SPA by a significant amount. We provide an explanation of how this happens and suggest several ways in which the model can be improved further

Noise Reduction by Multi-Target Learning

Abstract. We review the problems associated with neural networks learning from noisy and/or ambiguous training data and propose a simple procedure that appears to alleviate these problems. By making use of the readily available output error information, a network is able to choose the correct output targets from sets of possibilities and generate new targets if any of the correct ones appear to be missing from the given training data