A Neurocomputational Model of Grounded Language Comprehension and Production at the Sentence Level

While symbolic and statistical approaches to natural language processing have become undeniably impressive in recent years, such systems still display a tendency to make errors that are inscrutable to human onlookers. This disconnect with human processing may stem from the vast differences in the substrates that underly natural language processing in artificial systems versus biological systems. To create a more relatable system, this dissertation turns to the more biologically inspired substrate of neural networks, describing the design and implementation of a model that learns to comprehend and produce language at the sentence level. The model's task is to ground simulated speech streams, representing a simple subset of English, in terms of a virtual environment. The model learns to understand and answer full-sentence questions about the environment by mimicking the speech stream of another speaker, much as a human language learner would. It is the only known neural model to date that can learn to map natural language questions to full-sentence natural language answers, where both question and answer are represented sublexically as phoneme sequences. The model addresses important points for which most other models, neural and otherwise, fail to account. First, the model learns to ground its linguistic knowledge using human-like sensory representations, gaining language understanding at a deeper level than that of syntactic structure. Second, analysis provides evidence that the model learns combinatorial internal representations, thus gaining the compositionality of symbolic approaches to cognition, which is vital for computationally efficient encoding and decoding of meaning. The model does this while retaining the fully distributed representations characteristic of neural networks, providing the resistance to damage and graceful degradation that are generally lacking in symbolic and statistical approaches. Finally, the model learns via direct imitation of another speaker, allowing it to emulate human processing with greater fidelity, thus increasing the relatability of its behavior. Along the way, this dissertation develops a novel training algorithm that, for the first time, requires only local computations to train arbitrary second-order recurrent neural networks. This algorithm is evaluated on its overall efficacy, biological feasibility, and ability to reproduce peculiarities of human learning such as age-correlated effects in second language acquisition

Encoding Sequential Information in Semantic Space Models: Comparing Holographic Reduced Representation and Random Permutation

Circular convolution and random permutation have each been proposed as neurally plausible binding operators capable of encoding sequential information in semantic memory. We perform several controlled comparisons of circular convolution and random permutation as means of encoding paired associates as well as encoding sequential information. Random permutations outperformed convolution with respect to the number of paired associates that can be reliably stored in a single memory trace. Performance was equal on semantic tasks when using a small corpus, but random permutations were ultimately capable of achieving superior performance due to their higher scalability to large corpora. Finally, “noisy” permutations in which units are mapped to other units arbitrarily (no one-to-one mapping) perform nearly as well as true permutations. These findings increase the neurological plausibility of random permutations and highlight their utility in vector space models of semantics

An investigation of fast and slow mapping

Children learn words astonishingly skilfully. Even infants can reliably “fast map” novel category labels to their referents without feedback or supervision (Carey & Bartlett, 1978; Houston-Price, Plunkett, & Harris, 2005). Using both empirical and neural network modelling methods this thesis presents an examination of both the fast and slow mapping phases of children's early word learning in the context of object and action categorisation. A series of empirical experiments investigates the relationship between within-category perceptual variability on two-year-old children’s ability to learn labels for novel categories of objects and actions. Results demonstrate that variability profoundly affects both noun and verb learning. A review paper situates empirical word learning research in the context of recent advances in the application of computational models to developmental research. Data from the noun experiments are then simulated using a Dynamic Neural Field (DNF) model (see Spencer & Schöner, 2009), suggesting that children’s early object categories can emerge dynamically from simple label-referent associations strengthened over time. Novel predictions generated by the model are replicated empirically, providing proofof- concept for the use of DNF models in simulations of word learning, as well emphasising the strong featural basis of early categorisation. The noun data are further explored using a connectionist architecture (Morse, de Greef, Belpaeme & Cangelosi, 2010) in a robotic system, providing the groundwork for future research in cognitive robotics. The implications of these different approaches to cognitive modelling are discussed, situating the current work firmly in the dynamic systems tradition whilst emphasising the value of interdisciplinary research in motivating novel research paradigms

Representation Rectified

This abstract is a little involved and technical. Those looking for a more leisurely, thematic overview of the thesis can turn to the Introduction, which follows shortly. In the first chapter of the thesis, I examine Andy Clark’s argument for the extended mind thesis (EM henceforth). As Clark acknowledges, his argument for EM relies on a brand of functionalism developed by Frank Jackson and David Braddon-Mitchell. Mark Sprevak claims to have developed a reductio not only of Clark’s argument of EM, but of the functionalist position that Clark’s argument presupposes. I show that this reductio can be blocked, because it rests on an optional presupposition (a presupposition that is anyway implausible). But by rejecting this presupposition, we end up blocking Clark’s argument for EM as well as Sprevak’s reductio of that argument. This result is bad news for Clark, but rather better news for the functionalists on whose work Clark’s argument relies. In chapters 2 and 3, I develop and apply a model-based theory of mental representation. The basic idea is there is a type of mental representation (what I call ‘s-representation’) which is best understood by analogy with scientific models. In chapter 2, I develop a theory of content for s-representations, improving on the work of writers who have attempted to do so in the past. The theory of content I develop makes use of theoretical resources provided by those functionalist writers whose position I defended from Sprevak’s reductio in chapter 1. In chapter 3, I give reasons for thinking that s-representations are biologically ubiquitous and cognitively significant. I then show how my theory of s-representation differs from similar rival accounts in the literature. Finally, I argue that my account has the resources to deal with certain sceptical challenges raised by anti-representationalists (those sceptical of the claim that a certain class of mental capacities can be explained in representational terms). In Part II (chapters 4, 5 and 6), I apply the lessons learned from the first half of the thesis to develop some distinctive claims about the nature of visual perception. I do so by using Alva Noë’s theory of perception as a spring-board. I argue that many of Noë’s most notorious claims are false, but that there are still valuable resources to be gleaned from his theory. I then borrow and redeploy what is valuable in his theory (i.e. certain aspects of his ‘virtual content’ thesis and some of his claims about perspectival content). I do so, in part, by drawing on the s-representation story developed in Part I. I argue, in line with similar claims made by Rick Grush, that Noë’s notion of ‘sensorimotor knowledge’ can usefully be treated as a form of s-representation. With a fully reconstructed version of Noë’s theory in place, I show how it can make sense of some otherwise puzzling findings made by psychologists of perception