A distributional model of semantic context effects in lexical processinga

One of the most robust findings of experimental psycholinguistics is that the context in which a word is presented influences the effort involved in processing that word. We present a novel model of contextual facilitation based on word co-occurrence prob ability distributions, and empirically validate the model through simulation of three representative types of context manipulation: single word priming, multiple-priming and contextual constraint. In our simulations the effects of semantic context are mod eled using general-purpose techniques and representations from multivariate statistics, augmented with simple assumptions reflecting the inherently incremental nature of speech understanding. The contribution of our study is to show that special-purpose m echanisms are not necessary in order to capture the general pattern of the experimental results, and that a range of semantic context effects can be subsumed under the same principled account.›

Composition in distributional models of semantics

Distributional models of semantics have proven themselves invaluable both in cognitive modelling of semantic phenomena and also in practical applications. For example, they have been used to model judgments of semantic similarity (McDonald, 2000) and association (Denhire and Lemaire, 2004; Griffiths et al., 2007) and have been shown to achieve human level performance on synonymy tests (Landuaer and Dumais, 1997; Griffiths et al., 2007) such as those included in the Test of English as Foreign Language (TOEFL). This ability has been put to practical use in automatic thesaurus extraction (Grefenstette, 1994). However, while there has been a considerable amount of research directed at the most effective ways of constructing representations for individual words, the representation of larger constructions, e.g., phrases and sentences, has received relatively little attention. In this thesis we examine this issue of how to compose meanings within distributional models of semantics to form representations of multi-word structures. Natural language data typically consists of such complex structures, rather than just individual isolated words. Thus, a model of composition, in which individual word meanings are combined into phrases and phrases combine to form sentences, is of central importance in modelling this data. Commonly, however, distributional representations are combined in terms of addition (Landuaer and Dumais, 1997; Foltz et al., 1998), without any empirical evaluation of alternative choices. Constructing effective distributional representations of phrases and sentences requires that we have both a theoretical foundation to direct the development of models of composition and also a means of empirically evaluating those models. The approach we take is to first consider the general properties of semantic composition and from that basis define a comprehensive framework in which to consider the composition of distributional representations. The framework subsumes existing proposals, such as addition and tensor products, but also allows us to define novel composition functions. We then show that the effectiveness of these models can be evaluated on three empirical tasks. The first of these tasks involves modelling similarity judgements for short phrases gathered in human experiments. Distributional representations of individual words are commonly evaluated on tasks based on their ability to model semantic similarity relations, e.g., synonymy or priming. Thus, it seems appropriate to evaluate phrase representations in a similar manner. We then apply compositional models to language modelling, demonstrating that the issue of composition has practical consequences, and also providing an evaluation based on large amounts of natural data. In our third task, we use these language models in an analysis of reading times from an eye-movement study. This allows us to investigate the relationship between the composition of distributional representations and the processes involved in comprehending phrases and sentences. We find that these tasks do indeed allow us to evaluate and differentiate the proposed composition functions and that the results show a reasonable consistency across tasks. In particular, a simple multiplicative model is best for a semantic space based on word co-occurrence, whereas an additive model is better for the topic based model we consider. More generally, employing compositional models to construct representations of multi-word structures typically yields improvements in performance over non-compositonal models, which only represent individual words

Robo-CAMAL : anchoring in a cognitive robot

The CAMAL architecture (Computational Architectures for Motivation,Affect and Learning) provides an excellent framework within which to explore and investigate issues relevant to cognitive science and artificial intelligence. This thesis describes a small sub element of the CAMAL architecture that has been implemented on a mobile robot. The first area of investigation within this research relates to the anchoring problem. Can the robotic agent generate symbols based on responses within its perceptual systems and can it reason about its environment based on those symbols? Given that the agent can identify changes within its environment, can it then adapt its behaviour and alter its goals to mirror the change in its environment? The second area of interest involves agent learning. The agent has a domain model that details its goals, the actions it can perform and some of the possible environmental states it may encounter. The agent is not provided with the belief-goal-action combinations in order to achieve its goals. The agent is also unaware of the effect its actions have upon its environment. Can the agent experiment with its behaviour to generate its own belief-goal-action combinations that allow it to achieve its goals? A second related problem involves the case where the belief-goal-action combination is pre-programmed. This is when the agent is provided with several different methods with which to achieve a specific goal. Can the agent learn which combination is the best? This thesis will describe the sub-element of the CAMAL architecture that was developed for a robot (robo-CAMAL). It will also demonstrate how robo-CAMAL solves the anchoring problem, and learns how to act and adapt in its environment

Age of acquistion and phonology in lexical processing

The work presented in the current thesis was aimed at identifying the exact locus of the age of acquisition (AoA) effect within the systems responsible for word and picture processing. Chapter One reviews some of the current influential models of word and picture production and discusses the effects that AoA (and frequency) have upon these processes. Current theories of AoA are also discussed. Chapters Two and Three assess the locus of the AoA effect in the word naming task. The results of these experiments lead to the conclusion that AoA (and frequency) exert their effects in the connections between orthography and phonology in single word naming. Chapter Four then tested the alternative claim that AoA affects the level of phonological output processing by investigating the AoA effect in a phonological segmentation task and by relating the size of the AoA effect in this task and in a word naming task to individual differences in phonological skill. The results of this comparison demonstrate that AoA is unrelated to explicit phonological processing. Chapter Six then investigated the effect of AoA (and other variables) in the picture naming task by relating aphasic patient's level of impairment to the variables that affect their picture naming performance. The results of this study suggest that AoA influences the strength of the connections between semantics and phonology in picture naming. The present thesis concludes that AoA influences the strength of the connections between input (orthography and semantics) and phonological output. The final Chapter discusses the implications of the present results for current theories of AoA and for models of word and picture production

Insights into language processing in aphasia from semantic priming and semantic judgement tasks

The nature of semantic impairment in people with aphasia (PWA) provides the background to the current study, which examines whether different methods of semantic assessment can account for such deficits. Cognitive ability, which has previously been linked to language ability in PWA, may impact on test performance and was therefore also examined. The aims of the current study were to compare performance of control participants and PWA on implicit and explicit assessment of semantics, and to relate it to performance on tests of cognition. The impact of semantically similar versus associative relationship types between test stimuli was also considered. Three experimental semantic tasks were developed, including one implicit measure of semantic processing (Semantic Priming) and two explicit measures (Word to Picture Verification and Word to Picture Matching). Test stimuli were matched in terms of key psycholinguistic variables of frequency, imageability and length, and other factors including visual similarity, semantic similarity, and association. Performance of 40 control participants and 20 PWA was investigated within and between participant groups. The relationship between semantic task performance and existing semantic and cognitive assessments was also explored in PWA. An important finding related to a subgroup of PWA who were impaired on the explicit experimental semantic tasks but demonstrated intact semantic processing via the implicit method. Within tasks some differences were found in the effects of semantically related or associated stimuli. No relationships were found between experimental semantic task performance and cognitive task accuracy. The research offers insights into the role of implicit language testing, the impact of stimuli relationship type, and the complex relationship between semantic processing and cognition. The findings underline the need for valid and accurate measures of semantic processing to be in place to enable accurate diagnosis for PWA, in order to direct appropriate intervention choice and facilitate successful rehabilitation

Quantifying the psychological properties of words

This thesis explores the psychological properties of words – the idea that words carry links to additional information beyond their dictionary meaning. It does so by presenting three distinct publications and an applied project, the Macroscope. The published research respectively covers: the modelling of language networks to explain lexical growth; the use of high dimensional vector representations of words to discuss language learning; and the collection of a normative dataset of single word humour ratings. The first publication outlines the use of network science in psycholinguistics. The methodology is discussed, providing clear guidelines on the application of networks when answering psychologically motivated questions. A selection of psychological studies is presented as a demonstration of use cases for networks in cognitive psychology. The second publication uses referent feature norms to represent words in a high dimensional vector space. A correlative link between referent distinctiveness and age of acquisition is proposed. The shape bias literature (the idea that children only pay attention to the shape of objects early on) is evaluated in relation to the findings. The third publication collects and shares a normative dataset of single word humour ratings. Descriptive properties of the dataset are outlined and the potential future use in the field of humour is discussed. Finally, the thesis presents the Macroscope, a collaborative project put together with Li Ying. The Macroscope is an online platform, allowing for easy analysis of the psychological properties of target words. The platform is showcased, and its full functionality is presented, including visualisation examples. Overall, the thesis aims to give researchers all that’s necessary to start working with psychological properties of words – the understanding of network science in psycholinguistics, high dimensional vector spaces, normative datasets and the applied use of all the above through the Macroscope

Disordered speech in dementia

What is the effect on language of the progressive degenerative disorder, Alzheimer's disease (AD)? What are the functional consequences of this illness, particularly for speech? The majority of accounts interpret speech disorder in AD as reflecting underlying semantic disruption. In contrast I apply current theories of lexicalization in speech production to the speech disorder. Four competing hypotheses are derived from a two-stage model of lexicalization in speech production. This model contains separate semantic, lexical and phonological representations. Data are collected from patients with probable AD and age-matched controls using standard psycholinguistic techniques. The data support an explanation of progressively impaired higher level cognitive processing which interacts with impaired semantic to lexical processing in speech production

Artificial ontogenesis: a connectionist model of development

This thesis suggests that ontogenetic adaptive processes are important for generating intelligent beha- viour. It is thus proposed that such processes, as they occur in nature, need to be modelled and that such a model could be used for generating artificial intelligence, and specifically robotic intelligence. Hence, this thesis focuses on how mechanisms of intelligence are specified.A major problem in robotics is the need to predefine the behaviour to be followed by the robot. This makes design intractable for all but the simplest tasks and results in controllers that are specific to that particular task and are brittle when faced with unforeseen circumstances. These problems can be resolved by providing the robot with the ability to adapt the rules it follows and to autonomously create new rules for controlling behaviour. This solution thus depends on the predefinition of how rules to control behaviour are to be learnt rather than the predefinition of rules for behaviour themselves.Learning new rules for behaviour occurs during the developmental process in biology. Changes in the structure of the cerebral 'cortex underly behavioural and cognitive development throughout infancy and beyond. The uniformity of the neocortex suggests that there is significant computational uniformity across the cortex resulting from uniform mechanisms of development, and holds out the possibility of a general model of development. Development is an interactive process between genetic predefinition and environmental influences. This interactive process is constructive: qualitatively new behaviours are learnt by using simple abilities as a basis for learning more complex ones. The progressive increase in competence, provided by development, may be essential to make tractable the process of acquiring higher -level abilities.While simple behaviours can be triggered by direct sensory cues, more complex behaviours require the use of more abstract representations. There is thus a need to find representations at the correct level of abstraction appropriate to controlling each ability. In addition, finding the correct level of abstrac- tion makes tractable the task of associating sensory representations with motor actions. Hence, finding appropriate representations is important both for learning behaviours and for controlling behaviours. Representations can be found by recording regularities in the world or by discovering re- occurring pat- terns through repeated sensory -motor interactions. By recording regularities within the representations thus formed, more abstract representations can be found. Simple, non -abstract, representations thus provide the basis for learning more complex, abstract, representations.A modular neural network architecture is presented as a basis for a model of development. The pat- tern of activity of the neurons in an individual network constitutes a representation of the input to that network. This representation is formed through a novel, unsupervised, learning algorithm which adjusts the synaptic weights to improve the representation of the input data. Representations are formed by neurons learning to respond to correlated sets of inputs. Neurons thus became feature detectors or pat- tern recognisers. Because the nodes respond to patterns of inputs they encode more abstract features of the input than are explicitly encoded in the input data itself. In this way simple representations provide the basis for learning more complex representations. The algorithm allows both more abstract represent- ations to be formed by associating correlated, coincident, features together, and invariant representations to be formed by associating correlated, sequential, features together.The algorithm robustly learns accurate and stable representations, in a format most appropriate to the structure of the input data received: it can represent both single and multiple input features in both the discrete and continuous domains, using either topologically or non -topologically organised nodes. The output of one neural network is used to provide inputs for other networks. The robustness of the algorithm enables each neural network to be implemented using an identical algorithm. This allows a modular `assembly' of neural networks to be used for learning more complex abilities: the output activations of a network can be used as the input to other networks which can then find representations of more abstract information within the same input data; and, by defining the output activations of neurons in certain networks to have behavioural consequences it is possible to learn sensory -motor associations, to enable sensory representations to be used to control behaviour