A neural blackboard architecture of sentence structure

We present a neural architecture for sentence representation. Sentences are represented in terms of word representations as constituents. A word representation consists of a neural assembly distributed over the brain. Sentence representation does not result from associations between neural word assemblies. Instead, word assemblies are embedded in a neural architecture, in which the structural (thematic) relations between words can be represented. Arbitrary thematic relations between arguments and verbs can be represented. Arguments can consist of nouns and phrases, as in sentences with relative clauses. A number of sentences can be stored simultaneously in this architecture. We simulate how probe questions about thematic relations can be answered. We discuss how differences in sentence complexity, such as the difference between subject-extracted versus object-extracted relative clauses and the difference between right-branching versus center-embedded structures, can be related to the underlying neural dynamics of the model. Finally, we illustrate how memory capacity for sentence representation can be related to the nature of reverberating neural activity, which is used to store information temporarily in this architecture

Mechanisms for the generation and regulation of sequential behaviour

A critical aspect of much human behaviour is the generation and regulation of sequential activities. Such behaviour is seen in both naturalistic settings such as routine action and language production and laboratory tasks such as serial recall and many reaction time experiments. There are a variety of computational mechanisms that may support the generation and regulation of sequential behaviours, ranging from those underlying Turing machines to those employed by recurrent connectionist networks. This paper surveys a range of such mechanisms, together with a range of empirical phenomena related to human sequential behaviour. It is argued that the empirical phenomena pose difficulties for most sequencing mechanisms, but that converging evidence from behavioural flexibility, error data arising from when the system is stressed or when it is damaged following brain injury, and between-trial effects in reaction time tasks, point to a hybrid symbolic activation-based mechanism for the generation and regulation of sequential behaviour. Some implications of this view for the nature of mental computation are highlighted

Time and information in perceptual adaptation to speech

Presubmission manuscript and supplementary files (stimuli, stimulus presentation code, data, data analysis code).Perceptual adaptation to a talker enables listeners to efficiently resolve the many-to-many mapping between variable speech acoustics and abstract linguistic representations. However, models of speech perception have not delved into the variety or the quantity of information necessary for successful adaptation, nor how adaptation unfolds over time. In three experiments using speeded classification of spoken words, we explored how the quantity (duration), quality (phonetic detail), and temporal continuity of talker-specific context contribute to facilitating perceptual adaptation to speech. In single- and mixed-talker conditions, listeners identified phonetically-confusable target words in isolation or preceded by carrier phrases of varying lengths and phonetic content, spoken by the same talker as the target word. Word identification was always slower in mixed-talker conditions than single-talker ones. However, interference from talker variability decreased as the duration of preceding speech increased but was not affected by the amount of preceding talker-specific phonetic information. Furthermore, efficiency gains from adaptation depended on temporal continuity between preceding speech and the target word. These results suggest that perceptual adaptation to speech may be understood via models of auditory streaming, where perceptual continuity of an auditory object (e.g., a talker) facilitates allocation of attentional resources, resulting in more efficient perceptual processing.NIH NIDCD (R03DC014045

The role of the left head of caudate in suppressing irrelevant words

Suppressing irrelevant words is essential to successful speech production and is expected to involve general control mechanisms that reduce interference from task-unrelated processing. To investigate the neural mechanisms that suppress visual word interference, we used fMRI and a Stroop task, using a block design with an event-related analysis. Participants indicated with a finger press whether a visual stimulus was colored pink or blue. The stimulus was either the written word "BLUE," the written word "PINK," or a string of four Xs, with word interference introduced when the meaning of the word and its color were "incongruent" (e.g., BLUE in pink hue) relative to congruent (e.g., BLUE in blue) or neutral (e.g., XXXX in pink). The participants also made color decisions in the presence of spatial interference rather than word interference (i.e., the Simon task). By blocking incongruent, congruent, and neutral trials, we identified activation related to the mechanisms that suppress interference as that which was greater at the end relative to the start of incongruency. This highlighted the role of the left head of caudate in the control of word interference but not spatial interference. The response in the left head of caudate contrasted to bilateral inferior frontal activation that was greater at the start than at the end of incongruency, and to the dorsal anterior cingulate gyrus which responded to a change in the motor response. Our study therefore provides novel insights into the role of the left head of caudate in the mechanisms that suppress word interference

Order and disorder in everyday action: the roles of contention scheduling and supervisory attention

This paper describes the contention scheduling/supervisory attentional system approach to action selection and uses this account to structure a survey of current theories of the control of action. The focus is on how such theories account for the types of error produced by some patients with frontal and/or left temporoparietal damage when attempting everyday tasks. Four issues, concerning both the theories and their accounts of everyday action breakdown, emerge: first, whether multiple control systems, each capable of controlling action in different situations, exist; second, whether different forms of damage at the neural level result in conceptually distinct disorders; third, whether semantic/conceptual knowledge of objects and actions can be dissociated from control mechanisms, and if so what computational principles govern sequential control; and fourth, whether disorders of everyday action should be attributed to a loss of semantic/conceptual knowledge, a malfunction of control, or some combination of the two

The time course of routine action

Previous studies of action selection in routinized tasks have used error rates as their sole dependent measure (e.g. Reason, 1979; Schwartz et al., 1998). Consequently, conclusions about the underlying mechanisms of correct behavior are necessarily indirect. The present experiment examines the performance of normal subjects in the prototypical coffee task (Botvinick & Plaut, 2004) when carried out in a virtual environment on screen. This has the advantage of (a) constraining the possible errors more tightly than a real world environment, and (b) giving access to latencies as an additional, finer grained measure of performance. We report error data and timing of action selection at the crucial branching points for the production of routinized task sequences both with and without a secondary task. Processing branching points leads to increased latencies. The presence of the secondary task has a greater effect on latencies at branching points than at equivalent non-branching points. Furthermore, error data and latencies dissociate, suggesting that the exact timing is a valid and valuable source of information when trying to understand the processes that govern routine tasks. The results of the experiment are discussed in relation to their implication for computational accounts of routine action selection

THE RELATIONSHIP BETWEEN SELECTIVE ATTENTION AND GLOBAL COHERENCE IN NARRATIVE DISCOURSE FOLLOWING RIGHT HEMISPHERE STROKE

Stroke is the leading cause of serious, long-term disability and nearly half of older stroke survivors experience moderate to severe disability. A common impairment following stroke is impaired discourse production. Functional outcome studies have proposed that the recovery of discourse abilities is critical to achieving a good quality of life. Communication impairments often persist into the chronic stages of recovery and can cause individuals to withdraw from social situations. Discourse production deficits may not be apparent in basic communication interactions, but become more obvious during complex conversations following non-aphasic brain injury (NABI) or right hemisphere stroke (RHD). The purpose of this three-part dissertation was to examine discourse production deficits following NABI and the current practices of speech-language pathologists in the assessment and treatment of discourse deficits. The first study examined the macrolinguistic processes of discourse, which included local coherence, global coherence and cohesion, following NABI. Ten individuals with NABI and 10 heathy controls, closely matched for age, gender, and education, provided a narrative recount of an event. Discourse samples were analyzed for local coherence, global coherence, and cohesion. Results indicated that individuals with NABI demonstrated impaired global coherence compared to healthy controls with relatively intact local coherence and cohesion. Although global coherence deficits were identified in the discourse of individuals with NABI, empirical evidence suggests that speech-language pathologists do not routinely assess or treat discourse production deficits. The second study explored the current practices of speech-language pathologists in the assessment and treatment of individuals with NABI with a specific focus on discourse production deficits using a phenomenological approach. Nine speech-language pathologists participated in semi-structured 1:1 interviews. Results indicated that speech-language pathologists do not routinely assess and treat discourse production deficits due to competing internal values and external or environmental demands. However, speech-language pathologists reported that discourse deficits are present following NABI and are characterized by impaired topic maintenance or global coherence. Participants reported that they were routinely assessing and treating the underlying cognitive process of attention in hopes that it would generalize to improved topic maintenance or global coherence. However, little is known about the relationship between selective attention and global coherence. The third study examined the maintenance of global coherence across discourse tasks and explored the relationship between selective attention and global coherence. Eleven participants with non-aphasic brain injury secondary to right hemisphere stroke participated in the study. Participants provided discourse samples in response to five discourse tasks in a single-task and dual-task condition. Additionally, participants completed the Stroop test to examine selective attention abilities. Results indicated that mean global coherence scores differed across discourse tasks, and that mean global coherence scores for stories was significantly higher than for simple recount and single picture description tasks. To examine the relationship between selective attention and global coherence, mean global coherence scores in the single-task and dual-task condition were compared. Mean global coherence scores were lower in the dual-task condition for all five discourse tasks. Mean global coherence scores for the complex recount task were significantly lower in the dual-task condition. This suggests that the maintenance of global coherence is influenced by attention abilities. There was no significant correlation between performance on the Stroop task and the maintenance of global coherence for any of the five discourse tasks

Learning and Production of Movement Sequences: Behavioral, Neurophysiological, and Modeling Perspectives

A growing wave of behavioral studies, using a wide variety of paradigms that were introduced or greatly refined in recent years, has generated a new wealth of parametric observations about serial order behavior. What was a mere trickle of neurophysiological studies has grown to a more steady stream of probes of neural sites and mechanisms underlying sequential behavior. Moreover, simulation models of serial behavior generation have begun to open a channel to link cellular dynamics with cognitive and behavioral dynamics. Here we summarize the major results from prominent sequence learning and performance tasks, namely immediate serial recall, typing, 2XN, discrete sequence production, and serial reaction time. These populate a continuum from higher to lower degrees of internal control of sequential organization. The main movement classes covered are speech and keypressing, both involving small amplitude movements that are very amenable to parametric study. A brief synopsis of classes of serial order models, vis-à-vis the detailing of major effects found in the behavioral data, leads to a focus on competitive queuing (CQ) models. Recently, the many behavioral predictive successes of CQ models have been joined by successful prediction of distinctively patterend electrophysiological recordings in prefrontal cortex, wherein parallel activation dynamics of multiple neural ensembles strikingly matches the parallel dynamics predicted by CQ theory. An extended CQ simulation model-the N-STREAMS neural network model-is then examined to highlight issues in ongoing attemptes to accomodate a broader range of behavioral and neurophysiological data within a CQ-consistent theory. Important contemporary issues such as the nature of working memory representations for sequential behavior, and the development and role of chunks in hierarchial control are prominent throughout.Defense Advanced Research Projects Agency/Office of Naval Research (N00014-95-1-0409); National Institute of Mental Health (R01 DC02852