Phonological (un)certainty weights lexical activation

Spoken word recognition involves at least two basic computations. First is matching acoustic input to phonological categories (e.g. /b/, /p/, /d/). Second is activating words consistent with those phonological categories. Here we test the hypothesis that the listener's probability distribution over lexical items is weighted by the outcome of both computations: uncertainty about phonological discretisation and the frequency of the selected word(s). To test this, we record neural responses in auditory cortex using magnetoencephalography, and model this activity as a function of the size and relative activation of lexical candidates. Our findings indicate that towards the beginning of a word, the processing system indeed weights lexical candidates by both phonological certainty and lexical frequency; however, later into the word, activation is weighted by frequency alone.Comment: 6 pages, 4 figures, accepted at: Cognitive Modeling and Computational Linguistics (CMCL) 201

Indeterminacy in process type classification

Clausal analysis within Systemic Functional Linguistics (SFL) is generally based upon a classification of the clause into one of six process types. Although this allocation is often portrayed as clear-cut, in practice process distinction can be unclear, and a single verb may meet the coding criteria of a number of categories. The aim of this paper is to examine the nature of indeterminacy within a transitive SFL analysis, by surveying experienced SFL users for their classification of 20 clauses. Our main findings are threefold: 1) inconsistency of analysis was very prevalent - we find only one of the critical clauses to be unanimously categorised for process type; 2) the main area of disagreement between analysts was the selection of Material vs. Verbal processes; 3) clauses with low consistency ratings appeared to include performative main verbs. These findings are discussed in the light of the semantic properties of performativity, which may contribute to the difficulty in process type identification; further, possible alleviations to these issues are discussed in order to allow for a full consideration of both the syntactic and semantic realisation of the clause, in situations where these streams of information may diverge

Towards a Mechanistic Account of Speech Comprehension in the Human Brain

Humans understand speech with such speed and accuracy, it belies the complexity of transforming sound into meaning. The goal of my research is to develop a theoretically grounded, empirically tested and computationally explicit account of how the brain achieves this feat. In the work presented here, I overview a set of magneto-encephalography studies that describe (i) what linguistic representations the brain uses to bridge between sound and meaning; (ii) how those representations are combined to form hierarchical structures (e.g. phonemes into morphemes; morphemes into words); (iii) how information is exchanged across structures to guide comprehension from the bottom-up and top-down. The research also contributes to a broader analytical framework — informed by machine-learning and classic statistics — which allows neural signals to be decomposed into an interpretable sequence of operations. Overall, this dissertation showcases the utility of combining theoretical linguistics, machine-learning and cognitive neuroscience for developing empirically- and performance-optimised models of spoken language processing

How the brain composes morphemes into meaning

Morphemes (e.g. [tune], [-ful], [-ly]) are the basic blocks with which complex meaning is built. Here I explore the critical role that morpho-syntactic rules play in forming the meaning of morphologically complex words, from two primary standpoints: i) how semantically rich stem morphemes (e.g. explode, bake, post) combine with syntactic operators (e.g.-ion,-er,-age) to output a semantically predictable result; ii) how this process can be understood in terms of mathematical operations, easily allowing the brain to generate representations of novel morphemes and comprehend novel words. With these ideas in mind, I offer a model of morphological processing that incorporates semantic and morpho-syntactic operations in service to meaning composition, and discuss how such a model could be implemented in the human brain

Performance-optimized hierarchical models only partially predict neural responses during perceptual decision making

Models of perceptual decision making have historically been designed to maximally explain behaviour and brain activity independently of their ability to actually perform tasks. More recently, performance-optimized models have been shown to correlate with brain responses to images and thus present a complementary approach to understand perceptual processes. In the present study, we compare how these approaches comparatively account for the spatio-temporal organization of neural responses elicited by ambiguous visual stimuli. Forty-six healthy human subjects performed perceptual decisions on briefly flashed stimuli constructed from ambiguous characters. The stimuli were designed to have 7 orthogonal properties, ranging from low-sensory levels (e.g. spatial location of the stimulus) to conceptual (whether stimulus is a letter or a digit) and task levels (i.e. required hand movement). Magneto-encephalography source and decoding analyses revealed that these 7 levels of representations are sequentially encoded by the cortical hierarchy, and actively maintained until the subject responds. This hierarchy appeared poorly correlated to normative, drift-diffusion, and 5-layer convolutional neural networks (CNN) optimized to accurately categorize alpha-numeric characters, but partially matched the sequence of activations of 3/6 state-of-the-art CNNs trained for natural image labeling (VGG-16, VGG-19, MobileNet). Additionally, we identify several systematic discrepancies between these CNNs and brain activity, revealing the importance of single-trial learning and recurrent processing. Overall, our results strengthen the notion that performance-optimized algorithms can converge towards the computational solution implemented by the human visual system, and open possible avenues to improve artificial perceptual decision making

MASC-MEG

Magnetoencephalography recordings of healthy volunteers listening to short stories

Recurrent processes emulate a cascade of hierarchical decisions: evidence from spatio-temporal decoding of human brain activity

Mounting evidence suggests that perception depends on a largely-feedforward brain network. However, the discrepancy between (i) the latency of the corresponding feedforward responses (150-200 ms) and (ii) the time it takes human subjects to recognize brief images (often >500 ms) suggests that recurrent neuronal activity is critical to visual processing. Here, we use magneto-encephalography to localize, track and decode the feedforward and recurrent responses elicited by brief presentations of variably-ambiguous letters and digits. We first confirm that these stimuli trigger, within the first 200 ms, a feedforward response in the ventral and dorsal cortical pathways. The subsequent activity is distributed across temporal, parietal and prefrontal cortices and leads to a slow and incremental cascade of representations culminating in action-specific motor signals. We introduce an analytical framework to show that these brain responses are best accounted for by a hierarchy of recurrent neural assemblies. An accumulation of computational delays across specific processing stages explains subjects’ reaction times. Finally, the slow convergence of neural representations towards perceptual categories is quickly followed by all-or-none motor decision signals. Together, these results show how recurrent processes generate, over extended time periods, a cascade of hierarchical decisions that ultimately predicts subjects’ perceptual reports

Immediate ambiguity resolution in speech perception based on prior acoustic experience

Speech perception relies on the rapid resolution of uncertainty. Here we explore whether auditory experiences contribute to this process of ambiguity resolution. ~8000 participants were surveyed online for their (i) subjective percept of a speech stimulus with ambiguous formant allocation; (ii) demographic profile and auditory experiences. Both linguistic and non-linguistic auditory experiences significantly predict speech perception. Listeners were more likely to perceive the ambiguous stimulus in accordance with their own name, and were biased towards lower formant allocation as a function of being exposed to lower auditory frequencies in their environment. Overall, our results show that the subjective interpretation of an ambiguous stimulus in the auditory domain is determined by prior acoustic exposure, suggesting the operation of an exposure-dependent mechanism impacting sensitivity that resolves ambiguity in speech perception