Quantity and Quality: Not a Zero-Sum Game

Quantification of existing theories is a great challenge but also a great chance for the study of language in the brain. While quantification is necessary for the development of precise theories, it demands new methods and new perspectives. In light of this, four complementary methods were introduced to provide a quantitative and computational account of the extended Argument Dependency Model from Bornkessel-Schlesewsky and Schlesewsky. First, a computational model of human language comprehension was introduced on the basis of dependency parsing. This model provided an initial comparison of two potential mechanisms for human language processing, the traditional "subject" strategy, based on grammatical relations, and the "actor" strategy based on prominence and adopted from the eADM. Initial results showed an advantage for the traditional subject" model in a restricted context; however, the "actor" model demonstrated behavior in a test run that was more similar to human behavior than that of the "subject" model. Next, a computational-quantitative implementation of the "actor" strategy as weighted feature comparison between memory units was used to compare it to other memory-based models from the literature on the basis of EEG data. The "actor" strategy clearly provided the best model, showing a better global fit as well as better match in all details. Building upon the success modeling EEG data, the feasibility of estimating free parameters from empirical data was demonstrated. Both the procedure for doing so and the necessary software were introduced and applied at the level of individual participants. Using empirically estimated parameters, the models from the previous EEG experiment were calculated again and yielded similar results, thus reinforcing the previous work. In a final experiment, the feasibility of analyzing EEG data from a naturalistic auditory stimulus was demonstrated, which conventional wisdom says is not possible. The analysis suggested a new perspective on the nature of event-related potentials (ERPs), which does not contradict existing theory yet nonetheless goes against previous intuition. Using this new perspective as a basis, a preliminary attempt at a parsimonious neurocomputational theory of cognitive ERP components was developed

Spring School on Language, Music, and Cognition: Organizing Events in Time

The interdisciplinary spring school “Language, music, and cognition: Organizing events in time” was held from February 26 to March 2, 2018 at the Institute of Musicology of the University of Cologne. Language, speech, and music as events in time were explored from different perspectives including evolutionary biology, social cognition, developmental psychology, cognitive neuroscience of speech, language, and communication, as well as computational and biological approaches to language and music. There were 10 lectures, 4 workshops, and 1 student poster session. Overall, the spring school investigated language and music as neurocognitive systems and focused on a mechanistic approach exploring the neural substrates underlying musical, linguistic, social, and emotional processes and behaviors. In particular, researchers approached questions concerning cognitive processes, computational procedures, and neural mechanisms underlying the temporal organization of language and music, mainly from two perspectives: one was concerned with syntax or structural representations of language and music as neurocognitive systems (i.e., an intrapersonal perspective), while the other emphasized social interaction and emotions in their communicative function (i.e., an interpersonal perspective). The spring school not only acted as a platform for knowledge transfer and exchange but also generated a number of important research questions as challenges for future investigations

Predicting speech from a cortical hierarchy of event-based timescales

How do predictions in the brain incorporate the temporal unfolding of context in our natural environment? We here provide evidence for a neural coding scheme that sparsely updates contextual representations at the boundary of events. This yields a hierarchical, multilayered organization of predictive language comprehension. Training artificial neural networks to predict the next word in a story at five stacked time scales and then using model-based functional magnetic resonance imaging, we observe an event-based “surprisal hierarchy” evolving along a temporoparietal pathway. Along this hierarchy, surprisal at any given time scale gated bottom-up and top-down connectivity to neighboring time scales. In contrast, surprisal derived from continuously updated context influenced temporoparietal activity only at short time scales. Representing context in the form of increasingly coarse events constitutes a network architecture for making predictions that is both computationally efficient and contextually diverse

Neural mechanisms for flexible behaviour in humans and artificial neural networks

The natural environment is non-stationary – whilst our daily life often follows a routine, circumstances can change, and no two experiences are the same. To thrive, intelligent agents must therefore be able to flexibly adjust the way they process information to support the pursuit of their goals. In this thesis, I explored different aspects of such flexible, intelligent behaviour. To this end, I used a combination of artificial neural network modelling and human behavioural methods, comparing network activity patterns and behaviour against their biological counterparts. In the first set of experiments, I examined how artificial agents manipulate information in their short-term memory to match changing task demands. I demonstrated that neural networks used two distinct representational formats to prevent cross-interference and support the generalisation of information across contexts. Crucially, these representations were a striking match to those previously observed in biological brains. I then built on this result by analysing aspects of the neural network model not readily accessible to biological researchers, revealing the mechanistic and normative reasons for the observed results. In the second experiment, I explored how information can be prioritised in short term memory in situations where it can prove to be irrelevant for guiding behaviour at a later stage. I showed that artificial agents achieved this goal by amplifying the proportion of memory resources dedicated to prioritised information maintenance, without transforming it into a format that could guide subsequent behaviour. In the third experiment, I examined how logical categories can be learnt from an algorithmic perspective. I showed that a behavioural marker previously thought to index rule-based processes involving symbolic representations in humans was equally predictive of the behaviour of artificial agents not equipped with such features. In the last experiment, I investigated how humans learn simple problems and showed that they might have an innate predisposition to search for abstract rules, allowing them to generalise their knowledge to new contexts. Taken together, it is hoped that the work presented in this thesis furthers our understanding of the principles behind flexible, intelligent behaviour and sheds new light on their mechanistic implementation in neural circuits

Neurobiology of incremental speech comprehension

Understanding spoken language requires the rapid transition from perceptual processing of the auditory input through a variety of cognitive processes involved in constructing the mental representation of the message that the speaker is intending to convey. Listeners carry out these complex processes very rapidly and accurately as they hear each word incrementally unfolding in a sentence. However, little is known about the specific spatiotemporal patterning of this wide range of incremental processing operations that underpin the dynamic transitions from the speech input to the development of a meaning interpretation of an utterance. This thesis aims to address this set of issues by investigating the spatiotemporal dynamics of brain activity as spoken sentences unfold over time in order to illuminate the neurocomputational properties of the human language processing system and determine how the representation of a spoken sentence develops incrementally as each upcoming word is heard. Using a novel application of multidimensional probabilistic modelling combined with models from computational linguistics, I developed models of a variety of computational processes associated with accessing and processing the syntactic and semantic properties of sentences and tested these models at various points as sentences unfolded over time. Since a wide range of incremental processes occur very rapidly during speech comprehension, it is crucial to keep track of the temporal dynamics of the neural computations involved. To do this, I used combined electroencephalography and magnetoencephalography (EMEG) to record neural activity with millisecond resolution and analyzed the recordings in source space using univariate and/or multivariate approaches. The results confirm the value of this combination of methods in examining the properties of incremental speech processing. My findings corroborate the predictive nature of human speech comprehension and demonstrate that the effects of early semantic constraint are not dependent on explicit syntactic knowledge

Similar neural pathways link psychological stress and brain-age in health and multiple sclerosis

Clinical and neuroscientific studies suggest a link between psychological stress and reduced brain health in health and neurological disease but it is unclear whether mediating pathways are similar. Consequently, we applied an arterial-spin-labeling MRI stress task in 42 healthy persons and 56 with multiple sclerosis, and investigated regional neural stress responses, associations between functional connectivity of stress-responsive regions and the brain-age prediction error, a highly sensitive machine learning brain health biomarker, and regional brain-age constituents in both groups. Stress responsivity did not differ between groups. Although elevated brain-age prediction errors indicated worse brain health in patients, anterior insula–occipital cortex (healthy persons: occipital pole; patients: fusiform gyrus) functional connectivity correlated with brain-age prediction errors in both groups. Finally, also gray matter contributed similarly to regional brain-age across groups. These findings might suggest a common stress–brain health pathway whose impact is amplified in multiple sclerosis by disease-specific vulnerability factors