Structure-(in)dependent Interpretation of Phrases in Humans and LSTMs

In this study, we compared the performance of a long short-term memory (LSTM) neural network to the behavior of human participants on a language task that requires hierarchically structured knowledge. We show that humans interpret ambiguous noun phrases, such as second blue ball, in line with their hierarchical constituent structure. LSTMs, instead, only do so after unambiguous training, and they do not systematically generalize to novel items. Overall, the results of our simulations indicate that a model can behave hierarchically without relying on hierarchical constituent structure

Modelling metric violations in (geometric) conceptual spaces

Understanding how people represent similarity relations between concepts is one of the most fundamental problems in cognitive science, with implications for many theories of learning and reasoning. Human judgments of similarity violate basic metric assumptions, leading to effects such as judgment asymmetry and the triangle inequality. These effects have been difficult to capture with modern geometric representations of conceptual structure such as vector embeddings. Here we introduce a similarity function related to a feature-based view of concepts. We show how this function can be applied to geometric representations and that the resulting algorithm can account for classic judgment effects. Using representations extracted from a Large Language Model, we computed the predictions of this approach to similarity relations among a set of everyday concepts (world countries), and evaluated these predictions against human judgments of similarity in a behavioral experiment. The model's predictions correlate with human judgments. These results offer insight into human judgments of similarity relations and the design of algorithms that align with human reasoning

A mathematical neural process model of language comprehension, from syllable to sentence

Human beings effortlessly perceive structural meaning from a biophysical signal such as speech or sign. An explanation of the processes underlying this phenomenon needs to account for both the properties of the signal, as well as those of the neural architecture deploying linguistic knowledge. This article approaches the question from a mathematical perspective, providing a neurophysiologically grounded explanation of the process underlying linguistic structure building in the brain. For defining the properties of the signal, we rely on the mathematical linguistics of DisCoCat (Coecke et al. 2010), a syntactically sensitive formalism of distributional meaning that makes no claims about the neural processing underlying sentence comprehension. The neuroscientific architecture derives from a cue-integration model of language comprehension (Martin 2020). In it, the brain infers the latent structure of a cue or signal based on knowledge of the language, through a process of neural coordinate transform. In this work, we demonstrate how the DisCoCat formalism can interface with the neurophysiological process model and describe how the resulting incremental process model can return a formal description at each timestep. Second, we present an extension to show how structure building from phonological segments to syllables can be modelled within a categorial grammar setup, integrating it with our process model. Third, we introduce a temporal metric interpretation on the transformations occuring within the extended DisCoCat formalism for each level of representation - also known as categorical enrichment. As a result of this specification, we obtain a mechanistic account of neural oscillatory readouts during language comprehension

Inferring the nature of linguistic computations in the brain

Sentences contain structure that determines their meaning beyond that of individual words. An influential study by Ding and colleagues (2016) used frequency tagging of phrases and sentences to show that the human brain is sensitive to structure by finding peaks of neural power at the rate at which structures were presented. Since then, there has been a rich debate on how to best explain this pattern of results with profound impact on the language sciences. Models that use hierarchical structure building, as well as models based on associative sequence processing, can predict the neural response, creating an inferential impasse as to which class of models explains the nature of the linguistic computations reflected in the neural readout. In the current manuscript, we discuss pitfalls and common fallacies seen in the conclusions drawn in the literature illustrated by various simulations. We conclude that inferring the neural operations of sentence processing based on these neural data, and any like it, alone, is insufficient. We discuss how to best evaluate models and how to approach the modeling of neural readouts to sentence processing in a manner that remains faithful to cognitive, neural, and linguistic principles

Hierarchical structure in language and action: A formal comparison

Since the cognitive revolution, language and action have been compared as cognitive systems, with cross-domain convergent views recently gaining renewed interest in biology, neuroscience, and cognitive science. Language and action are both combinatorial systems whose mode of combination has been argued to be hierarchical, combining elements into constituents of increasingly larger size. This structural similarity has led to the suggestion that they rely on shared cognitive and neural resources. In this paper, we compare the conceptual and formal properties of hierarchy in language and action using tools from category theory. We show that the strong compositionality of language requires a formalism that describes the mapping between sentences and their syntactic structures as an order-embedded Galois connection, while the weak compositionality of actions only requires a monotonic mapping between action sequences and their goals, which we model as a monotone Galois connection. We aim to capture the different system properties of language and action in terms of the distinction between hierarchical sets and hierarchical sequences, and discuss the implications for the way both systems are represented in the brain