Semantics and the stratification of explanation in cognitive science

This work is concerned with a pervasive problem in Cognitive Science which I have called the "stratificational" approach. I argue that the division into "levels of explanation" that runs as a constant theme through much work in Cognitive Science and in particular natural language semantics, is in direct conflict with neuroscientific evidence. I claim it is also in conflict with a right understanding of the philosophical notion of "evidence". The neuroscientific work is linked with the philosophical problem to provide a critique of concrete cases of research within the natural language semantics community. More recent neuroscientifically aware research is examined and it is demonstrated that it suffers similar problems due to the same deep running assumptions as those which effect traditional formalist theory. The contribution of this thesis is thought to be that of a demonstration of the essential nature and indeed the ubiquity of the basic assumptions in the field. Also, a new link is forged between the concerns of the formalists and certain seemingly more abstract philosophical work. This link enables us to see how much philosophical problems infect research into cognition and language. It is argued that practical research in Cognitive Science simply cannot be seen to be independent of the philosophical basis of the entire subject. The resulting picture of Cognitive Science and its place is outlined and explored with special emphasis on what I have called the "Principle of Semantic Indistinguishabliity" which says that the contribution of what can be broadly termed "environment" is epitemologically opaque to our cognition. The importance of this principle is discussed.The purpose of this work is to draw out a fundamental thread of reasoning and methodology that underlies most traditional work, and some not so traditional work, in Cognitive Science. It will be argued that this line of reasoning is at odds with the implications of modern neuroscience and cannot base a reasonable claim to "explain" human cognition. The picture I shall identify is that which I shall call "stratified". This, in general, is an attempt at explanation that divides into "levels of explanation", each with its own concepts that are said to be essential to the explanation of a phenomenon. There are specific and pragmatic manifestations of this, I discuss these in Chapter 3 and 7 in particular. There are also more abstract expressions of the same tendency which I examine mainly in Chapter 6. One of the principle tasks is to demonstrate the links between the assumptions of the more abstract formulations of this approach and th eir pragmatic instantiations in work in Cognitive Science. This allows it to be made clear that certain methodological problems are ubiquitous within the field and are not simply a result of the particular pragmatics of a particular research area.In Cognitive Science as a whole, it is generally appreciated today that there are problems to do with integration of traditional formal systems and the evolutionary and biological aspects of human cognition. One aim of this work is exactly to give an argument, supported from work in the brain sciences, that a certain methodology - particularly that enshrined within formal systems in language semantics - is strongly denied its evidential basis as a result of certain empirical considerations. It is also denied much of its basis as a result of the incongruity between the original motivations of logical formalism and the use to which this formalism is put today. The conclusion of this is that Cognitive Science's role in certain areas is severely limited and it crucially relies on an amount of empirical brain research in places thought usually to be completely separate from the "low-level" evidence from neuroscience. Part of my thesis is that stratified systems and particularly systems of formal logic within linguistics and semantics, cannot possibly be independent in the way imagined. There is also exploration of a general point regarding the character of the relation between strata in a stratified theory. There is, I shall argue, an irresolvable tension between the desire to have separate strata which are both independent but related. We shall see this both in concrete terms in the discussion of Fodor and in the abstract in the discussion of McDowell.George Lakoffhas expressed agreement with this particular premise: " ... linguistic results ... indicate that human reason uses some of the same mechanisms involved in perception and ... human reason can be seen as growing out of perceptual and motor mechanisms."1If this is correct, then I think that there are enormous implications for Cognitive Science in its practise of semantics since the mechanisms of motor and perceptual systems impose radical constraints when applied in the area of semantics.Given this, my aim is to demonstrate that certain seemingly theoryindependent areas of research in Cognitive Science such as linguistics and natural language semantics are actually infected with damaging assumptions from certain misguided philosophical positions. The idea that we can simply model things in Cognitive Science and wait for someone else to sort out the theoretical structure into which all of the models will fit is not tenable. I shall demonstrate this in several concrete cases and couple this with a critique from neuroscience which is crucially related to a more philosophical critique of fundamental assumptions. The structure of the work is as follows. Firstly, I give an overview of foundational issues in Cognitive Science by discussing central works. Then, I introduce the main problems in concrete form by way of an examination of certain approaches to inference in formal semantics. Chapter 4 expands on this in an analysis of the notion of "compositionality" with reference to the "stratificational" approach I find apparent in traditional work in Cognitive Science and the assumptions it disguises. Chapter 5 introduces the themes from neuroscience and the relations they have to the philosophical critique in Chapter 6. In Chapter 7, I demonstrate that the assumptions I have identified are present even in work motivated by a desire to leave behind the formalist program. I explain why this is the case and the implications this has for a correct view of "evidence" in Cognitive Science. At this point, I deal with pertinent objections to my view stemming from the parts of the discipline I have mentioned. Chapter 8 condenses the problem and shows the fundamentals of the whole problem in relief, suggesting what all of the preceding means for Cognitive Science

State-of-the-art generalisation research in NLP: a taxonomy and review

The ability to generalise well is one of the primary desiderata of natural language processing (NLP). Yet, what `good generalisation' entails and how it should be evaluated is not well understood, nor are there any common standards to evaluate it. In this paper, we aim to lay the ground-work to improve both of these issues. We present a taxonomy for characterising and understanding generalisation research in NLP, we use that taxonomy to present a comprehensive map of published generalisation studies, and we make recommendations for which areas might deserve attention in the future. Our taxonomy is based on an extensive literature review of generalisation research, and contains five axes along which studies can differ: their main motivation, the type of generalisation they aim to solve, the type of data shift they consider, the source by which this data shift is obtained, and the locus of the shift within the modelling pipeline. We use our taxonomy to classify over 400 previous papers that test generalisation, for a total of more than 600 individual experiments. Considering the results of this review, we present an in-depth analysis of the current state of generalisation research in NLP, and make recommendations for the future. Along with this paper, we release a webpage where the results of our review can be dynamically explored, and which we intend to up-date as new NLP generalisation studies are published. With this work, we aim to make steps towards making state-of-the-art generalisation testing the new status quo in NLP.Comment: 35 pages of content + 53 pages of reference

Representation Of Lexical Stylistic Features In Language Models' Embedding Space

The representation space built by pretrained Language Models (LMs) encodes rich information about words and their relationships (e.g., similarity, hypernymy/hyponymy, polysemy) as well as abstract semantic notions (e.g., intensity). In this paper, we demonstrate that lexical stylistic notions such as complexity, formality, and figurativeness, can also be identified in this space. We show that it is possible to derive a vector representation for each of these stylistic notions, from only a small number of seed text pairs. Using these vectors, we can characterize new texts in terms of these dimensions using simple calculations in the corresponding embedding space. We perform experiments on five datasets and find that static embeddings encode these features more accurately at the level of words and phrases, whereas contextualized LMs perform better on longer texts. The lower performance of contextualized representations at the word level is partially attributable to the anisotropy of their vector space, which can be corrected through techniques like standardization to further improve performance.Comment: Accepted at *SEM 202