Calibrating Generative Models: The Probabilistic Chomsky-Schützenberger Hierarchy

A probabilistic Chomsky–Schützenberger hierarchy of grammars is introduced and studied, with the aim of understanding the expressive power of generative models. We offer characterizations of the distributions definable at each level of the hierarchy, including probabilistic regular, context-free, (linear) indexed, context-sensitive, and unrestricted grammars, each corresponding to familiar probabilistic machine classes. Special attention is given to distributions on (unary notations for) positive integers. Unlike in the classical case where the "semi-linear" languages all collapse into the regular languages, using analytic tools adapted from the classical setting we show there is no collapse in the probabilistic hierarchy: more distributions become definable at each level. We also address related issues such as closure under probabilistic conditioning

Generative grammar

Generative Grammar is the label of the most influential research program in linguistics and related fields in the second half of the 20. century. Initiated by a short book, Noam Chomsky's Syntactic Structures (1957), it became one of the driving forces among the disciplines jointly called the cognitive sciences. The term generative grammar refers to an explicit, formal characterization of the (largely implicit) knowledge determining the formal aspect of all kinds of language behavior. The program had a strong mentalist orientation right from the beginning, documented e.g. in a fundamental critique of Skinner's Verbal behavior (1957) by Chomsky (1959), arguing that behaviorist stimulus-response-theories could in no way account for the complexities of ordinary language use. The "Generative Enterprise", as the program was called in 1982, went through a number of stages, each of which was accompanied by discussions of specific problems and consequences within the narrower domain of linguistics as well as the wider range of related fields, such as ontogenetic development, psychology of language use, or biological evolution. Four stages of the Generative Enterprise can be marked off for expository purposes

Degraded acceptability and markedness in syntax, and the stochastic interpretation of optimality theory

The argument that I tried to elaborate on in this paper is that the conceptual problem behind the traditional competence/performance distinction does not go away, even if we abandon its original Chomskyan formulation. It returns as the question about the relation between the model of the grammar and the results of empirical investigations – the question of empirical verification The theoretical concept of markedness is argued to be an ideal correlate of gradience. Optimality Theory, being based on markedness, is a promising framework for the task of bridging the gap between model and empirical world. However, this task not only requires a model of grammar, but also a theory of the methods that are chosen in empirical investigations and how their results are interpreted, and a theory of how to derive predictions for these particular empirical investigations from the model. Stochastic Optimality Theory is one possible formulation of a proposal that derives empirical predictions from an OT model. However, I hope to have shown that it is not enough to take frequency distributions and relative acceptabilities at face value, and simply construe some Stochastic OT model that fits the facts. These facts first of all need to be interpreted, and those factors that the grammar has to account for must be sorted out from those about which grammar should have nothing to say. This task, to my mind, is more complicated than the picture that a simplistic application of (not only) Stochastic OT might draw

Graph Grammars, Insertion Lie Algebras, and Quantum Field Theory

Graph grammars extend the theory of formal languages in order to model distributed parallelism in theoretical computer science. We show here that to certain classes of context-free and context-sensitive graph grammars one can associate a Lie algebra, whose structure is reminiscent of the insertion Lie algebras of quantum field theory. We also show that the Feynman graphs of quantum field theories are graph languages generated by a theory dependent graph grammar.Comment: 19 pages, LaTeX, 3 jpeg figure

A short essay on the interplay between algebraic language theory, galois theory and class field theory : comparing physics and theory of computation (Mathematical aspects of quantum fields and related topics)

This paper is written as a technical report for our talk given at the RJMS workshop on quantum fields and related topics, held on 6th- 8th December 2021. In this talk we introduced our recent works [23, 24, 25, 26] in formal language theory to the community of mathematical physics, which concern some interplay between algebraic language theory, galois theory and class field theory. In this paper we discuss some conceptual contents of our recent works [23, 24, 25, 26] in more detail

Syntax-semantics interface: an algebraic model

We extend our formulation of Merge and Minimalism in terms of Hopf algebras to an algebraic model of a syntactic-semantic interface. We show that methods adopted in the formulation of renormalization (extraction of meaningful physical values) in theoretical physics are relevant to describe the extraction of meaning from syntactic expressions. We show how this formulation relates to computational models of semantics and we answer some recent controversies about implications for generative linguistics of the current functioning of large language models.Comment: LaTeX, 75 pages, 19 figure

Preliminares al estudio de la huella en lingüística

The present paper constitutes a brief advance of much longer and more detailed ongoing work on the concept of “trace” in contemporary linguistic theory, particularly in syntax. It is commonly believed that the idea was coined by Noam Chomsky. However, we already detect its use, with a very accurate value, in the early work of Zellig Harris on mathematical linguistics or, to be more precise, on mathematical structures of language. In its origins, rather than being an index responsible for marking the location occupied by a unit previous to its syntactic movement (which always takes the form of fronting ), the trace was the result of a matrix product between n-adic functions. Thus, in Harris the trace is primarily a concept anchored in matrix calculus, or, put it differently, an algebraic notion. Chomsky’s notion, on its turn, is closely related with the LISP programming language. This text seeks to provide a preliminary analysis of the conceptual complexity implied in the concept of trace, which linguists should become aware of, for otherwise they will be doomed to be entangled in misunderstandings unfruitful to our discipline for decades to come.El presente documento constituye un breve avance de una obra en curso mucho más larga y más detallada sobre el concepto de “huella” en la teoría lingüística contemporánea, particularmente en la sintaxis. Se cree, por lo común, que la idea fue acuñada por Noam Chomsky. Sin embargo, ya detectamos su uso, con un valor muy preciso, en los primeros trabajos de Zellig Harris sobre lingüística matemática o, para ser más exactos, sobre estructuras 2matemáticas del lenguaje. En sus orígenes, en lugar de ser un índice responsable de marcar la ubicación de una unidad antes de su movimiento sintáctico (que siempre toma la forma de fronting), la traza o huella era el resultado de un producto matricial entre funciones n-ádicas. Por lo tanto, en Harris la huella es principalmente un concepto anclado en el cálculo matricial o, dicho de otro modo, una noción algebraica. La noción de Chomsky, por su parte, está estrechamente relacionada con el lenguaje de programación LISP. EL presente texto busca proporcionar un análisis preliminar de la complejidad conceptual implícita en el concepto de huella, del cual los lingüistas deben tomar conciencia, porque de lo contrario estarán condenados a enredarse en malentendidos infructuosos para nuestra disciplina durante las próximas décadas

Lexical information from a minimalist point of view

Simplicity as a methodological orientation applies to linguistic theory just as to any other field of research: ‘Occam’s razor’ is the label for the basic heuristic maxim according to which an adequate analysis must ultimately be reduced to indispensible specifications. In this sense, conceptual economy has been a strict and stimulating guideline in the development of Generative Grammar from the very beginning. Halle’s (1959) argument discarding the level of taxonomic phonemics in order to unify two otherwise separate phonological processes is an early characteristic example; a more general notion is that of an evaluation metric introduced in Chomsky (1957, 1975), which relates the relative simplicity of alternative linguistic descriptions systematically to the quest for explanatory adequacy of the theory underlying the descriptions to be evaluated. Further proposals along these lines include the theory of markedness developed in Chomsky and Halle (1968), Kean (1975, 1981), and others, the notion of underspecification proposed e.g. in Archangeli (1984), Farkas (1990), the concept of default values and related notions. An important step promoting this general orientation was the idea of Principles and Parameters developed in Chomsky (1981, 1986), which reduced the notion of language particular rule systems to universal principles, subject merely to parametrization with restricted options, largely related to properties of particular lexical items. On this account, the notion of a simplicity metric is to be dispensed with, as competing analyses of relevant data are now supposed to be essentially excluded by the restrictive system of principles