Linear indexed languages

AbstractIn this paper one characterization of linear indexed languages based on controlling linear context-free grammars with context-free languages and one based on homomorphic images of context-free languages are given. By constructing a generator for the family of linear indexed languages, it is shown that this family is a full principal semi-AFL. Furthermore a Parikh theorem for linear indexed languages is stated which implies that there are indexed languages which are not linear

On Finite-Index Indexed Grammars and Their Restrictions

The family, L(INDLIN), of languages generated by linear indexed grammars has been studied in the literature. It is known that the Parikh image of every language in L(INDLIN) is semi-linear. However, there are bounded semi-linear languages that are not in L(INDLIN). Here, we look at larger families of (restricted) indexed languages and study their combinatorial and decidability properties, and their relationships

Tree-Adjoining Grammars and Lexicalized Grammars

In this paper, we will describe a tree generating system called tree-adjoining grammar(TAG)and state some of the recent results about TAGs. The work on TAGS is motivated by linguistic considerations. However, a number of formal results have been established for TAGs, which we believe, would be of interest to researchers in tree grammars and tree automata. After giving a short introduction to TAG, we briefly state these results concerning both the properties of the string sets and tree sets (Section 2). We will also describe the notion of lexicalization of grammars (Section 3) and investigate the relationship of lexicalization to context-free grammars (CFGs) and TAGS (Section 4)

The range of non-linear natural polynomials cannot be context-free

Suppose that some polynomial $f$ with rational coefficients takes only natural values at natural numbers, i.e., $L=\{f(n)\mid n\in \mathbb N\}\subset\mathbb N$. We show that the base-$q$ representation of $L$ is a context-free language if and only if $f$ is linear, answering a question of Shallit. The proof is based on a new criterion for context-freeness, which is a combination of the Interchange lemma and a generalization of the Pumping lemma.Comment: This paper should be assigned to cs.FL, but I'm not endorsed over ther

An Alternative Conception of Tree-Adjoining Derivation

The precise formulation of derivation for tree-adjoining grammars has important ramifications for a wide variety of uses of the formalism, from syntactic analysis to semantic interpretation and statistical language modeling. We argue that the definition of tree-adjoining derivation must be reformulated in order to manifest the proper linguistic dependencies in derivations. The particular proposal is both precisely characterizable through a definition of TAG derivations as equivalence classes of ordered derivation trees, and computationally operational, by virtue of a compilation to linear indexed grammars together with an efficient algorithm for recognition and parsing according to the compiled grammar.Comment: 33 page

Calibrating Generative Models: The Probabilistic Chomsky-Schützenberger Hierarchy

A probabilistic Chomsky–Schützenberger hierarchy of grammars is introduced and studied, with the aim of understanding the expressive power of generative models. We offer characterizations of the distributions definable at each level of the hierarchy, including probabilistic regular, context-free, (linear) indexed, context-sensitive, and unrestricted grammars, each corresponding to familiar probabilistic machine classes. Special attention is given to distributions on (unary notations for) positive integers. Unlike in the classical case where the "semi-linear" languages all collapse into the regular languages, using analytic tools adapted from the classical setting we show there is no collapse in the probabilistic hierarchy: more distributions become definable at each level. We also address related issues such as closure under probabilistic conditioning