Modelling syntactic development in a cross-linguistic context

Mainstream linguistic theory has traditionally assumed that children come into the world with rich innate knowledge about language and grammar. More recently, computational work using distributional algorithms has shown that the information contained in the input is much richer than proposed by the nativist approach. However, neither of these approaches has been developed to the point of providing detailed and quantitative predictions about the developmental data. In this paper, we champion a third approach, in which computational models learn from naturalistic input and produce utterances that can be directly compared with the utterances of language-learning children. We demonstrate the feasibility of this approach by showing how MOSAIC, a simple distributional analyser, simulates the optional-infinitive phenomenon in English, Dutch, and Spanish. The model accounts for young children's tendency to use both correct finites and incorrect (optional) infinitives in finite contexts, for the generality of this phenomenon across languages, and for the sparseness of other types of errors (e.g., word order errors). It thus shows how these phenomena, which have traditionally been taken as evidence for innate knowledge of Universal Grammar, can be explained in terms of a simple distributional analysis of the language to which children are exposed

Towards a Unified Model of Language Acquisition

In this theoretical paper, we first review and rebut standard criticisms against distributional approaches to language acquisition. We then present two closely-related models that use distributional analysis. The first deals with the acquisition of vocabulary, the second with grammatical development. We show how these two models can be combined with a semantic network grown using Hebbian learning, and briefly illustrate the advantages of this combination. An important feature of this hybrid system is that it combines two different types of distributional learning, the first based on order, and the second based on co-occurrences within a context

On the Utility of Conjoint and Compositional Frames and Utterance

This paper reports the results of a series of connectionist simulations aimed at establishing the value of different types of contexts as predictors of the grammatical categories of words. A comparison is made between ‘compositional’ frames (Monaghan & Christiansen, 2004), and non-compositional or ‘conjoint’ frames (Mintz, 2003). Attention is given to the role of utterance boundaries both as a category to be predicted and as a predictor. The role of developmental constraints is investigated by examining the effect of restricting the analysis to utterance-final frames. In line with results reported by Monaghan and Christiansen compositional frames are better predictors than conjoint frames, though the latter provide a small performance improvement when combined with compositional frames. Utterance boundaries are shown to be detrimental to performance when included as an item to be predicted while improving performance when included as a predictor. The utility of utterance boundaries is further supported by the finding that when the analysis is restricted to utterance-final frames (which are likely to be a particularly important source of information early in development) frames including utterance boundaries are far better predictors than lexical frames

Simulating optional infinitive errors in child speech through the omission of sentence-internal elements.

A new version of the MOSAIC model of syntax acquisition is presented. The modifications to the model aim to address two weaknesses in its earlier simulations of the Optional Infinitive phenomenon: an over-reliance on questions in the input as the source for Optional Infinitive errors, and the use of an utterance-final bias in learning (recency effect), without a corresponding utterance-initial bias (primacy effect). Where the old version only produced utterance-final phrases, the new version of MOSAIC learns from both the left and right edge of the utterance, and associates utterance-initial and utterancefinal phrases. The new model produces both utterance-final phrases and concatenations of utterance-final and utteranceinitial phrases. MOSAIC now also differentiates between phrases learned from declarative and interrogative input. It will be shown that the new version is capable of simulating the Optional Infinitive phenomenon in English and Dutch without relying on interrogative input. Unlike the previous version of MOSAIC, the new version is also capable of simulating cross-linguistic variation in the occurrence of Optional Infinitive errors in Wh-questions

Modeling the optional infinite stage in MOSAIC: A generalization to Dutch

This paper presents a model of a stage in children’s language development known as the optional infinitive stage. The model was originally developed for English, where it was shown to provide a good account of several phenomena. The model, which uses a discrimination network, analyzes the distribution of words in the input, and derives word classes from them by linking words that are used in a similar context. While the earlier version of the model is sensitive only to characteristics of phrases that follow target words, the present version also takes preceding input into consideration. Also, the present version uses a probabilistic rather than a deterministic learning mechanism. Generalisation of the model to Dutch is considered a strong test of the model, since Dutch displays the optional infinitive phenomenon, while its syntax differs substantially from that of English. The model was presented with child-directed input from two Dutch mothers, and its output was compared to that of the respective children. Despite the fact that the model was developed for a different language, it captures the optional infinitive phenomenon in Dutch as it does in English, while showing sensitivity to Dutch syntax. These results suggest that a simple distributional analyzer can capture the regularities of different languages despite the apparent differences in their syntax

Simulating the Noun-Verb Asymmetry in the Productivity of Children’s Speech

Several authors propose that children may acquire syntactic categories on the basis of co-occurrence statistics of words in the input. This paper assesses the relative merits of two such accounts by assessing the type and amount of productive language that results from computing co-occurrence statistics over conjoint and independent preceding and following contexts. This is achieved through the implementation of these methods in MOSAIC, a computational model of syntax acquisition that produces utterances that can be directly compared to child speech, and has a developmental component (i.e. produces increasingly long utterances). It is shown that the computation of co-occurrence statistics over conjoint contexts or frames results in a pattern of productive speech that more closely resembles that displayed by language learning children. The simulation of the developmental patterning of children’s productive speech furthermore suggests two refinements to this basic mechanism: inclusion of utterance boundaries, and the weighting of frames for their lexical content

Subject omission in children's language; The case for performance limitations in learning.

Several theories have been put forward to explain the phenomenon that children who are learning to speak their native language tend to omit the subject of the sentence. According to the pro-drop hypothesis, children represent the wrong grammar. According to the performance limitations view, children represent the full grammar, but omit subjects due to performance limitations in production. This paper proposes a third explanation and presents a model which simulates the data relevant to subject omission. The model consists of a simple learning mechanism that carries out a distributional analysis of naturalistic input. It does not have any overt representation of grammatical categories, and its performance limitations reside mainly in its learning mechanism. The model clearly simulates the data at hand, without the need to assume large amounts of innate knowledge in the child, and can be considered more parsimonious on these grounds alone. Importantly, it employs a unified and objective measure of processing load, namely the length of the utterance, which interacts with frequency in the input. The standard performance limitations view assumes that processing load is dependent on a phrase’s syntactic role, but does not specify a unifying underlying principle

Unifying cross-linguistic and within-language patterns of finiteness marking in MOSAIC

MOSAIC, a model that has already simulated cross-linguistic differences in the occurrence of the Optional Infinitive phenomenon, is applied to the simulation of the pattern of finiteness marking within Dutch. This within-language pattern, which includes verb placement, low rates of Optional Infinitives in Wh-questions and the correlation between finiteness marking and subject provision, has been taken as evidence for the view that children have correctly set the clause structure and inflectional parameters for their language. MOSAIC, which employs no built-in linguistic knowledge, clearly simulates the pattern of results as a function of its utterance-final bias, the same mechanism that is responsible for its successful simulation of the crosslinguistic data. These results suggest that both the crosslinguistic and within–language pattern of finiteness marking can be understood in terms of the interaction between a simple resource-limited learning mechanism and the distributional statistics of the input to which it is exposed. Thus, these phenomena do not provide any evidence for abstract or innate knowledge on the part of the child

The role of input size and generativity in simulating language acquisition.

This paper presents an analysis of the role of input size and generativity (ability to produce novel utterances) in simulating developmental data on a phenomenon in first language acquisition. An existing model that has already simulated the basic phenomenon is trained on input sets of varying sizes (13,000 to 40,000 utterances). The ability of the model to produce novel utterances is also manipulated. Both input size and generativity affect the fits for later stages of development. Higher generativity improves fits for later stages, but worsens them for early stages, suggesting generativity is best increased as a function of mean length of utterance (MLU). The effect of training set is variable. Results are discussed in terms of optimal training sets for simulations, and children’s developing ability to produce utterances beyond the input they have heard

Meter based omission of function words in MOSAIC

MOSAIC (Model of Syntax Acquisition in Children) is augmented with a new mechanism that allows for the omission of unstressed function words based on the prosodic structure of the utterance in which they occur. The mechanism allows MOSAIC to omit elements from multiple locations in a target utterance, which it was previously unable to do. It is shown that, although the new mechanism results in Optional Infinitive errors when run on children’s input, it is insufficient to simulate the high rate OI errors in children’s speech unless combined with MOSAIC’s edge-first learning mechanism. It is also shown that the addition of the new mechanism does not adversely affect MOSAIC’s fit to the Optional Infinitive phenomenon. The mechanism does, however, make MOSAIC’s output more child-like, both in terms of the range of utterances it can simulate, and the level and type of determiner omission that the model displays