How Well Do LSTM Language Models Learn Filler-gap Dependencies?

This paper revisits the question of what LSTMs know about the syntax of filler-gap dependencies in English. One contribution of this paper is to adjust the metrics used by Wilcox et al. 2018 and show that their language models (LMs) learn embedded wh-questions -- a kind of filler-gap dependencies -- better than they originally claimed. Another contribution of this paper is to examine four additional filler-gap dependency constructions to see whether LMs perform equally on all types of filler-gap dependencies. We find that different constructions are learned to different extents, and there is a correlation between performance and frequency of constructions in the Penn Treebank Wall Street Journal corpus

A common framework for quantifying the learnability of nouns and verbs

Across the world's languages, children reliably learn nouns more easily than verbs. Attempts to understand the difficulty of verb learning have focused on determining whether the challenge stems from differences in the linguistic usage of nouns and verbs, or instead conceptual differences in the categories that they label. We introduce a novel metric to quantify the contributions of both sources of difficulty using unsupervised learning models trained on corpora of language and images. We find that there is less alignment between the linguistic usage of verbs and their categories than for nouns and their categories. However, this difference is driven almost entirely by differences in the structure of their visual categories: Relative to nouns, events described by the same verb are more variable and events described by two different verbs are more similar. We conclude that differences between noun and verb learning need not be due to fundamental differences in learning processes, but may instead be driven by the difficulty of one-shot generalization from verbs' visual categories

Communicative pressure can lead to input that supports language learning

While children must learn language from the statistical structure of the input they receive, parents play a critical role shap-ing the structure of this input. Even without an explicit pedagogical goal, parents’ desire to communicate successfully maycause them to produce language calibrated to their child’s linguistic development. We designed a Mechanical Turk studyto experimentally validate this idea, putting Turkers in the role of parents talking with children less familiar with a novellanguage. Participants could communicate in 3 ways: pointingexpensive but unambiguous, labelingcheap but knowledge-dependent, or both. They won points only for communicating successfully. Participants adapted their communicativebehavior to their own knowledge and their partners knowledge. Teaching emerged when the speaker had more linguisticknowledge than their partner. We implemented a rational planning model that fits these data and demonstrates that suchpatterns could result from maximizing expected utilities, accounting for the expected utilities of future interactions

Interlocutors preserve complexity in language

Why do languages change? One possibility is they evolve in response to two competing pressures: (1) to be easily learned,and (2) to be effective for communication. In a number of domains, variation in the worlds natural languages appears tobe accounted for by different but near-optimal tradeoffs between these pressures. Models of these evolutionary processeshave used transmission chain paradigms in which errors of learning by one agent become the input for the subsequentgeneration. However, a critical feature of human language is that children do not learn in isolation. Rather, they learn incommunicative interactions with caregivers who draw inferences from their errorful productions to their intended interests.In a set of iterated reproduction experiments, we show that this supportive context can have a powerful stabilizing role inthe development of artificial patterned systems, allowing them to achieve higher levels of complexity than they would byvertical transmission alone while retaining equivalent transmission accuracies

Listeners use descriptive contrast to disambiguate novel referents

People often face referential ambiguity; one cue to resolve it is adjectival description. Beyond narrowing potential referentsto those that match a descriptor, listeners may infer that a described object is one that contrasts with other present objectsof the same type (tall cup contrasts with another, shorter cup). This contrastive inference guides the visual identificationof a familiar referent as an utterance progresses (Sedivy et al., 1999). We extend this work, asking whether listeners usethis type of inference to guide explicit referent choice when reference is ambiguous, and whether this varies with adjectivetype. We find that participants consistently use size adjectives contrastively, but not color adjectives (Experiment 1)evenwhen color is described with more relative language (Experiment 2) or emphasized with prosodic stress (Experiment 3).Listeners can use adjective contrast to disambiguate a novel words referent, but do not treat all adjective types as equallycontrastive

Lexical diversity and language development

Previous research has demonstrated a relationship between quantity of language input and childrens rate of languagedevelopment: Children who hear more words learn faster. This work takes on two mutually-constraining questions:(1) How should we define quality, and (2) what is the relationship between input quality and language development?We analyzed a longitudinal corpus of interactions between 50 children and their parents using four measures of lexicaldiversity: Type Token Ratio (TTR), Moving Average TTR, and two more recent measuresvocd-D and MTLD. We foundthat only MTLD gave a prima-facie correct characterization of childrens development, and parents MTLD was correlatedwith childrens over development. Results of simulations showed that MTLD was distinct from the other measures in itssensitivity to both lexical diversity and word order, suggesting that quality should be defined not just by diversity of words,but also by the variability of sentence structures in which they occur

Available referents and prompt specificity influence induction of feature typicality

Prior work suggests that speakers and listeners use discourse pragmatics to constrain potential referents and make infer-ences about the relationship of a novel referent to its category. This work addresses the use of discourse specificity andavailable referents in combination to make inferences about category feature typicality. In a visual search task and sub-sequent typicality rating task, participants ratings of typicality for an novel object’s color were affected by whether theobjects color was specified in the search prompt (e.g., Find the (blue) dax), the color of distractor objects (same as ordifferent from target), and the shape of distractor objects (same as or different from target). Specification of target colorin the prompt decreased typicality ratings, in keeping with work suggesting that over-informative utterances can induceinference of atypicality

Information Distribution Depends on Language-Specific Features

Language can be thought of as a code: A system for packaging a speakers thoughts into a signal that a listener mustdecode to recover some intended meaning. If language is a near-optimal code, then speakers should structure informationin their utterances to minimizes the impact of errors in production or comprehension. To examine the distribution ofinformation within utterances, we apply information-theoretic methods to a diverse set of languages in various spoken andwritten corpora. We find reliably non-uniform and cross-linguistically variable information distributions across languages.These distributions are consistent across contexts, and are predictable from typological features, most notably canonicalword order. However, when we include even a small amount of predictive context (bigrams or trigrams), the language-specific shapes disappear, and all languages are characterized by uniform information distribution. Despite cross-linguisticvariability in communicative codes, speakers structure their utterances to preserve uniform information distribution andsupport successful communication

Interpretation of Generic Language is Dependent on Listener’s BackgroundKnowledge

Generic statements, like ”birds lay eggs” or ”dogs bark” are simple and ubiquitous in naturally produced speech. However,the inherent vagueness of generics makes their interpretation highly context-dependent. Building on work by Tessler &Goodman (in press) showing that generics can be thought of as inherently relative (i.e. more birds lay eggs than youwould expect), we explore the consequences of different implied comparison categories on the interpretation of novelgenerics. In Experiments 1 and 2, we manipulated the set of categories salient to a listener by directly providing themthe comparison sets. In Experiments 3 and 4, we collected participants demographic information and used these naturallyoccurring differences as a basis for differences in the participants’ comparison sets. Our results confirmed the hypothesisthat prevalence judgments of features in novel categories are sensitive to differences in their corresponding comparisoncategories. These results suggest a possible source for well-intentioned miscommunications

Of mouses and mans: the role of production and feedback in language learning

Do children learn language from the words that they produce themselves? Because children know that they have imperfect knowledge of language, they could simply ignore their own productions. However, children could also learn from their productions -- using what they say and how their caregivers respond to update their language models. Using irregular plurals as a case study, we conducted a large-scale corpus analysis and two experimental studies to understand the role of children's productions and caregivers' responses in language learning. We demonstrate that children do learn from their own production, with errorful utterances leading to more errors. However, at least in some contexts, children can use implicit corrections from parents to offset the negative effects of their errors. Children thus appear to learn not only from their caregivers' productions, but also from their own productions and from the relationship between the two