Do not forget: Full memory in memory-based learning of word pronunciation

Memory-based learning, keeping full memory of learning material, appears a viable approach to learning NLP tasks, and is often superior in generalisation accuracy to eager learning approaches that abstract from learning material. Here we investigate three partial memory-based learning approaches which remove from memory specific task instance types estimated to be exceptional. The three approaches each implement one heuristic function for estimating exceptionality of instance types: (i) typicality, (ii) class prediction strength, and (iii) friendly-neighbourhood size. Experiments are performed with the memory-based learning algorithm IB1-IG trained on English word pronunciation. We find that removing instance types with low prediction strength (ii) is the only tested method which does not seriously harm generalisation accuracy. We conclude that keeping full memory of types rather than tokens, and excluding minority ambiguities appear to be the only performance-preserving optimisations of memory-based learning.Comment: uses conll98, epsf, and ipamacs (WSU IPA

Forgetting Exceptions is Harmful in Language Learning

We show that in language learning, contrary to received wisdom, keeping exceptional training instances in memory can be beneficial for generalization accuracy. We investigate this phenomenon empirically on a selection of benchmark natural language processing tasks: grapheme-to-phoneme conversion, part-of-speech tagging, prepositional-phrase attachment, and base noun phrase chunking. In a first series of experiments we combine memory-based learning with training set editing techniques, in which instances are edited based on their typicality and class prediction strength. Results show that editing exceptional instances (with low typicality or low class prediction strength) tends to harm generalization accuracy. In a second series of experiments we compare memory-based learning and decision-tree learning methods on the same selection of tasks, and find that decision-tree learning often performs worse than memory-based learning. Moreover, the decrease in performance can be linked to the degree of abstraction from exceptions (i.e., pruning or eagerness). We provide explanations for both results in terms of the properties of the natural language processing tasks and the learning algorithms.Comment: 31 pages, 7 figures, 10 tables. uses 11pt, fullname, a4wide tex styles. Pre-print version of article to appear in Machine Learning 11:1-3, Special Issue on Natural Language Learning. Figures on page 22 slightly compressed to avoid page overloa

Learning Exceptionality and Variation with Lexically Scaled MaxEnt

A growing body of research in phonology addresses the representation and learning of variable processes and exceptional, lexically conditioned processes. Linzen et al. (2013) present a MaxEnt model with additive lexical scales to account for data exhibiting both variation and exceptionality. In this paper, we implement a learning model for lexically scaled MaxEnt grammars which we show to be successful across a range of data containing patterns of variation and exceptionality. We also explore how the model\u27s parameters and the rate of exceptionality in the data influence its performance and predictions for novel forms

Self-Paced and Video-Based Learning: Parent Training and Language Skills in Japanese Children with ASD

While no exact information on the prevalence exists, it is assumed that the overall incidence of children with autism spectrum disorder (ASD) has risen every year in Japan. However, given the lack of resources and services for families of children with ASD in Japan, there is a dearth of practical guidance for the support for those families. This study examined the effects of an asynchronous training package (i.e., self-paced and video-based learning manual) to teach two Japanese mothers to implement incidental teaching. Effectiveness of the instruction was determined using a multiple-baseline design across mother–child dyads. Results indicated that the mother participants were able to implement the intervention with high fidelity over time. However, mixed effects of the mother-delivered intervention on target language behaviours were found across the child participants’ behaviours. This study adds an evidence to support that parents can be essential and efficient intervention agents for children with ASD

Emergent Typological Effects of Agent-Based Learning Models in Maximum Entropy Grammar

This dissertation shows how a theory of grammatical representations and a theory of learning can be combined to generate gradient typological predictions in phonology, predicting not only which patterns are expected to exist, but also their relative frequencies: patterns which are learned more easily are predicted to be more typologically frequent than those which are more difficult. In Chapter 1 I motivate and describe the specific implementation of this methodology in this dissertation. Maximum Entropy grammar (Goldwater & Johnson 2003) is combined with two agent-based learning models, the iterated and the interactive learning model, each of which mimics a type of learning dynamic observed in natural language acquisition. In Chapter 2 I illustrate how this system works using a simplified, abstract example typology, and show how the models generate a bias away from patterns which rely on cumulative constraint interaction ( gang effects ), and a bias away from variable patterns. Both of these biases match observed trends in natural language typology and psycholinguistic experiments. Chapter 3 further explores the models\u27 bias away from cumulative constraint interaction using an empirical test case: the typology of possible patterns of contrast between two fricatives. This typology yields five possible patterns, the rarest of which is the result of a gang effect. The results of simulations performed with both models produce a bias against the gang effect pattern. Chapter 4 further explores the models\u27 bias away from variation using evidence from artificial grammar learning experiments, in which human participants show a bias away from variable patterns (e.g. Smith & Wonnacott 2010). This test case was chosen additionally to disambiguate between variable behavior within a lexical item (variation), and variable behavior across lexical items (exceptionality). The results of simulations performed with both learning models are consistent with the observed bias away from variable patterns in humans. The results of the iterated and interactive learning models presented in this dissertation provide support for the use of this methodology in investigating the typological predictions of linguistic theories of grammar and learning, as well as in addressing broader questions regarding the source of gradient typological trends, and whether certain properties of natural language must be innately specified, or might emerge through other means

Critical Literacy: Deaf Adults Speak Out

The purpose of this paper is to describe a variety of teaching and learning strate-gies that were used within a classroom of Deaf adults participating in a high school English course as part of an upgrading program. The class was conducted in a bilingual manner; that is, being Deaf and communicating with American Sign Language (ASL) was not regarded as a deficit, but as a cultural experience com-parable to and distinct from cultures based on oral languages. The students‟ knowledge of ASL was used to help them develop their skills in English literacy. The emphasis in the classroom was to empower students to take responsibility for their own learning. Teaching activities were designed to help students create meaning around larger social issues. The goal was to improve their English read-ing and writing skills, and help them relate to what was happening in the world around them and lead them into action

Social-Skill Interventions for Culturally and Linguistically Diverse Students with Disabilities: A Comprehensive Review

Teachers and researchers have considered social-skill interventions to be an essential component in the development and progress of students with disabilities. However, there is still relatively limited research on these interventions for individuals from culturally and linguistically diverse (CLD) backgrounds. This literature review was conducted to examine the effectiveness of social-skill interventions for CLD students with disabilities in school settings. Electronic database searches and a manual search were completed to identify studies published between 2000 and 2017 (February). Seven studies (n = 18 participants) were identified for inclusion in this review, and five types of social interventions were identified. Most participants were male, aged between 8 and 13 years old, were considered at risk for having developmental delay or had developmental delay, and were identified as African Americans. The majority of studies we reviewed utilized single-subject research designs and focused on social interactions as the goal for their individual interventions. Peer-mediated interventions and social story intervention were the most frequently used interventions. Findings suggest that, when exposed to the social-skill interventions, CLD children with disabilities improved their social behaviours and skills. Some children with disabilities maintained and generalized these behaviours across settings or playmates