Multi-VALUE: A Framework for Cross-Dialectal English NLP

Dialect differences caused by regional, social, and economic factors cause performance discrepancies for many groups of language technology users. Inclusive and equitable language technology must critically be dialect invariant, meaning that performance remains constant over dialectal shifts. Current systems often fall short of this ideal since they are designed and tested on a single dialect: Standard American English (SAE). We introduce a suite of resources for evaluating and achieving English dialect invariance. The resource is called Multi-VALUE, a controllable rule-based translation system spanning 50 English dialects and 189 unique linguistic features. Multi-VALUE maps SAE to synthetic forms of each dialect. First, we use this system to stress tests question answering, machine translation, and semantic parsing. Stress tests reveal significant performance disparities for leading models on non-standard dialects. Second, we use this system as a data augmentation technique to improve the dialect robustness of existing systems. Finally, we partner with native speakers of Chicano and Indian English to release new gold-standard variants of the popular CoQA task. To execute the transformation code, run model checkpoints, and download both synthetic and gold-standard dialectal benchmark datasets, see http://value-nlp.org.Comment: ACL 202

Using parent report to assess bilingual vocabulary acquisition: a model from Irish

This chapter describes the adaptation of a parent report instrument on early language development to a bilingual context. Beginning with general issues of adapting tests to any language, particular attention is placed on the issue of using parents as evaluators of child language acquisition of a minority language in a bilingual context. In Ireland, Irish is the first official language and is spoken by about 65,000 people on a daily basis. However all Irish speakers are bilingual, and children are exposed to the dominant English language at an early age. Using an adaptation of a parent report instrument, 21 typically developing children between 16 and 40 months were assessed repeatedly over two years to monitor their language development. The form allowed parents to document their children’s vocabulary development in both languages. Results showed that when knowledge of both languages was accounted for, the children acquired vocabulary at rates similar to those of monolingual speakers and used translational equivalents relatively early in language development. The study also showed that parents of bilingual children could accurately identify and differentiate language development in both of the child’s languages. Recommendations for adapting and using parent report instruments in bilingual language acquisition contexts are outlined