Imitation Learning for Neural Morphological String Transduction

We employ imitation learning to train a neural transition-based string transducer for morphological tasks such as inflection generation and lemmatization. Previous approaches to training this type of model either rely on an external character aligner for the production of gold action sequences, which results in a suboptimal model due to the unwarranted dependence on a single gold action sequence despite spurious ambiguity, or require warm starting with an MLE model. Our approach only requires a simple expert policy, eliminating the need for a character aligner or warm start. It also addresses familiar MLE training biases and leads to strong and state-of-the-art performance on several benchmarks.Comment: 6 pages; accepted to EMNLP 201

CUNI-Malta system at SIGMORPHON 2019 shared task on morphological analysis and lemmatization in context : operation-based word formation

This paper presents the submission by the Charles University-University of Malta team to the SIGMORPHON 2019 Shared Task on Morphological Analysis and Lemmatization in context. We present a lemmatization model based on previous work on neural transducers (Makarov and Clematide, 2018b; Aharoni and Goldberg, 2016). The key difference is that our model transforms the whole word form in every step, instead of consuming it character by character. We propose a merging strategy inspired by Byte-Pair-Encoding that reduces the space of valid operations by merging frequent adjacent operations. The resulting operations not only encode the actions to be performed but the relative position in the word token and how characters need to be transformed. Our morphological tagger is a vanilla biLSTM tagger that operates over operation representations, encoding operations and words in a hierarchical manner. Even though relative performance according to metrics is below the baseline, experiments show that our models capture important associations between interpretable operation labels and fine-grained morpho-syntax labelspeer-reviewe

The shape of the language-ready brain

Our core hypothesis is that the emergence of our species-specific language-ready brain ought to be understood in light of the developmental changes expressed at the levels of brain morphology and neural connectivity that occurred in our species after the split from Neanderthals-Denisovans and that gave us a more globular braincase configuration. In addition to changes at the cortical level, we hypothesize that the anatomical shift that led to globularity also entailed significant changes at the subcortical level. We claim that the functional consequences of such changes must also be taken into account to gain a fuller understanding of our linguistic capacity. Here we focus on the thalamus, which we argue is central to language and human cognition, as it modulates fronto-parietal activity. With this new neurobiological perspective in place, we examine its possible molecular basis. We construct a candidate gene set whose members are involved in the development and connectivity of the thalamus, in the evolution of the human head, and are known to give rise to language-associated cognitive disorders. We submit that the new gene candidate set opens up new windows into our understanding of the genetic basis of our linguistic capacity. Thus, our hypothesis aims at generating new testing grounds concerning core aspects of language ontogeny and phylogeny