Tabula nearly rasa: probing the linguistic knowledge of character-level neural language models trained on unsegmented text

Recurrent neural networks (RNNs) have reached striking performance in many natural language processing tasks. This has renewed interest in whether these generic sequence processing devices are inducing genuine linguistic knowledge. Nearly all current analytical studies, however, initialize the RNNs with a vocabulary of known words, and feed them tokenized input during training. We present a multi-lingual study of the linguistic knowledge encoded in RNNs trained as character-level language models, on input data with word boundaries removed. These networks face a tougher and more cognitively realistic task, having to discover any useful linguistic unit from scratch based on input statistics. The results show that our “near tabula rasa” RNNs are mostly able to solve morphological, syntactic and semantic tasks that intuitively presuppose word-level knowledge, and indeed they learned, to some extent, to track word boundaries. Our study opens the door to speculations about the necessity of an explicit, rigid word lexicon in language learning and usage

Exonic deletions in AUTS2 cause a syndromic form of intellectual disability and suggest a critical role for the C terminus.

Genomic rearrangements involving AUTS2 (7q11.22) are associated with autism and intellectual disability (ID), although evidence for causality is limited. By combining the results of diagnostic testing of 49,684 individuals, we identified 24 microdeletions that affect at least one exon of AUTS2, as well as one translocation and one inversion each with a breakpoint within the AUTS2 locus. Comparison of 17 well-characterized individuals enabled identification of a variable syndromic phenotype including ID, autism, short stature, microcephaly, cerebral palsy, and facial dysmorphisms. The dysmorphic features were more pronounced in persons with 3'AUTS2 deletions. This part of the gene is shown to encode a C-terminal isoform (with an alternative transcription start site) expressed in the human brain. Consistent with our genetic data, suppression of auts2 in zebrafish embryos caused microcephaly that could be rescued by either the full-length or the C-terminal isoform of AUTS2. Our observations demonstrate a causal role of AUTS2 in neurocognitive disorders, establish a hitherto unappreciated syndromic phenotype at this locus, and show how transcriptional complexity can underpin human pathology. The zebrafish model provides a valuable tool for investigating the etiology of AUTS2 syndrome and facilitating gene-function analysis in the future