Conditional Random Field Autoencoders for Unsupervised Structured Prediction

We introduce a framework for unsupervised learning of structured predictors with overlapping, global features. Each input's latent representation is predicted conditional on the observable data using a feature-rich conditional random field. Then a reconstruction of the input is (re)generated, conditional on the latent structure, using models for which maximum likelihood estimation has a closed-form. Our autoencoder formulation enables efficient learning without making unrealistic independence assumptions or restricting the kinds of features that can be used. We illustrate insightful connections to traditional autoencoders, posterior regularization and multi-view learning. We show competitive results with instantiations of the model for two canonical NLP tasks: part-of-speech induction and bitext word alignment, and show that training our model can be substantially more efficient than comparable feature-rich baselines

Establishing diagnostic criteria: the role of clinical pragmatics

The study of pragmatic disorders is of interest to speech-language pathologists who have a professional responsibility to assess and treat communication impairments. However, these disorders, it will be argued in this paper, have a significance beyond the clinical management of clients with communication impairments. Specifically, pragmatic disorders can now make a contribution to the diagnosis of a range of clinical conditions in which communication is adversely affected. These conditions include attention deficit hyperactivity disorder (ADHD), the autistic spectrum disorders, schizophrenia and the dementias. Pragmatic disorders are already among the criteria used to diagnose some of these conditions (e.g. ADHD), although they are not described in these terms. In other conditions (e.g. the dementias), pragmatic disorders have potential diagnostic value in the absence of reliable biomarkers markers of these conditions and similar initial presenting symptoms. Using clinical data, and the findings of empirical studies, the case is made for the inclusion and/or greater integration of pragmatic disorders in the formal classificatory systems that are used to diagnose a range of disorders. A previously unrecognised role for pragmatic impairments in the nosology and diagnosis of clinical disorders is thereby established

Sparse Coding of Neural Word Embeddings for Multilingual Sequence Labeling

In this paper we propose and carefully evaluate a sequence labeling framework which solely utilizes sparse indicator features derived from dense distributed word representations. The proposed model obtains (near) state-of-the art performance for both part-of-speech tagging and named entity recognition for a variety of languages. Our model relies only on a few thousand sparse coding-derived features, without applying any modification of the word representations employed for the different tasks. The proposed model has favorable generalization properties as it retains over 89.8% of its average POS tagging accuracy when trained at 1.2% of the total available training data, i.e.~150 sentences per language