226 research outputs found
Conditional Random Field Autoencoders for Unsupervised Structured Prediction
We introduce a framework for unsupervised learning of structured predictors
with overlapping, global features. Each input's latent representation is
predicted conditional on the observable data using a feature-rich conditional
random field. Then a reconstruction of the input is (re)generated, conditional
on the latent structure, using models for which maximum likelihood estimation
has a closed-form. Our autoencoder formulation enables efficient learning
without making unrealistic independence assumptions or restricting the kinds of
features that can be used. We illustrate insightful connections to traditional
autoencoders, posterior regularization and multi-view learning. We show
competitive results with instantiations of the model for two canonical NLP
tasks: part-of-speech induction and bitext word alignment, and show that
training our model can be substantially more efficient than comparable
feature-rich baselines
Convolutional Neural Networks for Sentence Classification
We report on a series of experiments with convolutional neural networks (CNN)
trained on top of pre-trained word vectors for sentence-level classification
tasks. We show that a simple CNN with little hyperparameter tuning and static
vectors achieves excellent results on multiple benchmarks. Learning
task-specific vectors through fine-tuning offers further gains in performance.
We additionally propose a simple modification to the architecture to allow for
the use of both task-specific and static vectors. The CNN models discussed
herein improve upon the state of the art on 4 out of 7 tasks, which include
sentiment analysis and question classification.Comment: To appear in EMNLP 201
- …