16,662 research outputs found
Learning to Embed Words in Context for Syntactic Tasks
We present models for embedding words in the context of surrounding words.
Such models, which we refer to as token embeddings, represent the
characteristics of a word that are specific to a given context, such as word
sense, syntactic category, and semantic role. We explore simple, efficient
token embedding models based on standard neural network architectures. We learn
token embeddings on a large amount of unannotated text and evaluate them as
features for part-of-speech taggers and dependency parsers trained on much
smaller amounts of annotated data. We find that predictors endowed with token
embeddings consistently outperform baseline predictors across a range of
context window and training set sizes.Comment: Accepted by ACL 2017 Repl4NLP worksho
Convex Learning of Multiple Tasks and their Structure
Reducing the amount of human supervision is a key problem in machine learning
and a natural approach is that of exploiting the relations (structure) among
different tasks. This is the idea at the core of multi-task learning. In this
context a fundamental question is how to incorporate the tasks structure in the
learning problem.We tackle this question by studying a general computational
framework that allows to encode a-priori knowledge of the tasks structure in
the form of a convex penalty; in this setting a variety of previously proposed
methods can be recovered as special cases, including linear and non-linear
approaches. Within this framework, we show that tasks and their structure can
be efficiently learned considering a convex optimization problem that can be
approached by means of block coordinate methods such as alternating
minimization and for which we prove convergence to the global minimum.Comment: 26 pages, 1 figure, 2 table
Multi-GCN: Graph Convolutional Networks for Multi-View Networks, with Applications to Global Poverty
With the rapid expansion of mobile phone networks in developing countries,
large-scale graph machine learning has gained sudden relevance in the study of
global poverty. Recent applications range from humanitarian response and
poverty estimation to urban planning and epidemic containment. Yet the vast
majority of computational tools and algorithms used in these applications do
not account for the multi-view nature of social networks: people are related in
myriad ways, but most graph learning models treat relations as binary. In this
paper, we develop a graph-based convolutional network for learning on
multi-view networks. We show that this method outperforms state-of-the-art
semi-supervised learning algorithms on three different prediction tasks using
mobile phone datasets from three different developing countries. We also show
that, while designed specifically for use in poverty research, the algorithm
also outperforms existing benchmarks on a broader set of learning tasks on
multi-view networks, including node labelling in citation networks
- …