6 research outputs found
Output Space Search for Structured Prediction
We consider a framework for structured prediction based on search in the
space of complete structured outputs. Given a structured input, an output is
produced by running a time-bounded search procedure guided by a learned cost
function, and then returning the least cost output uncovered during the search.
This framework can be instantiated for a wide range of search spaces and search
procedures, and easily incorporates arbitrary structured-prediction loss
functions. In this paper, we make two main technical contributions. First, we
define the limited-discrepancy search space over structured outputs, which is
able to leverage powerful classification learning algorithms to improve the
search space quality. Second, we give a generic cost function learning
approach, where the key idea is to learn a cost function that attempts to mimic
the behavior of conducting searches guided by the true loss function. Our
experiments on six benchmark domains demonstrate that using our framework with
only a small amount of search is sufficient for significantly improving on
state-of-the-art structured-prediction performance.Comment: Appears in Proceedings of the 29th International Conference on
Machine Learning (ICML 2012
Parser for Abstract Meaning Representation using Learning to Search
We develop a novel technique to parse English sentences into Abstract Meaning
Representation (AMR) using SEARN, a Learning to Search approach, by modeling
the concept and the relation learning in a unified framework. We evaluate our
parser on multiple datasets from varied domains and show an absolute
improvement of 2% to 6% over the state-of-the-art. Additionally we show that
using the most frequent concept gives us a baseline that is stronger than the
state-of-the-art for concept prediction. We plan to release our parser for
public use
Learning Reductions that Really Work
We provide a summary of the mathematical and computational techniques that
have enabled learning reductions to effectively address a wide class of
problems, and show that this approach to solving machine learning problems can
be broadly useful
A Credit Assignment Compiler for Joint Prediction
Many machine learning applications involve jointly predicting multiple
mutually dependent output variables. Learning to search is a family of methods
where the complex decision problem is cast into a sequence of decisions via a
search space. Although these methods have shown promise both in theory and in
practice, implementing them has been burdensomely awkward. In this paper, we
show the search space can be defined by an arbitrary imperative program,
turning learning to search into a credit assignment compiler. Altogether with
the algorithmic improvements for the compiler, we radically reduce the
complexity of programming and the running time. We demonstrate the feasibility
of our approach on multiple joint prediction tasks. In all cases, we obtain
accuracies as high as alternative approaches, at drastically reduced execution
and programming time
Rectifying Classifier Chains for Multi-Label Classification
Classifier chains have recently been proposed as an appealing method for
tackling the multi-label classification task. In addition to several empirical
studies showing its state-of-the-art performance, especially when being used in
its ensemble variant, there are also some first results on theoretical
properties of classifier chains. Continuing along this line, we analyze the
influence of a potential pitfall of the learning process, namely the
discrepancy between the feature spaces used in training and testing: While true
class labels are used as supplementary attributes for training the binary
models along the chain, the same models need to rely on estimations of these
labels at prediction time. We elucidate under which circumstances the attribute
noise thus created can affect the overall prediction performance. As a result
of our findings, we propose two modifications of classifier chains that are
meant to overcome this problem. Experimentally, we show that our variants are
indeed able to produce better results in cases where the original chaining
process is likely to fail.Comment: 18 pages, 3 figures, 4 tables, extended version of: Robin Senge,
Jos\'e del Coz, E. H\"ullermeier. Rectifying Classifier Chains for
Multi-Label Classification. Proceedings Workshop LWA 2013,
Lernen-Wissensentdeckung-Adaptivit\"at,151-158, Bamberg, Germany, 201
Output space search for structured prediction
We consider a framework for structured prediction based on search in the space of complete structured outputs. Given a structured input, an output is produced by running a time-bounded search procedure guided by a learned cost function, and then returning the least cost output uncovered during the search. This framework can be instantiated for a wide range of search spaces and search procedures, and easily incorporates arbitrary structured-prediction loss functions. In this paper, we make two main technical contributions. First, we define the limited-discrepancy search space over structured outputs, which is able to leverage powerful classification learning algorithms to improve the search space quality. Second, we give a generic cost function learning approach, where the key idea is to learn a cost function that attempts to mimic the behavior of conducting searches guided by the true loss function. Our experiments on six benchmark domains demonstrate that using our framework with only a small amount of search is sufficient for significantly improving on state-of-the-art structuredprediction performance. 1