20 research outputs found
CharManteau: Character Embedding Models For Portmanteau Creation
Portmanteaus are a word formation phenomenon where two words are combined to
form a new word. We propose character-level neural sequence-to-sequence (S2S)
methods for the task of portmanteau generation that are end-to-end-trainable,
language independent, and do not explicitly use additional phonetic
information. We propose a noisy-channel-style model, which allows for the
incorporation of unsupervised word lists, improving performance over a standard
source-to-target model. This model is made possible by an exhaustive candidate
generation strategy specifically enabled by the features of the portmanteau
task. Experiments find our approach superior to a state-of-the-art FST-based
baseline with respect to ground truth accuracy and human evaluation.Comment: Accepted for publication in EMNLP 201
SPINE: SParse Interpretable Neural Embeddings
Prediction without justification has limited utility. Much of the success of
neural models can be attributed to their ability to learn rich, dense and
expressive representations. While these representations capture the underlying
complexity and latent trends in the data, they are far from being
interpretable. We propose a novel variant of denoising k-sparse autoencoders
that generates highly efficient and interpretable distributed word
representations (word embeddings), beginning with existing word representations
from state-of-the-art methods like GloVe and word2vec. Through large scale
human evaluation, we report that our resulting word embedddings are much more
interpretable than the original GloVe and word2vec embeddings. Moreover, our
embeddings outperform existing popular word embeddings on a diverse suite of
benchmark downstream tasks.Comment: AAAI 201
Interpreting User Requests in the Context of Natural Language Standing Instructions
Users of natural language interfaces, generally powered by Large Language
Models (LLMs),often must repeat their preferences each time they make a similar
request. We describe an approach to LLM-based dialogue modeling in which
persistent user constraints and preferences -- collectively termed standing
instructions -- as additional context for such interfaces. For example, when a
user states "I'm hungry", a previously expressed preference for Persian food
can be automatically added to the LLM prompt, influencing the search for
relevant restaurants. We develop NLSI, a language-to-program dataset consisting
of over 2.4K dialogues spanning 17 domains, where each dialogue is paired with
a user profile (a set of users specific standing instructions) and
corresponding structured representations (API calls). A key challenge in NLSI
is to identify which subset of the standing instructions is applicable to a
given dialogue. NLSI contains diverse phenomena, from simple preferences to
interdependent instructions such as triggering a hotel search whenever the user
is booking tickets to an event. We conduct experiments on NLSI using prompting
with large language models and various retrieval approaches, achieving a
maximum of 44.7% exact match on API prediction. Our results demonstrate the
challenges in identifying the relevant standing instructions and their
interpretation into API calls.Comment: Updated with results from LLaMA-