14 research outputs found
Knowledge-Augmented Large Language Models for Personalized Contextual Query Suggestion
Large Language Models (LLMs) excel at tackling various natural language
tasks. However, due to the significant costs involved in re-training or
fine-tuning them, they remain largely static and difficult to personalize.
Nevertheless, a variety of applications could benefit from generations that are
tailored to users' preferences, goals, and knowledge. Among them is web search,
where knowing what a user is trying to accomplish, what they care about, and
what they know can lead to improved search experiences. In this work, we
propose a novel and general approach that augments an LLM with relevant context
from users' interaction histories with a search engine in order to personalize
its outputs. Specifically, we construct an entity-centric knowledge store for
each user based on their search and browsing activities on the web, which is
then leveraged to provide contextually relevant LLM prompt augmentations. This
knowledge store is light-weight, since it only produces user-specific aggregate
projections of interests and knowledge onto public knowledge graphs, and
leverages existing search log infrastructure, thereby mitigating the privacy,
compliance, and scalability concerns associated with building deep user
profiles for personalization. We then validate our approach on the task of
contextual query suggestion, which requires understanding not only the user's
current search context but also what they historically know and care about.
Through a number of experiments based on human evaluation, we show that our
approach is significantly better than several other LLM-powered baselines,
generating query suggestions that are contextually more relevant, personalized,
and useful
Making Large Language Models Better Data Creators
Although large language models (LLMs) have advanced the state-of-the-art in
NLP significantly, deploying them for downstream applications is still
challenging due to cost, responsiveness, control, or concerns around privacy
and security. As such, trainable models are still the preferred option in some
cases. However, these models still require human-labeled data for optimal
performance, which is expensive and time-consuming to obtain. In order to
address this issue, several techniques to reduce human effort involve labeling
or generating data using LLMs. Although these methods are effective for certain
applications, in practice they encounter difficulties in real-world scenarios.
Labeling data requires careful data selection, while generating data
necessitates task-specific prompt engineering. In this paper, we propose a
unified data creation pipeline that requires only a single formatting example,
and which is applicable to a broad range of tasks, including traditionally
problematic ones with semantically devoid label spaces. In our experiments we
demonstrate that instruction-following LLMs are highly cost-effective data
creators, and that models trained with these data exhibit performance better
than those trained with human-labeled data (by up to 17.5%) on
out-of-distribution evaluation, while maintaining comparable performance on
in-distribution tasks. These results have important implications for the
robustness of NLP systems deployed in the real-world.Comment: Accepted to EMNLP 2023 main conference. 12 pages, 5 figures, 6
tables. Code is available at https://github.com/microsoft/llm-data-creatio
Prosody-Based Unsupervised Speech Summarization with Two-Layer Mutually Reinforced Random Walk
<p>This paper presents a graph-based model that integrates prosodic features into an unsupervised speech summarization framework without any lexical information. In particular it builds on previous work using mutually reinforced random walks, in which a two-layer graph structure is used to select the most salient utterances of a conversation. The model consists of one layer of utterance nodes and another layer of prosody nodes. The random walk algorithm propagates scores between layers to use shared information for selecting utterance nodes with highest scores as summaries. A comparative evaluation of our prosody-based model against several baselines on a corpus of academic multi-party meetings reveals that it performs competitively on very short summaries, and better on longer summaries according to ROUGE scores as well as the average relevance of selected utterances.</p
A structured distributional semantic model for event co-reference
Abstract In this paper we present a novel approach to modelling distributional semantics that represents meaning as distributions over relations in syntactic neighborhoods. We argue that our model approximates meaning in compositional configurations more effectively than standard distributional vectors or bag-of-words models. We test our hypothesis on the problem of judging event coreferentiality, which involves compositional interactions in the predicate-argument structure of sentences, and demonstrate that our model outperforms both state-of-the-art window-based word embeddings as well as simple approaches to compositional semantics previously employed in the literature