109 research outputs found
Response Generation by Context-aware Prototype Editing
Open domain response generation has achieved remarkable progress in recent
years, but sometimes yields short and uninformative responses. We propose a new
paradigm for response generation, that is response generation by editing, which
significantly increases the diversity and informativeness of the generation
results. Our assumption is that a plausible response can be generated by
slightly revising an existing response prototype. The prototype is retrieved
from a pre-defined index and provides a good start-point for generation because
it is grammatical and informative. We design a response editing model, where an
edit vector is formed by considering differences between a prototype context
and a current context, and then the edit vector is fed to a decoder to revise
the prototype response for the current context. Experiment results on a large
scale dataset demonstrate that the response editing model outperforms
generative and retrieval-based models on various aspects
Unsupervised Lexical Substitution with Decontextualised Embeddings
We propose a new unsupervised method for lexical substitution using
pre-trained language models. Compared to previous approaches that use the
generative capability of language models to predict substitutes, our method
retrieves substitutes based on the similarity of contextualised and
decontextualised word embeddings, i.e. the average contextual representation of
a word in multiple contexts. We conduct experiments in English and Italian, and
show that our method substantially outperforms strong baselines and establishes
a new state-of-the-art without any explicit supervision or fine-tuning. We
further show that our method performs particularly well at predicting
low-frequency substitutes, and also generates a diverse list of substitute
candidates, reducing morphophonetic or morphosyntactic biases induced by
article-noun agreement.Comment: 14 pages, accepted for COLING 202
Graph Reasoning for Question Answering with Triplet Retrieval
Answering complex questions often requires reasoning over knowledge graphs
(KGs). State-of-the-art methods often utilize entities in questions to retrieve
local subgraphs, which are then fed into KG encoder, e.g. graph neural networks
(GNNs), to model their local structures and integrated into language models for
question answering. However, this paradigm constrains retrieved knowledge in
local subgraphs and discards more diverse triplets buried in KGs that are
disconnected but useful for question answering. In this paper, we propose a
simple yet effective method to first retrieve the most relevant triplets from
KGs and then rerank them, which are then concatenated with questions to be fed
into language models. Extensive results on both CommonsenseQA and OpenbookQA
datasets show that our method can outperform state-of-the-art up to 4.6%
absolute accuracy.Comment: Findings of ACL 202
Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters
Natural Language Inference (NLI) has been extensively studied by the NLP
community as a framework for estimating the semantic relation between sentence
pairs. While early work identified certain biases in NLI models, recent
advancements in modeling and datasets demonstrated promising performance. In
this work, we further explore the direct zero-shot applicability of NLI models
to real applications, beyond the sentence-pair setting they were trained on.
First, we analyze the robustness of these models to longer and out-of-domain
inputs. Then, we develop new aggregation methods to allow operating over full
documents, reaching state-of-the-art performance on the ContractNLI dataset.
Interestingly, we find NLI scores to provide strong retrieval signals, leading
to more relevant evidence extractions compared to common similarity-based
methods. Finally, we go further and investigate whole document clusters to
identify both discrepancies and consensus among sources. In a test case, we
find real inconsistencies between Wikipedia pages in different languages about
the same topic.Comment: Findings of EMNLP 202
Effective and Efficient Similarity Search in Scientific Workflow Repositories
International audienceScientific workflows have become a valuable tool for large-scale data processing and analysis. This has led to the creation of specialized online repositories to facilitate worflkow sharing and reuse. Over time, these repositories have grown to sizes that call for advanced methods to support workflow discovery, in particular for similarity search. Effective similarity search requires both high quality algorithms for the comparison of scientific workflows and efficient strategies for indexing, searching, and ranking of search results. Yet, the graph structure of scientific workflows poses severe challenges to each of these steps. Here, we present a complete system for effective and efficient similarity search in scientific workflow repositories, based on the Layer Decompositon approach to scientific workflow comparison. Layer Decompositon specifically accounts for the directed dataflow underlying scientific workflows and, compared to other state-of-the-art methods, delivers best results for similarity search at comparably low runtimes. Stacking Layer Decomposition with even faster, structure-agnostic approaches allows us to use proven, off-the-shelf tools for workflow indexing to further reduce runtimes and scale similarity search to sizes of current repositories
Diversify Question Generation with Retrieval-Augmented Style Transfer
Given a textual passage and an answer, humans are able to ask questions with
various expressions, but this ability is still challenging for most question
generation (QG) systems. Existing solutions mainly focus on the internal
knowledge within the given passage or the semantic word space for diverse
content planning. These methods, however, have not considered the potential of
external knowledge for expression diversity. To bridge this gap, we propose
RAST, a framework for Retrieval-Augmented Style Transfer, where the objective
is to utilize the style of diverse templates for question generation. For
training RAST, we develop a novel Reinforcement Learning (RL) based approach
that maximizes a weighted combination of diversity reward and consistency
reward. Here, the consistency reward is computed by a Question-Answering (QA)
model, whereas the diversity reward measures how much the final output mimics
the retrieved template. Experimental results show that our method outperforms
previous diversity-driven baselines on diversity while being comparable in
terms of consistency scores. Our code is available at
https://github.com/gouqi666/RAST.Comment: EMNLP2023 camera-read
Large Language Models are Few-Shot Summarizers: Multi-Intent Comment Generation via In-Context Learning
Code comment generation aims at generating natural language descriptions for
a code snippet to facilitate developers' program comprehension activities.
Despite being studied for a long time, a bottleneck for existing approaches is
that given a code snippet, they can only generate one comment while developers
usually need to know information from diverse perspectives such as what is the
functionality of this code snippet and how to use it. To tackle this
limitation, this study empirically investigates the feasibility of utilizing
large language models (LLMs) to generate comments that can fulfill developers'
diverse intents. Our intuition is based on the facts that (1) the code and its
pairwise comment are used during the pre-training process of LLMs to build the
semantic connection between the natural language and programming language, and
(2) comments in the real-world projects, which are collected for the
pre-training, usually contain different developers' intents. We thus postulate
that the LLMs can already understand the code from different perspectives after
the pre-training. Indeed, experiments on two large-scale datasets demonstrate
the rationale of our insights: by adopting the in-context learning paradigm and
giving adequate prompts to the LLM (e.g., providing it with ten or more
examples), the LLM can significantly outperform a state-of-the-art supervised
learning approach on generating comments with multiple intents. Results also
show that customized strategies for constructing the prompts and
post-processing strategies for reranking the results can both boost the LLM's
performances, which shed light on future research directions for using LLMs to
achieve comment generation.Comment: Accepted by the 46th International Conference on Software Engineering
(ICSE 2024
- …