4,684 research outputs found
Harnessing Evolution of Multi-Turn Conversations for Effective Answer Retrieval
With the improvements in speech recognition and voice generation technologies
over the last years, a lot of companies have sought to develop conversation
understanding systems that run on mobile phones or smart home devices through
natural language interfaces. Conversational assistants, such as Google
Assistant and Microsoft Cortana, can help users to complete various types of
tasks. This requires an accurate understanding of the user's information need
as the conversation evolves into multiple turns. Finding relevant context in a
conversation's history is challenging because of the complexity of natural
language and the evolution of a user's information need. In this work, we
present an extensive analysis of language, relevance, dependency of user
utterances in a multi-turn information-seeking conversation. To this aim, we
have annotated relevant utterances in the conversations released by the TREC
CaST 2019 track. The annotation labels determine which of the previous
utterances in a conversation can be used to improve the current one.
Furthermore, we propose a neural utterance relevance model based on BERT
fine-tuning, outperforming competitive baselines. We study and compare the
performance of multiple retrieval models, utilizing different strategies to
incorporate the user's context. The experimental results on both classification
and retrieval tasks show that our proposed approach can effectively identify
and incorporate the conversation context. We show that processing the current
utterance using the predicted relevant utterance leads to a 38% relative
improvement in terms of nDCG@20. Finally, to foster research in this area, we
have released the dataset of the annotations.Comment: To appear in ACM CHIIR 2020, Vancouver, BC, Canad
A Study on the Implementation of Generative AI Services Using an Enterprise Data-Based LLM Application Architecture
This study presents a method for implementing generative AI services by
utilizing the Large Language Model (LLM) application architecture. With recent
advancements in generative AI technology, LLMs have gained prominence across
various domains. In this context, the research addresses the challenge of
information scarcity and proposes specific remedies by harnessing LLM
capabilities. The investigation delves into strategies for mitigating the issue
of inadequate data, offering tailored solutions. The study delves into the
efficacy of employing fine-tuning techniques and direct document integration to
alleviate data insufficiency. A significant contribution of this work is the
development of a Retrieval-Augmented Generation (RAG) model, which tackles the
aforementioned challenges. The RAG model is carefully designed to enhance
information storage and retrieval processes, ensuring improved content
generation. The research elucidates the key phases of the information storage
and retrieval methodology underpinned by the RAG model. A comprehensive
analysis of these steps is undertaken, emphasizing their significance in
addressing the scarcity of data. The study highlights the efficacy of the
proposed method, showcasing its applicability through illustrative instances.
By implementing the RAG model for information storage and retrieval, the
research not only contributes to a deeper comprehension of generative AI
technology but also facilitates its practical usability within enterprises
utilizing LLMs. This work holds substantial value in advancing the field of
generative AI, offering insights into enhancing data-driven content generation
and fostering active utilization of LLM-based services within corporate
settings
Query Performance Prediction:From Ad-hoc to Conversational Search
Query performance prediction (QPP) is a core task in information retrieval. The QPP task is to predict the retrieval quality of a search system for a query without relevance judgments. Research has shown the effectiveness and usefulness of QPP for ad-hoc search. Recent years have witnessed considerable progress in conversational search (CS). Effective QPP could help a CS system to decide an appropriate action to be taken at the next turn. Despite its potential, QPP for CS has been little studied. We address this research gap by reproducing and studying the effectiveness of existing QPP methods in the context of CS. While the task of passage retrieval remains the same in the two settings, a user query in CS depends on the conversational history, introducing novel QPP challenges. In particular, we seek to explore to what extent findings from QPP methods for ad-hoc search generalize to three CS settings: (i) estimating the retrieval quality of different query rewriting-based retrieval methods, (ii) estimating the retrieval quality of a conversational dense retrieval method, and (iii) estimating the retrieval quality for top ranks vs. deeper-ranked lists. Our findings can be summarized as follows: (i) supervised QPP methods distinctly outperform unsupervised counterparts only when a large-scale training set is available; (ii) point-wise supervised QPP methods outperform their list-wise counterparts in most cases; and (iii) retrieval score-based unsupervised QPP methods show high effectiveness in assessing the conversational dense retrieval method, ConvDR.</p
CLosER: Conversational Legal Longformer with Expertise-Aware Passage Response Ranker for Long Contexts
In this paper, we investigate the task of response ranking in conversational legal search. We propose a novel method for conversational passage response retrieval (ConvPR) for long conversations in domains with mixed levels of expertise. Conversational legal search is challenging because the domain includes long, multi-participant dialogues with domain-specific language. Furthermore, as opposed to other domains, there typically is a large knowledge gap between the questioner (a layperson) and the responders (lawyers), participating in the same conversation. We collect and release a large-scale real-world dataset called LegalConv with nearly one million legal conversations from a legal community question answering (CQA) platform. We address the particular challenges of processing legal conversations, with our novel Conversational Legal Longformer with Expertise-Aware Response Ranker, called CLosER. The proposed method has two main innovations compared to state-of-the-art methods for ConvPR: (i) Expertise-Aware Post-Training; a learning objective that takes into account the knowledge gap difference between participants to the conversation; and (ii) a simple but effective strategy for re-ordering the context utterances in long conversations to overcome the limitations of the sparse attention mechanism of the Longformer architecture. Evaluation on LegalConv shows that our proposed method substantially and significantly outperforms existing state-of-the-art models on the response selection task. Our analysis indicates that our Expertise-Aware Post-Training, i.e., continued pre-training or domain/task adaptation, plays an important role in the achieved effectiveness. Our proposed method is generalizable to other tasks with domain-specific challenges and can facilitate future research on conversational search in other domains.</p
Conversational Machine Comprehension: a Literature Review
Conversational Machine Comprehension (CMC), a research track in
conversational AI, expects the machine to understand an open-domain natural
language text and thereafter engage in a multi-turn conversation to answer
questions related to the text. While most of the research in Machine Reading
Comprehension (MRC) revolves around single-turn question answering (QA),
multi-turn CMC has recently gained prominence, thanks to the advancement in
natural language understanding via neural language models such as BERT and the
introduction of large-scale conversational datasets such as CoQA and QuAC. The
rise in interest has, however, led to a flurry of concurrent publications, each
with a different yet structurally similar modeling approach and an inconsistent
view of the surrounding literature. With the volume of model submissions to
conversational datasets increasing every year, there exists a need to
consolidate the scattered knowledge in this domain to streamline future
research. This literature review attempts at providing a holistic overview of
CMC with an emphasis on the common trends across recently published models,
specifically in their approach to tackling conversational history. The review
synthesizes a generic framework for CMC models while highlighting the
differences in recent approaches and intends to serve as a compendium of CMC
for future researchers.Comment: Accepted to COLING 202
Teaching Text-to-Image Models to Communicate in Dialog
A picture is worth a thousand words, thus, it is crucial for conversational
agents to understand, perceive, and effectively respond with pictures. However,
we find that directly employing conventional image generation techniques is
inadequate for conversational agents to produce image responses effectively. In
this paper, we focus on the innovative dialog-to-image generation task, where
the model synthesizes a high-resolution image aligned with the given dialog
context as a response. To tackle this problem, we design a tailored fine-tuning
approach on the top of state-of-the-art text-to-image generation models to
fully exploit the structural and semantic features in dialog context during
image generation. Concretely, we linearize the dialog context with specific
indicators to maintain the dialog structure, and employ in-domain data to
alleviate the style mismatch between dialog-to-image and conventional image
generation tasks. Empirical results on PhotoChat and MMDialog Corpus show that
our approach brings consistent and remarkable improvement with 3
state-of-the-art pre-trained text-to-image generation backbones.Comment: Work in progres
- …