4,227 research outputs found
Survey on Evaluation Methods for Dialogue Systems
In this paper we survey the methods and concepts developed for the evaluation
of dialogue systems. Evaluation is a crucial part during the development
process. Often, dialogue systems are evaluated by means of human evaluations
and questionnaires. However, this tends to be very cost and time intensive.
Thus, much work has been put into finding methods, which allow to reduce the
involvement of human labour. In this survey, we present the main concepts and
methods. For this, we differentiate between the various classes of dialogue
systems (task-oriented dialogue systems, conversational dialogue systems, and
question-answering dialogue systems). We cover each class by introducing the
main technologies developed for the dialogue systems and then by presenting the
evaluation methods regarding this class
A Review of Reinforcement Learning for Natural Language Processing, and Applications in Healthcare
Reinforcement learning (RL) has emerged as a powerful approach for tackling
complex medical decision-making problems such as treatment planning,
personalized medicine, and optimizing the scheduling of surgeries and
appointments. It has gained significant attention in the field of Natural
Language Processing (NLP) due to its ability to learn optimal strategies for
tasks such as dialogue systems, machine translation, and question-answering.
This paper presents a review of the RL techniques in NLP, highlighting key
advancements, challenges, and applications in healthcare. The review begins by
visualizing a roadmap of machine learning and its applications in healthcare.
And then it explores the integration of RL with NLP tasks. We examined dialogue
systems where RL enables the learning of conversational strategies, RL-based
machine translation models, question-answering systems, text summarization, and
information extraction. Additionally, ethical considerations and biases in
RL-NLP systems are addressed
Human-technology integration with industrial conversational agents: A conceptual architecture and a taxonomy for manufacturing
Conversational agents are systems with great potential to enhance human-computer interaction in industrial settings. Although the number of applications of conversational agents in many fields is growing, there is no shared view of the elements to design and implement for chatbots in the industrial field. The paper presents the combination of many research contributions into an integrated conceptual architecture, for developing industrial conversational agents using Nickerson's methodology. The conceptual architecture consists of five core modules; every module consists of specific elements and approaches. Furthermore, the paper defines a taxonomy from the study of empirical applications of manufacturing conversational agents. Indeed, some applications of chatbots in manufacturing are available but those have never been collected in single research. The paper fills this gap by analyzing the empirical cases and presenting a qualitative analysis, with verification of the proposed taxonomy. The contribution of the article is mainly to illustrate the elements needed for the development of a conversational agent in manufacturing: researchers and practitioners can use the proposed conceptual architecture and taxonomy to more easily investigate, define, and develop all the elements for chatbot implementation
Review of Research on Speech Technology: Main Contributions From Spanish Research Groups
In the last two decades, there has been an important increase in research on speech technology in Spain, mainly due to a higher level of funding from European, Spanish and local institutions and also due to a growing interest in these technologies for developing new services and applications. This paper provides a review of the main areas of speech technology addressed by research groups in Spain, their main contributions in the recent years and the main focus of interest these days. This description is classified in five main areas: audio processing including speech, speaker characterization, speech and language processing, text to speech conversion and spoken language applications. This paper also introduces the Spanish Network of Speech Technologies (RTTH. Red Temática en Tecnologías del Habla) as the research network that includes almost all the researchers working in this area, presenting some figures, its objectives and its main activities developed in the last years
Lifelong learning and task-oriented dialogue system: what does it mean?
International audienceThe main objective of this paper is to propose a functional definition of lifelong learning system adapted to the framework of task-oriented system. We mainly identified two aspects where a lifelong learning technology could be applied in such system: improve the natural language understanding module and enrich the database used by the system. Given our definition, we present an example of how it could be implemented in an actual task-oriented dialogue system that is developed in the LIHLITH project
Survey on evaluation methods for dialogue
In this paper we survey the methods and concepts developed for the evaluation of dialogue systems. Evaluation is a crucial part during the development process. Often, dialogue systems are evaluated by means of human evaluations and questionnaires. However, this tends to be very cost and time intensive. Thus, much work has been put into finding methods, which allow to reduce the involvement of human labour. In this survey, we present the main concepts and methods. For this, we differentiate between the various classes of dialogue systems (task-oriented dialogue systems, conversational dialogue systems, and question-answering dialogue systems). We cover each class by introducing the main technologies developed for the dialogue systems and then by presenting the evaluation methods regarding this class
Selecting and Generating Computational Meaning Representations for Short Texts
Language conveys meaning, so natural language processing (NLP) requires representations of meaning. This work addresses two broad questions: (1) What meaning representation should we use? and (2) How can we transform text to our chosen meaning representation? In the first part, we explore different meaning representations (MRs) of short texts, ranging from surface forms to deep-learning-based models. We show the advantages and disadvantages of a variety of MRs for summarization, paraphrase detection, and clustering. In the second part, we use SQL as a running example for an in-depth look at how we can parse text into our chosen MR. We examine the text-to-SQL problem from three perspectives—methodology, systems, and applications—and show how each contributes to a fuller understanding of the task.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/143967/1/cfdollak_1.pd
Dialogue Agents 101: A Beginner's Guide to Critical Ingredients for Designing Effective Conversational Systems
Sharing ideas through communication with peers is the primary mode of human
interaction. Consequently, extensive research has been conducted in the area of
conversational AI, leading to an increase in the availability and diversity of
conversational tasks, datasets, and methods. However, with numerous tasks being
explored simultaneously, the current landscape of conversational AI becomes
fragmented. Therefore, initiating a well-thought-out model for a dialogue agent
can pose significant challenges for a practitioner. Towards highlighting the
critical ingredients needed for a practitioner to design a dialogue agent from
scratch, the current study provides a comprehensive overview of the primary
characteristics of a dialogue agent, the supporting tasks, their corresponding
open-domain datasets, and the methods used to benchmark these datasets. We
observe that different methods have been used to tackle distinct dialogue
tasks. However, building separate models for each task is costly and does not
leverage the correlation among the several tasks of a dialogue agent. As a
result, recent trends suggest a shift towards building unified foundation
models. To this end, we propose UNIT, a UNified dIalogue dataseT constructed
from conversations of existing datasets for different dialogue tasks capturing
the nuances for each of them. We also examine the evaluation strategies used to
measure the performance of dialogue agents and highlight the scope for future
research in the area of conversational AI.Comment: 21 pages, 3 figures, 3 table
- …