24 research outputs found
Learning from Easy to Complex: Adaptive Multi-curricula Learning for Neural Dialogue Generation
Current state-of-the-art neural dialogue systems are mainly data-driven and
are trained on human-generated responses. However, due to the subjectivity and
open-ended nature of human conversations, the complexity of training dialogues
varies greatly. The noise and uneven complexity of query-response pairs impede
the learning efficiency and effects of the neural dialogue generation models.
What is more, so far, there are no unified dialogue complexity measurements,
and the dialogue complexity embodies multiple aspects of
attributes---specificity, repetitiveness, relevance, etc. Inspired by human
behaviors of learning to converse, where children learn from easy dialogues to
complex ones and dynamically adjust their learning progress, in this paper, we
first analyze five dialogue attributes to measure the dialogue complexity in
multiple perspectives on three publicly available corpora. Then, we propose an
adaptive multi-curricula learning framework to schedule a committee of the
organized curricula. The framework is established upon the reinforcement
learning paradigm, which automatically chooses different curricula at the
evolving learning process according to the learning status of the neural
dialogue generation model. Extensive experiments conducted on five
state-of-the-art models demonstrate its learning efficiency and effectiveness
with respect to 13 automatic evaluation metrics and human judgments.Comment: Accepted to AAAI 202
Evaluating and Enhancing Large Language Models for Conversational Reasoning on Knowledge Graphs
The development of large language models (LLMs) has been catalyzed by
advancements in pre-training techniques. These models have demonstrated robust
reasoning capabilities through manually designed prompts. In this work, we
evaluate the conversational reasoning capabilities of the current
state-of-the-art LLM (GPT-4) on knowledge graphs (KGs). However, the
performance of LLMs is constrained due to a lack of KG environment awareness
and the difficulties in developing effective optimization mechanisms for
intermediary reasoning stages. We further introduce LLM-ARK, a LLM grounded KG
reasoning agent designed to deliver precise and adaptable predictions on KG
paths. LLM-ARK leverages Full Textual Environment (FTE) prompt to assimilate
state information within each reasoning step. We reframe the challenge of
multi-hop reasoning on the KG as a sequential decision-making task. Utilizing
the Proximal Policy Optimization (PPO) online policy gradient reinforcement
learning algorithm, our model is optimized to learn from rich reward signals.
Additionally, we conduct an evaluation of our model and GPT-4 on the OpenDialKG
dataset. The experimental results reveal that LLaMA-2-7B-ARK outperforms the
current state-of-the-art model by 5.28 percentage points, with a performance
rate of 36.39% on the target@1 evaluation metric. Meanwhile, GPT-4 scored
14.91%, further demonstrating the effectiveness of our method. Our code is
available on GitHub (https://github.com/Aipura/LLM-ARK) for further access