The task of Question Generation over Knowledge Bases (KBQG) aims to convert a
logical form into a natural language question. For the sake of expensive cost
of large-scale question annotation, the methods of KBQG under low-resource
scenarios urgently need to be developed. However, current methods heavily rely
on annotated data for fine-tuning, which is not well-suited for few-shot
question generation. The emergence of Large Language Models (LLMs) has shown
their impressive generalization ability in few-shot tasks. Inspired by
Chain-of-Thought (CoT) prompting, which is an in-context learning strategy for
reasoning, we formulate KBQG task as a reasoning problem, where the generation
of a complete question is splitted into a series of sub-question generation.
Our proposed prompting method KQG-CoT first retrieves supportive logical forms
from the unlabeled data pool taking account of the characteristics of the
logical form. Then, we write a prompt to explicit the reasoning chain of
generating complicated questions based on the selected demonstrations. To
further ensure prompt quality, we extend KQG-CoT into KQG-CoT+ via sorting the
logical forms by their complexity. We conduct extensive experiments over three
public KBQG datasets. The results demonstrate that our prompting method
consistently outperforms other prompting baselines on the evaluated datasets.
Remarkably, our KQG-CoT+ method could surpass existing few-shot SoTA results of
the PathQuestions dataset by 18.25, 10.72, and 10.18 absolute points on BLEU-4,
METEOR, and ROUGE-L, respectively.Comment: Accepted by EMNLP 2023 main conferenc