49 research outputs found
Deep Recurrent Generative Decoder for Abstractive Text Summarization
We propose a new framework for abstractive text summarization based on a
sequence-to-sequence oriented encoder-decoder model equipped with a deep
recurrent generative decoder (DRGN).
Latent structure information implied in the target summaries is learned based
on a recurrent latent random model for improving the summarization quality.
Neural variational inference is employed to address the intractable posterior
inference for the recurrent latent variables.
Abstractive summaries are generated based on both the generative latent
variables and the discriminative deterministic states.
Extensive experiments on some benchmark datasets in different languages show
that DRGN achieves improvements over the state-of-the-art methods.Comment: 10 pages, EMNLP 201
InfoDiffusion: Information Entropy Aware Diffusion Process for Non-Autoregressive Text Generation
Diffusion models have garnered considerable interest in the field of text
generation. Several studies have explored text diffusion models with different
structures and applied them to various tasks, including named entity
recognition and summarization. However, there exists a notable disparity
between the "easy-first" text generation process of current diffusion models
and the "keyword-first" natural text generation process of humans, which has
received limited attention. To bridge this gap, we propose InfoDiffusion, a
non-autoregressive text diffusion model. Our approach introduces a
"keyinfo-first" generation strategy and incorporates a noise schedule based on
the amount of text information. In addition, InfoDiffusion combines
self-conditioning with a newly proposed partially noising model structure.
Experimental results show that InfoDiffusion outperforms the baseline model in
terms of generation quality and diversity, as well as exhibiting higher
sampling efficiency.Comment: EMNLP 2023 Finding