Text summarization models are approaching human levels of fidelity. Existing
benchmarking corpora provide concordant pairs of full and abridged versions of
Web, news or, professional content. To date, all summarization datasets operate
under a one-size-fits-all paradigm that may not reflect the full range of
organic summarization needs. Several recently proposed models (e.g., plug and
play language models) have the capacity to condition the generated summaries on
a desired range of themes. These capacities remain largely unused and
unevaluated as there is no dedicated dataset that would support the task of
topic-focused summarization.
This paper introduces the first topical summarization corpus NEWTS, based on
the well-known CNN/Dailymail dataset, and annotated via online crowd-sourcing.
Each source article is paired with two reference summaries, each focusing on a
different theme of the source document. We evaluate a representative range of
existing techniques and analyze the effectiveness of different prompting
methods