1 research outputs found
Interpretable Multi-Headed Attention for Abstractive Summarization at Controllable Lengths
Abstractive summarization at controllable lengths is a challenging task in
natural language processing. It is even more challenging for domains where
limited training data is available or scenarios in which the length of the
summary is not known beforehand. At the same time, when it comes to trusting
machine-generated summaries, explaining how a summary was constructed in
human-understandable terms may be critical. We propose Multi-level Summarizer
(MLS), a supervised method to construct abstractive summaries of a text
document at controllable lengths. The key enabler of our method is an
interpretable multi-headed attention mechanism that computes attention
distribution over an input document using an array of timestep independent
semantic kernels. Each kernel optimizes a human-interpretable syntactic or
semantic property. Exhaustive experiments on two low-resource datasets in the
English language show that MLS outperforms strong baselines by up to 14.70% in
the METEOR score. Human evaluation of the summaries also suggests that they
capture the key concepts of the document at various length-budgets.Comment: 9 pages, 5 figure