We propose an unsupervised method for sentence summarization using only
language modeling. The approach employs two language models, one that is
generic (i.e. pretrained), and the other that is specific to the target domain.
We show that by using a product-of-experts criteria these are enough for
maintaining continuous contextual matching while maintaining output fluency.
Experiments on both abstractive and extractive sentence summarization data sets
show promising results of our method without being exposed to any paired data