1 research outputs found
Explainable and Discourse Topic-aware Neural Language Understanding
Marrying topic models and language models exposes language understanding to a
broader source of document-level context beyond sentences via topics. While
introducing topical semantics in language models, existing approaches
incorporate latent document topic proportions and ignore topical discourse in
sentences of the document. This work extends the line of research by
additionally introducing an explainable topic representation in language
understanding, obtained from a set of key terms correspondingly for each latent
topic of the proportion. Moreover, we retain sentence-topic associations along
with document-topic association by modeling topical discourse for every
sentence in the document. We present a novel neural composite language model
that exploits both the latent and explainable topics along with topical
discourse at sentence-level in a joint learning framework of topic and language
models. Experiments over a range of tasks such as language modeling, word sense
disambiguation, document classification, retrieval and text generation
demonstrate ability of the proposed model in improving language understanding.Comment: Accepted at ICML2020 (13 pages, 2 figures), acknowledgements adde