1 research outputs found
Building Language Models for Text with Named Entities
Text in many domains involves a significant amount of named entities.
Predict- ing the entity names is often challenging for a language model as they
appear less frequent on the training corpus. In this paper, we propose a novel
and effective approach to building a discriminative language model which can
learn the entity names by leveraging their entity type information. We also
introduce two benchmark datasets based on recipes and Java programming codes,
on which we evalu- ate the proposed model. Experimental re- sults show that our
model achieves 52.2% better perplexity in recipe generation and 22.06% on code
generation than the state-of-the-art language models