Contextualized embeddings are proven to be powerful tools in multiple NLP
tasks. Nonetheless, challenges regarding their interpretability and capability
to represent lexical semantics still remain. In this paper, we propose that the
task of definition modeling, which aims to generate the human-readable
definition of the word, provides a route to evaluate or understand the high
dimensional semantic vectors. We propose a `Vec2Gloss' model, which produces
the gloss from the target word's contextualized embeddings. The generated
glosses of this study are made possible by the systematic gloss patterns
provided by Chinese Wordnet. We devise two dependency indices to measure the
semantic and contextual dependency, which are used to analyze the generated
texts in gloss and token levels. Our results indicate that the proposed
`Vec2Gloss' model opens a new perspective to the lexical-semantic applications
of contextualized embeddings