3 research outputs found
Short Text Pre-training with Extended Token Classification for E-commerce Query Understanding
E-commerce query understanding is the process of inferring the shopping
intent of customers by extracting semantic meaning from their search queries.
The recent progress of pre-trained masked language models (MLM) in natural
language processing is extremely attractive for developing effective query
understanding models. Specifically, MLM learns contextual text embedding via
recovering the masked tokens in the sentences. Such a pre-training process
relies on the sufficient contextual information. It is, however, less effective
for search queries, which are usually short text. When applying masking to
short search queries, most contextual information is lost and the intent of the
search queries may be changed. To mitigate the above issues for MLM
pre-training on search queries, we propose a novel pre-training task
specifically designed for short text, called Extended Token Classification
(ETC). Instead of masking the input text, our approach extends the input by
inserting tokens via a generator network, and trains a discriminator to
identify which tokens are inserted in the extended input. We conduct
experiments in an E-commerce store to demonstrate the effectiveness of ETC
Tähendusepüüdja : pühendusteos professor Haldur Õimu 60. sünnipäevaks 22. jaanuaril 2002 = Catcher of the meaning : festschrift for Professor Haldur Õim on the occasion of his 60th birthday
Kopeerimine ja printimine lubatudhttp://www.ester.ee/record=b1589994*es
Proceedings of the Seventh Congress of the European Society for Research in Mathematics Education
International audienceThis volume contains the Proceedings of the Seventh Congress of the European Society for Research in Mathematics Education (ERME), which took place 9-13 February 2011, at Rzeszñw in Poland