1 research outputs found
Domain-shift Conditioning using Adaptable Filtering via Hierarchical Embeddings for Robust Chinese Spell Check
Spell check is a useful application which processes noisy human-generated
text. Spell check for Chinese poses unresolved problems due to the large number
of characters, the sparse distribution of errors, and the dearth of resources
with sufficient coverage of heterogeneous and shifting error domains. For
Chinese spell check, filtering using confusion sets narrows the search space
and makes finding corrections easier. However, most, if not all, confusion sets
used to date are fixed and thus do not include new, shifting error domains. We
propose a scalable adaptable filter that exploits hierarchical character
embeddings to (1) obviate the need to handcraft confusion sets, and (2) resolve
sparsity problems related to infrequent errors. Our approach compares favorably
with competitive baselines and obtains SOTA results on the 2014 and 2015
Chinese Spelling Check Bake-off datasets