70,892 research outputs found
Design and Evaluation of Approaches for Automatic Chinese Text Categorization
[[abstract]]In this paper, we propose and evaluate approaches to categorizing Chinese
texts, which consist of term extraction, term selection, term clustering and text
classification. We propose a scalable approach which uses frequency counts to
identify left and right boundaries of possibly significant terms. We used the
combination of term selection and term clustering to reduce the dimension of the
vector space to a practical level. While the huge number of possible Chinese terms
makes most of the machine learning algorithms impractical, results obtained in an
experiment on a CAN news collection show that the dimension could be
dramatically reduced to 1200 while approximately the same level of classification
accuracy was maintained using our approach. We also studied and compared the
performance of three well known classifiers, the Rocchio linear classifier, naive
Bayes probabilistic classifier and k-nearest neighbors(kNN) classifier, when they
were applied to categorize Chinese texts. Overall, kNN achieved the best accuracy,
about 78.3%, but required large amounts of computation time and memory when
used to classify new texts. Rocchio was very time and memory efficient, and
achieved a high level of accuracy, about 75.4%. In practical implementation,
Rocchio may be a good choice
A Benchmark for Text Expansion: Datasets, Metrics, and Baselines
This work presents a new task of Text Expansion (TE), which aims to insert
fine-grained modifiers into proper locations of the plain text to concretize or
vivify human writings. Different from existing insertion-based writing
assistance tasks, TE requires the model to be more flexible in both locating
and generation, and also more cautious in keeping basic semantics. We leverage
four complementary approaches to construct a dataset with 12 million
automatically generated instances and 2K human-annotated references for both
English and Chinese. To facilitate automatic evaluation, we design various
metrics from multiple perspectives. In particular, we propose Info-Gain to
effectively measure the informativeness of expansions, which is an important
quality dimension in TE. On top of a pre-trained text-infilling model, we build
both pipelined and joint Locate&Infill models, which demonstrate the
superiority over the Text2Text baselines, especially in expansion
informativeness. Experiments verify the feasibility of the TE task and point
out potential directions for future research toward better automatic text
expansion
Natural language processing
Beginning with the basic issues of NLP, this chapter aims to chart the major research activities in this area since the last ARIST Chapter in 1996 (Haas, 1996), including: (i) natural language text processing systems - text summarization, information extraction, information retrieval, etc., including domain-specific applications; (ii) natural language interfaces; (iii) NLP in the context of www and digital libraries ; and (iv) evaluation of NLP systems
Spoken content retrieval: A survey of techniques and technologies
Speech media, that is, digital audio and video containing spoken content, has blossomed in recent years. Large collections are accruing on the Internet as well as in private and enterprise settings. This growth has motivated extensive research on techniques and technologies that facilitate reliable indexing and retrieval. Spoken content retrieval (SCR) requires the combination of audio and speech processing technologies with methods from information retrieval (IR). SCR research initially investigated planned speech structured in document-like units, but has subsequently shifted focus to more informal spoken content produced spontaneously, outside of the studio and in conversational settings. This survey provides an overview of the field of SCR encompassing component technologies, the relationship of SCR to text IR and automatic speech recognition and user interaction issues. It is aimed at researchers with backgrounds in speech technology or IR who are seeking deeper insight on how these fields are integrated to support research and development, thus addressing the core challenges of SCR
- âŠ