Natural language processing

Beginning with the basic issues of NLP, this chapter aims to chart the major research activities in this area since the last ARIST Chapter in 1996 (Haas, 1996), including: (i) natural language text processing systems - text summarization, information extraction, information retrieval, etc., including domain-specific applications; (ii) natural language interfaces; (iii) NLP in the context of www and digital libraries ; and (iv) evaluation of NLP systems

Knowledge Discovery Model in Chinese Industrial News

With prevalence of Internet, users can easily retrieve the information what they want from Internet. Information explosion shows that efficient information summarization is aspired to all users. Therefore, an efficient knowledge management methodology becomes very important. Some technologies, such as text mining, for acquiring knowledge from huge amount of electronic documents are recognized as important technology in this field. This work focuses on text-mining applications on Chinese industrial news and knowledge discovery. We use information extract method to extract news into companies, event keyword, time, location, and person categories based on the characteristics of news. The set of five extracted categories is called information template. The templates are summarized by rule induction. We can discover unexpected knowledge from these summarized rules. We built an integrated industrial news text-mining model by using induction rule learner. This model is suitable to manipulate rules in bag-of-word form. Furthermore, we proposed interestingness to measure interesting strength of rules. The users can analyze the discovered rules based this measure. These are helpful to discover unexpected knowledge. It is meaningful to commercial activities if we can discover valuable rules. Besides industrial news application, we believe this model is suitable for knowledge discovery application in other fields

A Latent Dirichlet Allocation and Fuzzy Clustering Based Machine Learning Model for Text Thesaurus

It is not quite possible to use manual methods to process the huge amount of structured and semi-structured data. This study aims to solve the problem of processing huge data through machine learning algorithms. We collected the text data of the company’s public opinion through crawlers, and use Latent Dirichlet Allocation (LDA) algorithm to extract the keywords of the text, and uses fuzzy clustering to cluster the keywords to form different topics. The topic keywords will be used as a seed dictionary for new word discovery. In order to verify the efficiency of machine learning in new word discovery, algorithms based on association rules, N-Gram, PMI, andWord2vec were used for comparative testing of new word discovery. The experimental results show that the Word2vec algorithm based on machine learning model has the highest accuracy, recall and F-value indicators