A new unsupervised feature selection method for text clustering based on genetic algorithms

Nowadays a vast amount of textual information is collected and stored in various databases around the world, including the Internet as the largest database of all. This rapidly increasing growth of published text means that even the most avid reader cannot hope to keep up with all the reading in a field and consequently the nuggets of insight or new knowledge are at risk of languishing undiscovered in the literature. Text mining offers a solution to this problem by replacing or supplementing the human reader with automatic systems undeterred by the text explosion. It involves analyzing a large collection of documents to discover previously unknown information. Text clustering is one of the most important areas in text mining, which includes text preprocessing, dimension reduction by selecting some terms (features) and finally clustering using selected terms. Feature selection appears to be the most important step in the process. Conventional unsupervised feature selection methods define a measure of the discriminating power of terms to select proper terms from corpus. However up to now the valuation of terms in groups has not been investigated in reported works. In this paper a new and robust unsupervised feature selection approach is proposed that evaluates terms in groups. In addition a new Modified Term Variance measuring method is proposed for evaluating groups of terms. Furthermore a genetic based algorithm is designed and implemented for finding the most valuable groups of terms based on the new measure. These terms then will be utilized to generate the final feature vector for the clustering process . In order to evaluate and justify our approach the proposed method and also a conventional term variance method are implemented and tested using corpus collection Reuters-21578. For a more accurate comparison, methods have been tested on three corpuses and for each corpus clustering task has been done ten times and results are averaged. Results of comparing these two methods are very promising and show that our method produces better average accuracy and F1-measure than the conventional term variance method

A Taxonomy of Information Retrieval Models and Tools

Information retrieval is attracting significant attention due to the exponential growth of the amount of information available in digital format. The proliferation of information retrieval objects, including algorithms, methods, technologies, and tools, makes it difficult to assess their capabilities and features and to understand the relationships that exist among them. In addition, the terminology is often confusing and misleading, as different terms are used to denote the same, or similar, tasks. This paper proposes a taxonomy of information retrieval models and tools and provides precise definitions for the key terms. The taxonomy consists of superimposing two views: a vertical taxonomy, that classifies IR models with respect to a set of basic features, and a horizontal taxonomy, which classifies IR systems and services with respect to the tasks they support. The aim is to provide a framework for classifying existing information retrieval models and tools and a solid point to assess future developments in the field

Exploring Document Clustering Techniques for Personalized Peer Assessment in Exploratory Courses

Proceedings of: Computer-Supported Peer Review in Education: Synergies with Intelligent Tutoring Systems (CSPRED 2010), Pittsburgh, Pennsylvania USA, June 14th, 2010.Peer review has been proposed as a complement to project-based learning in courses covering a wide and heterogeneous syllabus. By reviewing peers' projects, students can explore other subjects thoroughly apart from their own project topic. This objective relies however in a proper distribution of the works to review, which is a complex and time-consuming task. Beyond simple topic selection, students may report different types of works, which influence their peers' assessment; for example, works focused on a project development approach versus in-depth literature researches. Introducing detailed metadata is time-consuming (thus users are typically reluctant) and, even more important, prone to error. In this paper we explore the potential of text mining and natural language processing technologies for automatic classification of texts, in order to facilitate the adaptation and diversification of the works assigned to the students for review, in the context of a course on Artificial Intelligence.This work was partially funded by Best Practice Network ICOPER (Grant No. ECP-2007-EDU-417007), Learn3 project, “Plan Nacional de I+D+I” TIN2008-05163/TSI, and eMadrid network, S2009/TIC-1650, “Investigación y Desarrollo de tecnologías para el e-learning en la Comunidad de Madrid”.Publicad

A Survey of Website Phishing Detection Techniques

This article surveys the literature on website phishing detection. Web Phishing lures the user to interact with the fake website. The main objective of this attack is to steal the sensitive information from the user. The attacker creates similar website that looks like original website. It allows attacker to obtain sensitive information such as username, password, credit card details etc. This paper aims to survey many of the recently proposed website phishing detection techniques. A high-level overview of various types of phishing detection techniques is also presented

Neurocognitive Informatics Manifesto.

Informatics studies all aspects of the structure of natural and artificial information systems. Theoretical and abstract approaches to information have made great advances, but human information processing is still unmatched in many areas, including information management, representation and understanding. Neurocognitive informatics is a new, emerging field that should help to improve the matching of artificial and natural systems, and inspire better computational algorithms to solve problems that are still beyond the reach of machines. In this position paper examples of neurocognitive inspirations and promising directions in this area are given