A customizable multi-agent system for distributed data mining

We present a general Multi-Agent System framework for distributed data mining based on a Peer-to-Peer model. Agent protocols are implemented through message-based asynchronous communication. The framework adopts a dynamic load balancing policy that is particularly suitable for irregular search algorithms. A modular design allows a separation of the general-purpose system protocols and software components from the specific data mining algorithm. The experimental evaluation has been carried out on a parallel frequent subgraph mining algorithm, which has shown good scalability performances

DIAMOnDS - DIstributed Agents for MObile & Dynamic Services

Distributed Services Architecture with support for mobile agents between services, offer significantly improved communication and computational flexibility. The uses of agents allow execution of complex operations that involve large amounts of data to be processed effectively using distributed resources. The prototype system Distributed Agents for Mobile and Dynamic Services (DIAMOnDS), allows a service to send agents on its behalf, to other services, to perform data manipulation and processing. Agents have been implemented as mobile services that are discovered using the Jini Lookup mechanism and used by other services for task management and communication. Agents provide proxies for interaction with other services as well as specific GUI to monitor and control the agent activity. Thus agents acting on behalf of one service cooperate with other services to carry out a job, providing inter-operation of loosely coupled services in a semi-autonomous way. Remote file system access functionality has been incorporated by the agent framework and allows services to dynamically share and browse the file system resources of hosts, running the services. Generic database access functionality has been implemented in the mobile agent framework that allows performing complex data mining and processing operations efficiently in distributed system. A basic data searching agent is also implemented that performs a query based search in a file system. The testing of the framework was carried out on WAN by moving Connectivity Test agents between AgentStations in CERN, Switzerland and NUST, Pakistan.Comment: 7 pages, 4 figures, CHEP03, La Jolla, California, March 24-28, 200

An investigation into the issues of multi-agent data mining

Multi-agent systems (MAS) often deal with complex applications that require distributedproblem solving. In many applications the individual and collective behaviourof the agents depends on the observed data from distributed sources. The field of DistributedData Mining (DDM) deals with these challenges in analyzing distributed dataand offers many algorithmic solutions to perform different data analysis and miningoperations in a fundamentally distributed manner that pays careful attention to the resourceconstraints. Since multi-agent systems are often distributed and agents haveproactive and reactive features, combining DM with MAS for data intensive applicationsis therefore appealing.This Chapter discusses a number of research issues concerned with the use ofMulti-Agent Systems for Data Mining (MADM), also known as agent-driven datamining. The Chapter also examines the issues affecting the design and implementationof a generic and extendible agent-based data mining framework. An ExtendibleMulti-Agent Data mining System (EMADS) Framework for integrating distributeddata sources is presented. This framework achieves high-availability and highperformance without compromising the data integrity and security. © 2010 Nova Science Publishers, Inc. All rights reserved

An agent-based service oriented architecture for risk mining

University of Technology, Sydney. Faculty of Engineering and Information Technology.Risk Mining (RM) is the process of analyzing data including risk information by data mining methods, with the mining results for risk prevention. In the last few years, some researchers have proposed the combination of data mining and agent technology (agent mining) to improve the performance of data mining methodology in the heterogeneous business environments. However, problems exist for further research with the application of risk mining systems in real industry environments to enhance the robustness of system architect, dynamic business process and model accuracy etc. Therefore, in this thesis we present an Agent-based Service-oriented Risk Mining Architecture (ABSORM), which has been designed to facilitate the development of agent mining systems to address the above issues. This thesis focuses on developing the following strategies: • The integration of agent technology with web service. In this framework, we propose a new and easier method, by which the system functions are not integrated into the structure of the agents, rather modeled as distributed services and applications which are invoked by the agents acting as controllers and coordinators. Therefore, techniques developed in this framework can improve the interoperability between different modules, distribution of resources, and the lack of dependency of programming languages. • The integration of agent technology with business process management. In this work, we develop the autonomous agents that can collaborate in a business flow, which not only increases the reusability of the system, but also eases the system development in terms of re-usability of the computational resources. A group of agents solves problems in the following way: each individual agent solves the problem individually, and then interacts with each other to finalize a business process. • The integration of agent technology with ensemble learning methods. In this thesis, we are interested in developing agent-based ensemble learning strategies for risk mining: each ensemble agent individually gathers the evidence about model evaluation, and then ensembles learning methods like bagging and boosting is used to obtain prediction from the individually gathered evidence. Agent based ensemble learning can provide a critical boost to risk mining where predictive accuracy is more vital than model interpretability. The proposed architecture has been evaluated for building an online banking fraud detection system and a student risk management system. These two applications have been proved to be a sophisticated, yet user friendly, risk analysis and management tool. They are modular, interactive, dynamic and globally oriented

Distributed Data Mining: The JAM system architecture

This paper describes the system architecture of JAM (Java Agents for Meta-learning), a distributed data mining system that scales up to large and physically separated data sets. An earlyversion of the JAM system was described in Stolfo-et-al-97-KDD-JAM. Since then, JAM has evolved both architecturally and functionally and here we present the final design and implementation details of this system architecture. JAM is an extensible agent-based distributed data mining system that supports the remote dispatch and exchange of agents among participating datasites and employs meta-learning techniques to combine the multiple models that are learned. One of JAM's target applications is fraud and intrusion detection in financial information systems. A brief description of this learning task and JAM's applicability and summary results are also discussed

Agent-based learning classifier systems for grid data mining

Grid Data Mining tools must be able to cope with very large, high dimensional and, frequently heterogeneous data sets that are geographically distributed and stored in different types of repositories, produced from different devices and retrieved through different protocols. This paper presents an agent-based version of a Learning Classifier System. An experimental study was conducted in a computer network in order to determine the systems’ efficiency. The results showed that the model is suitable to be applied in inherently distributed problems and is scalable, i.e., when the latency communication times are not considerable, the system obtains an interesting speedup

Enhancing association rules algorithms for mining distributed databases. Integration of fast BitTable and multi-agent association rules mining in distributed medical databases for decision support.

Over the past few years, mining data located in heterogeneous and geographically distributed sites have been designated as one of the key important issues. Loading distributed data into centralized location for mining interesting rules is not a good approach. This is because it violates common issues such as data privacy and it imposes network overheads. The situation becomes worse when the network has limited bandwidth which is the case in most of the real time systems. This has prompted the need for intelligent data analysis to discover the hidden information in these huge amounts of distributed databases. In this research, we present an incremental approach for building an efficient Multi-Agent based algorithm for mining real world databases in geographically distributed sites. First, we propose the Distributed Multi-Agent Association Rules algorithm (DMAAR) to minimize the all-to-all broadcasting between distributed sites. Analytical calculations show that DMAAR reduces the algorithm complexity and minimizes the message communication cost. The proposed Multi-Agent based algorithm complies with the Foundation for Intelligent Physical Agents (FIPA), which is considered as the global standards in communication between agents, thus, enabling the proposed algorithm agents to cooperate with other standard agents. Second, the BitTable Multi-Agent Association Rules algorithm (BMAAR) is proposed. BMAAR includes an efficient BitTable data structure which helps in compressing the database thus can easily fit into the memory of the local sites. It also includes two BitWise AND/OR operations for quick candidate itemsets generation and support counting. Moreover, the algorithm includes three transaction trimming techniques to reduce the size of the mined data. Third, we propose the Pruning Multi-Agent Association Rules algorithm (PMAAR) which includes three candidate itemsets pruning techniques for reducing the large number of generated candidate itemsets, consequently, reducing the total time for the mining process. The proposed PMAAR algorithm has been compared with existing Association Rules algorithms against different benchmark datasets and has proved to have better performance and execution time. Moreover, PMAAR has been implemented on real world distributed medical databases obtained from more than one hospital in Egypt to discover the hidden Association Rules in patients¿ records to demonstrate the merits and capabilities of the proposed model further. Medical data was anonymously obtained without the patients¿ personal details. The analysis helped to identify the existence or the absence of the disease based on minimum number of effective examinations and tests. Thus, the proposed algorithm can help in providing accurate medical decisions based on cost effective treatments, improving the medical service for the patients, reducing the real time response for the health system and improving the quality of clinical decision making

SenMinCom: Pervasive Distributed Dynamic Sensor Data Mining for Effective Commerce

In last few years, the use of wireless sensor networks and cell phones has become ubiquitous; fusing these technologies in the field of business will open up new possibilities. To fill this lacuna, I propose a novel idea where the combination of these will facilitate companies to receive feedback on their products and services. System\u27s unobtrusive sensors will not only collect shopping, mobile usage data from consumers but will also make effective use of this information to increase revenue, cut costs, etc.; the use of mobile agent based data mining allows analyzing the data from different dimensions, categorizing it on factors such as product positioning, promotion of goods, etc. as in the case of a shopping store. Additionally, because of the dynamic mining system the companies get on-the-scene recommendation of products rather than off-the-scene. In this thesis, a novel distributed pervasive mining system is proposed to get dynamic shopping information and mobile device usage of the customers