Distributed data mining in grid computing environments

The official published version of this article can be found at the link below.The computing-intensive data mining for inherently Internet-wide distributed data, referred to as Distributed Data Mining (DDM), calls for the support of a powerful Grid with an effective scheduling framework. DDM often shares the computing paradigm of local processing and global synthesizing. It involves every phase of Data Mining (DM) processes, which makes the workflow of DDM very complex and can be modelled only by a Directed Acyclic Graph (DAG) with multiple data entries. Motivated by the need for a practical solution of the Grid scheduling problem for the DDM workflow, this paper proposes a novel two-phase scheduling framework, including External Scheduling and Internal Scheduling, on a two-level Grid architecture (InterGrid, IntraGrid). Currently a DM IntraGrid, named DMGCE (Data Mining Grid Computing Environment), has been developed with a dynamic scheduling framework for competitive DAGs in a heterogeneous computing environment. This system is implemented in an established Multi-Agent System (MAS) environment, in which the reuse of existing DM algorithms is achieved by encapsulating them into agents. Practical classification problems from oil well logging analysis are used to measure the system performance. The detailed experiment procedure and result analysis are also discussed in this paper

A Novel Workload Allocation Strategy for Batch Jobs

The distribution of computational tasks across a diverse set of geographically distributed heterogeneous resources is a critical issue in the realisation of true computational grids. Conventionally, workload allocation algorithms are divided into static and dynamic approaches. Whilst dynamic approaches frequently outperform static schemes, they usually require the collection and processing of detailed system information at frequent intervals - a task that can be both time consuming and unreliable in the real-world. This paper introduces a novel workload allocation algorithm for optimally distributing the workload produced by the arrival of batches of jobs. Results show that, for the arrival of batches of jobs, this workload allocation algorithm outperforms other commonly used algorithms in the static case. A hybrid scheduling approach (using this workload allocation algorithm), where information about the speed of computational resources is inferred from previously completed jobs, is then introduced and the efficiency of this approach demonstrated using a real world computational grid. These results are compared to the same workload allocation algorithm used in the static case and it can be seen that this hybrid approach comprehensively outperforms the static approach

Decentralized Greedy-Based Algorithm for Smart Energy Management in Plug-in Electric Vehicle Energy Distribution Systems

Variations in electricity tariffs arising due to stochastic demand loads on the power grids have stimulated research in finding optimal charging/discharging scheduling solutions for electric vehicles (EVs). Most of the current EV scheduling solutions are either centralized, which suffer from low reliability and high complexity, while existing decentralized solutions do not facilitate the efficient scheduling of on-move EVs in large-scale networks considering a smart energy distribution system. Motivated by smart cities applications, we consider in this paper the optimal scheduling of EVs in a geographically large-scale smart energy distribution system where EVs have the flexibility of charging/discharging at spatially-deployed smart charging stations (CSs) operated by individual aggregators. In such a scenario, we define the social welfare maximization problem as the total profit of both supply and demand sides in the form of a mixed integer non-linear programming (MINLP) model. Due to the intractability, we then propose an online decentralized algorithm with low complexity which utilizes effective heuristics to forward each EV to the most profitable CS in a smart manner. Results of simulations on the IEEE 37 bus distribution network verify that the proposed algorithm improves the social welfare by about 30% on average with respect to an alternative scheduling strategy under the equal participation of EVs in charging and discharging operations. Considering the best-case performance where only EV profit maximization is concerned, our solution also achieves upto 20% improvement in flatting the final electricity load. Furthermore, the results reveal the existence of an optimal number of CSs and an optimal vehicle-to-grid penetration threshold for which the overall profit can be maximized. Our findings serve as guidelines for V2G system designers in smart city scenarios to plan a cost-effective strategy for large-scale EVs distributed energy management