A new heuristic for broadcasting in clusters of clusters

International audienceThis paper deals with the problem of broadcasting for clus- ter of clusters. The construction of partial minimum spanning trees being NP-complete, several heuristic algorithms have been already formulated. Many of these heuristics (like the heuristic of Kruskal) use the shortest path to connect the components of the tree. They are not relevant in case of forwarding or overlapping communication during a step of the algorithm. In this paper, we study a new heuristic for the minimum broadcasting tree and we evaluate it through simulations with different communication parameters and also through real experimentation over the Grid'5000 testbed

A Framework for Adaptive Collective Communications on Heterogeneous Hierarchical Networks

Extended version of the IPDPS 2006 paperToday, due to the wide variety of existing parallel systems consisting on collections of heterogeneous machines, it is very difficult for a user to solve a target problem by using a single algorithm or to write portable programs that perform well on multiple computational supports. The inherent heterogeneity and the diversity of networks of such environments represent a great challenge to model the communications for high performance computing applications. Our objective within this work is to propose a generic framework based on communication models and adaptive techniques for dealing with prediction of communication performances on cluster-based hierarchical platforms. Toward this goal, we introduce the concept of polyalgorithmic model of communications, which correspond to selection of the most adapted communication algorithms and scheduling strategies, giving the characteristics of the hardware resources of the target parallel system. We apply this methodology on collective communication operations and show that the framework provides significant performances while determining the best algorithm depending on the problem and architecture parameters

Decentralized Greedy-Based Algorithm for Smart Energy Management in Plug-in Electric Vehicle Energy Distribution Systems

Variations in electricity tariffs arising due to stochastic demand loads on the power grids have stimulated research in finding optimal charging/discharging scheduling solutions for electric vehicles (EVs). Most of the current EV scheduling solutions are either centralized, which suffer from low reliability and high complexity, while existing decentralized solutions do not facilitate the efficient scheduling of on-move EVs in large-scale networks considering a smart energy distribution system. Motivated by smart cities applications, we consider in this paper the optimal scheduling of EVs in a geographically large-scale smart energy distribution system where EVs have the flexibility of charging/discharging at spatially-deployed smart charging stations (CSs) operated by individual aggregators. In such a scenario, we define the social welfare maximization problem as the total profit of both supply and demand sides in the form of a mixed integer non-linear programming (MINLP) model. Due to the intractability, we then propose an online decentralized algorithm with low complexity which utilizes effective heuristics to forward each EV to the most profitable CS in a smart manner. Results of simulations on the IEEE 37 bus distribution network verify that the proposed algorithm improves the social welfare by about 30% on average with respect to an alternative scheduling strategy under the equal participation of EVs in charging and discharging operations. Considering the best-case performance where only EV profit maximization is concerned, our solution also achieves upto 20% improvement in flatting the final electricity load. Furthermore, the results reveal the existence of an optimal number of CSs and an optimal vehicle-to-grid penetration threshold for which the overall profit can be maximized. Our findings serve as guidelines for V2G system designers in smart city scenarios to plan a cost-effective strategy for large-scale EVs distributed energy management

A First Step Towards Automatically Building Network Representations

To fully harness Grids, users or middlewares must have some knowledge on the topology of the platform interconnection network. As such knowledge is usually not available, one must uses tools which automatically build a topological network model through some measurements. In this article, we define a methodology to assess the quality of these network model building tools, and we apply this methodology to representatives of the main classes of model builders and to two new algorithms. We show that none of the main existing techniques build models that enable to accurately predict the running time of simple application kernels for actual platforms. However some of the new algorithms we propose give excellent results in a wide range of situations

A Taxonomy of Data Grids for Distributed Data Sharing, Management and Processing

Data Grids have been adopted as the platform for scientific communities that need to share, access, transport, process and manage large data collections distributed worldwide. They combine high-end computing technologies with high-performance networking and wide-area storage management techniques. In this paper, we discuss the key concepts behind Data Grids and compare them with other data sharing and distribution paradigms such as content delivery networks, peer-to-peer networks and distributed databases. We then provide comprehensive taxonomies that cover various aspects of architecture, data transportation, data replication and resource allocation and scheduling. Finally, we map the proposed taxonomy to various Data Grid systems not only to validate the taxonomy but also to identify areas for future exploration. Through this taxonomy, we aim to categorise existing systems to better understand their goals and their methodology. This would help evaluate their applicability for solving similar problems. This taxonomy also provides a "gap analysis" of this area through which researchers can potentially identify new issues for investigation. Finally, we hope that the proposed taxonomy and mapping also helps to provide an easy way for new practitioners to understand this complex area of research.Comment: 46 pages, 16 figures, Technical Repor

A distributed platform for the volunteer execution of workflows on a local area network

Thesis submitted in fulfilment of the requirements for the Degree of Master of Science in Computer ScienceAlbatroz Engineering has developed a framework for over-head power lines inspection data acquisition and analysis, which includes hardware and software. The framework’s software components include inspection data analysis and reporting tools, commonly known as PLMI2 application/platform. In PLMI2, the analysis of over-head power line maintenance inspection data consists of a sequence of Automatic Tasks (ATs) interleaved with Manual Tasks (MTs). An AT consists of a set of algorithms that receives as input one or more datasets, processes them and returns new datasets. In turn, an MT enables human supervisors (also known as lines inspection operators) to correct, improve and validate the results of ATs. ATs run faster than MTs and in the overall work cycle, ATs take less than 10% of total processing time, but still take a few minutes. There is data flow dependency among tasks, which can be modelled with a workflow and even if MTs are omitted from this workflow, it is possible to carry the sequence of ATs, postponing MTs. In fact, if the computing cost and waiting time are negligible, it may be advantageous to run ATs earlier in the workflow, prior to validation. To address this opportunity, Albatroz Engineering has invested in a new procedure to stream the data through all ATs fully unattended. Considering these scenarios, it could be useful to have a system capable of detecting available workstations at a given instant and subsequently distribute the ATs to them. In this way, operators could schedule the execution of future ATs for a given inspection data, while they are performing MTs of another. The requirements of the system to implement fall within the field Volunteer Computing Systems and we will address some of the challenges posed by these kinds of systems, namely the hosts volatility and failures. Volunteer Computing is a type of distributed computing which exploits idle CPU cycles from computing resources donated by volunteers and connected through the Internet/Intranet to compute large-scale simulations. This thesis proposes and designs a new distributed task scheduling system in the context of Volunteer Computing Systems, able to schedule the ATs of PLMI2 and exploit idle CPU cycles from workstations within the company’s local area network (LAN) to accelerate the data analysis, being aware of data flow interdependencies. To evaluate the proposed system, a prototype has been implemented, and the simulations results have shown that it is scalable and supports fault-tolerance of tasks execution, by employing the rescheduling mechanism

Design and Evaluation of a Collective IO Model for Loosely Coupled Petascale Programming

Loosely coupled programming is a powerful paradigm for rapidly creating higher-level applications from scientific programs on petascale systems, typically using scripting languages. This paradigm is a form of many-task computing (MTC) which focuses on the passing of data between programs as ordinary files rather than messages. While it has the significant benefits of decoupling producer and consumer and allowing existing application programs to be executed in parallel with no recoding, its typical implementation using shared file systems places a high performance burden on the overall system and on the user who will analyze and consume the downstream data. Previous efforts have achieved great speedups with loosely coupled programs, but have done so with careful manual tuning of all shared file system access. In this work, we evaluate a prototype collective IO model for file-based MTC. The model enables efficient and easy distribution of input data files to computing nodes and gathering of output results from them. It eliminates the need for such manual tuning and makes the programming of large-scale clusters using a loosely coupled model easier. Our approach, inspired by in-memory approaches to collective operations for parallel programming, builds on fast local file systems to provide high-speed local file caches for parallel scripts, uses a broadcast approach to handle distribution of common input data, and uses efficient scatter/gather and caching techniques for input and output. We describe the design of the prototype model, its implementation on the Blue Gene/P supercomputer, and present preliminary measurements of its performance on synthetic benchmarks and on a large-scale molecular dynamics application.Comment: IEEE Many-Task Computing on Grids and Supercomputers (MTAGS08) 200

Optimisation problems and resolution methods in satellite scheduling and space-craft operation: a survey

The fast development in the production of small, low-cost satellites is propelling an important increase in satellite mission planning and operations projects. Central to satellite mission planning is the resolution of scheduling problem for an optimised allocation of user requests for efficient communication between operations teams at the ground and spacecraft systems. The aim of this paper is to survey the state of the art in the satellite scheduling problem, analyse its mathematical formulations, examine its multi-objective nature and resolution through meta-heuristics methods. Finally, we consider some optimisation problems arising in spacecraft design, operation and satellite deployment systemsPeer ReviewedPostprint (author's final draft