A classification of emerging and traditional grid systems

The grid has evolved in numerous distinct phases. It started in the early ’90s as a model of metacomputing in which supercomputers share resources; subsequently, researchers added the ability to share data. This is usually referred to as the first-generation grid. By the late ’90s, researchers had outlined the framework for second-generation grids, characterized by their use of grid middleware systems to “glue” different grid technologies together. Third-generation grids originated in the early millennium when Web technology was combined with second-generation grids. As a result, the invisible grid, in which grid complexity is fully hidden through resource virtualization, started receiving attention. Subsequently, grid researchers identified the requirement for semantically rich knowledge grids, in which middleware technologies are more intelligent and autonomic. Recently, the necessity for grids to support and extend the ambient intelligence vision has emerged. In AmI, humans are surrounded by computing technologies that are unobtrusively embedded in their surroundings. However, third-generation grids’ current architecture doesn’t meet the requirements of next-generation grids (NGG) and service-oriented knowledge utility (SOKU).4 A few years ago, a group of independent experts, arranged by the European Commission, identified these shortcomings as a way to identify potential European grid research priorities for 2010 and beyond. The experts envision grid systems’ information, knowledge, and processing capabilities as a set of utility services.3 Consequently, new grid systems are emerging to materialize these visions. Here, we review emerging grids and classify them to motivate further research and help establish a solid foundation in this rapidly evolving area

Autonomic Cloud Computing: Open Challenges and Architectural Elements

As Clouds are complex, large-scale, and heterogeneous distributed systems, management of their resources is a challenging task. They need automated and integrated intelligent strategies for provisioning of resources to offer services that are secure, reliable, and cost-efficient. Hence, effective management of services becomes fundamental in software platforms that constitute the fabric of computing Clouds. In this direction, this paper identifies open issues in autonomic resource provisioning and presents innovative management techniques for supporting SaaS applications hosted on Clouds. We present a conceptual architecture and early results evidencing the benefits of autonomic management of Clouds.Comment: 8 pages, 6 figures, conference keynote pape

Decentralized Scheduling for Many-Task Applications in the Hybrid Cloud

While Cloud Computing has transformed how we solve many computing tasks, some scientific and many-task applications are not efficiently executed on cloud resources. Decentralized scheduling, as studied in grid computing, can provide a scalable system to organize cloud resources and schedule a variety of work. By measuring simulations of two algorithms, the fully decentralized Organic Grid, and the partially decentralized Air Traffic Controller from IBM, we establish that decentralization is a workable approach, and that there are bottlenecks that can impact partially centralized algorithms. Through measurements in the cloud, we verify that our simulation approach is sound, and assess the variable performance of cloud resources. We propose a scheduler that measures the capabilities of the resources available to execute a task and distributes work dynamically at run time. Our scheduling algorithm is evaluated experimentally, and we show that performance-aware scheduling in a cloud environment can provide improvements in execution time. This provides a framework by which a variety of parameters can be weighed to make job-specific and context-aware scheduling decisions. Our measurements examine the usefulness of benchmarking as a metric used to measure a node\u27s performance, and drive scheduling. Benchmarking provides an advantage over simple queue-based scheduling on distributed systems whose members vary in actual performance, but the NAS benchmark we use does not always correlate perfectly with actual performance. The utilized hardware is examined, as are enforced performance variations, and we observe changes in performance that result in running on a system in which different workers receive different CPU allocations. As we see that performance metrics are useful near the end of the execution of a large job, we create a new metric from historical data of partially completed work, and use that to drive execution time down further. Interdependent task graph work is introduced and described as a next step in improving cloud scheduling. Realistic task graph problems are defined and a scheduling approach is introduced. This dissertation lays the groundwork to expand the types of problems that can be solved efficiently in the cloud environment

Stigmergy-based Load Scheduling in a Demand Side Management Context

This work proposes an approach, based on a fundamental coordination mechanism from nature, namely stigmergy. The proposed meta-heuristic is utilized to distributively calculate global schedules for a population of customers provided with intelligent devices. These schedules maximize renewable energy sources utilization. Furthermore, this approach is adapted and utilized as a coordination mechanism of autonomous customers to modify their consumption behavior in a real-time optimization context

Modular System for Shelves and Coasts (MOSSCO v1.0) - a flexible and multi-component framework for coupled coastal ocean ecosystem modelling

Shelf and coastal sea processes extend from the atmosphere through the water column and into the sea bed. These processes are driven by physical, chemical, and biological interactions at local scales, and they are influenced by transport and cross strong spatial gradients. The linkages between domains and many different processes are not adequately described in current model systems. Their limited integration level in part reflects lacking modularity and flexibility; this shortcoming hinders the exchange of data and model components and has historically imposed supremacy of specific physical driver models. We here present the Modular System for Shelves and Coasts (MOSSCO, http://www.mossco.de), a novel domain and process coupling system tailored---but not limited--- to the coupling challenges of and applications in the coastal ocean. MOSSCO builds on the existing coupling technology Earth System Modeling Framework and on the Framework for Aquatic Biogeochemical Models, thereby creating a unique level of modularity in both domain and process coupling; the new framework adds rich metadata, flexible scheduling, configurations that allow several tens of models to be coupled, and tested setups for coastal coupled applications. That way, MOSSCO addresses the technology needs of a growing marine coastal Earth System community that encompasses very different disciplines, numerical tools, and research questions.Comment: 30 pages, 6 figures, submitted to Geoscientific Model Development Discussion

08141 Abstracts Collection -- Organic Computing - Controlled Self-organization

From March 30th to April 4th 2008, the Dagstuhl Seminar 08141 "Organic Computing - Controlled Self-organization"\u27 was held in the International Conference and Research Center (IBFI), Schloss Dagstuhl. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed. Abstracts of the presentations given during the seminar as well as abstracts of seminar results and ideas are put together in this paper. The first section describes the seminar topics and goals in general. Links to extended abstracts or full papers are provided, if available

Revisiting Matrix Product on Master-Worker Platforms

This paper is aimed at designing efficient parallel matrix-product algorithms for heterogeneous master-worker platforms. While matrix-product is well-understood for homogeneous 2D-arrays of processors (e.g., Cannon algorithm and ScaLAPACK outer product algorithm), there are three key hypotheses that render our work original and innovative: - Centralized data. We assume that all matrix files originate from, and must be returned to, the master. - Heterogeneous star-shaped platforms. We target fully heterogeneous platforms, where computational resources have different computing powers. - Limited memory. Because we investigate the parallelization of large problems, we cannot assume that full matrix panels can be stored in the worker memories and re-used for subsequent updates (as in ScaLAPACK). We have devised efficient algorithms for resource selection (deciding which workers to enroll) and communication ordering (both for input and result messages), and we report a set of numerical experiments on various platforms at Ecole Normale Superieure de Lyon and the University of Tennessee. However, we point out that in this first version of the report, experiments are limited to homogeneous platforms

Scheduling Many-Task Computing Applications for a Hybrid Cloud

A centralized scheduler can become a bottleneck for placing the tasks of a many-task application on heterogeneous cloud resources. Previously, it was demonstrated that a decentralized vector scheduling approach based on performance measurements can be used successfully for this task placement scenario. In this dissertation, we extend this approach to task placement based on latency measurements. Each node collects performance metrics from its neighbors on an overlay graph, measures the communication latency, and then makes local decisions on where to move tasks. We present a decentralized and a centralized algorithm for configuring the overlay graph based on latency measurements and extend the vector scheduling approach to take latency into consideration. Our experiments in CloudLab, both in a simulated environment and in realistic conditions, demonstrate that this approach results in better performance and resource utilization than without latency information

A generic holonic control architecture for heterogeneous multi-scale and multi-objective smart microgrids

Designing the control infrastructure of future “smart” power grids is a challenging task. Future grids will integrate a wide variety of heterogeneous producers and consumers that are unpredictable and operate at various scales. Information and Communication Technology (ICT) solutions will have to control these in order to attain global objectives at the macrolevel, while also considering private interests at the microlevel. This article proposes a generic holonic architecture to help the development of ICT control systems that meet these requirements. We show how this architecture can integrate heterogeneous control designs, including state-of-the-art smart grid solutions. To illustrate the applicability and utility of this generic architecture, we exemplify its use via a concrete proof-of-concept implementation for a holonic controller, which integrates two types of control solutions and manages a multiscale, multiobjective grid simulator in several scenarios. We believe that the proposed contribution is essential for helping to understand, to reason about, and to develop the “smart” side of future power grids