The maximal utilization of processor co-allocation in multicluster systems

In systems consisting of multiple clusters of processors which employ space sharing for scheduling jobs, such as our distributed ASCI supercomputer (DAS), co-allocation, i.e., the simultaneous allocation of processors to single jobs in multiple clusters, may be required. In studies of scheduling in single clusters it has been shown that the achievable (maximal) utilization may be much less than 100%, a problem that may be aggravated in multicluster systems. In this paper we study the maximal utilization when co-allocating jobs in multicluster systems, both with analytic means (we derive exact and approximate formulas when the service-time distribution is exponential), and with simulations with synthetic workloads and with workloads derived from the logs of actual systems

Performance comparison of parallel programming environments for implementing AIAC algorithms

International audienceAIAC algorithms (Asynchronous Iterations Asynchronous Communications) are a particular class of parallel iterative algorithms. Their asynchronous nature makes them more efficient than their synchronous counterparts in numerous cases as has already been shown in previous works. The first goal of this article is to compare several parallel programming environments in order to see if there is one of them which is best suited to efficiently implement AIAC algorithms. The main criterion for this comparison consists in the performances achieved in a global context of grid computing for two classical scientific problems. Nevertheless, we also take into account two secondary criteria which are the ease of programming and the ease of deployment. The second goal of this study is to extract from this comparison the important features that a parallel programming environment must have in order to be suited for the implementation of AIAC algorithms

Performance impact of the grid middleware

The Open Grid Services Architecture (OGSA) defines a new vision of the Grid based on the use of Web Services (Grid Services). The standard interfaces, behaviors and schemes that are consistent with the OGSA specification are defined by the Open Grid Service Infrastructure (OGSI). Grid Services, as an extension of the Web Services, run on top of rich execution frameworks that make them accessible and interoperable with other applications. Two examples of these frameworks are Sun’s J2EE platform and Microsoft’s .NET. The Globus Project implements the OGSI Specification for the J2EE framework in the Globus Toolkit. As any J2EE application, the performance of the Globus Toolkit is constrained by the performance obtained by the J2EE execution stack This performance can be influenced by many points of the execution stack: operating system, JVM, middleware or the same grid service, without forgetting the processing overheads related to the parsing of the communication protocols. In the scope of this chapter, all this levels together will be referred to as the grid middleware. In order to avoid the grid middleware to become a performance bottleneck for a distributed grid-enabled application, grid nodes have to be tuned for an efficient execution of I/O intensive applications because they can receive a high volume of requests every second and have to deal with a big amount of invocations, message parsing operations and a continuous task of marshaling and unmarshalling service parameters. All the parameters of the system affecting these operations have to be tuned according with the expected system load intensity. A Grid node is connected to to other nodes through a network connection which is also a decisive factor to obtain a high performance for a grid application. If the inter-node data transmission time overlaps completely the processing time for a computational task, the benefits of the grid architecture will be lost. Additionally, in many situations the content exchanged between grid nodes can be considered confidential and should be protected from curious sights. But the cost of data encryption/decryption can be an important performance weak that must be taken into account. In this chapter we will study the process of receiving and executing a Grid job from the perspective of the underlying levels existing below the Grid application. We will analyze the different performance parameters that can influence in the performance of the Grid middleware and will show the general schema of tasks involved in the service of an execution request.Postprint (author’s final draft

Classification of Robustness and Resilience in Changeability

Turbulenzen im Fabrikumfeld stehen seit den Neunzigerjahren für einen permanenten Strukturwandel durch kontinuierliche Veränderungen, die Megatrends wie die Digitalisierung verstärken. Da die Prognosefähigkeit von Entwicklungen praktisch nicht mehr gegeben ist, basiert die Fabrikplanung auf Annahmen und geht im Gegensatz zu Entscheidungen unter Sicherheit mit Planungsrisiken einher. Um in diesem unsicheren Umfeld erfolgreich am Markt zu bestehen, benötigen Unternehmen veränderungsfähige Fabriken. Auf-grund der zunehmenden Organisation von Fabriken in Produktionsnetzwerken und einer steigenden Komplexität von Produktionsprozessen ist die Unsicherheit im Fabrikbetrieb zunehmend mit Risiken verbunden. Die Anzahl an störenden Ereignissen hat zugenommen, die zu einer Betriebsunterbrechung führen können. Deshalb haben vermehrt die Begriffe Robustheit und Resilienz Einzug in den Produktionskontext gehalten. Aktuell bleibt es unklar, wie genau die Fabrikplanung gestalterisch tätig werden kann, um dem Fabrikbetrieb den notwendigen Handlungsspielraum zu übertragen. Das Ziel dieses Artikels ist es deshalb, die Robustheit und Resilienz in den Kontext der Veränderungsfähigkeit einzuordnen, deren Konzepte bisher ausreichten, um Fabriken als elementares Strukturelement eines Produktionsnetzwerkes auf den Umgang mit turbulenten Einflüssen einzustellen.Turbulences in the factory environment have represented permanent structural transformation since the nineteen nineties through continuous changes that are intensified by megatrends such as digitalization. As the ability to forecast developments is practically non-existent, factory planning is based on assumptions and is associated with planning risks, in contrast to decisions made under certainty. In order to compete successfully on the market in this uncertain environment, companies need changeable factories. Due to the increasing organization of factories in production networks and a growing complexity of production processes, uncertainty in factory operation is increasingly associated with risks. The number of disturbing events has increased, which can lead to a business disruption. Therefore, the concepts of robustness and resilience have increasingly entered the production context. Currently, it remains unclear how exactly factory planning can take formative measures to give factory operation the necessary scope for action. Therefore, the goal of this article is to classify robustness and resilience in the context of changeability, whose concepts have been sufficient so far to adapt factories as an elementary structural element of a production network to handle turbulent influences

Power and energy profiling of scientific applications on distributed systems

Abstract , by node (for each of 32 nodes), and by system scale (2, 4, 8, 16, and 32 node

Designing a Resource Broker for Heterogeneous Grids

Grids provide uniform access to aggregations of heterogeneous resources and services such as computers, networks and storage owned by multiple organizations. However, such a dynamic environment poses many challenges for application composition and deployment. In this paper, we present the design of the Gridbus Grid resource broker that allows users to create applications and specify different objectives through different interfaces without having to deal with the complexity of Grid infrastructure. We present the unique requirements that motivated our design and discuss how these provide flexibility in extending the functionality of the broker to support different low-level middlewares and user interfaces. We evaluate the broker with different job profiles and Grid middleware and conclude with the lessons learnt from our development experience.Comment: 26 pages, 15 figure

Time and Space Optimization for Processing Groups of Multi-Dimensional Scientific Queries

Data analysis applications in areas as diverse as remote sensing and telepathology require operating on and processing very large datasets. For such applications to execute efficiently, careful attention must be paid to the storage, retrieval, and manipulation of the datasets. This paper addresses the optimizations performed by a high performance database system that processes groups of data analysis requests for these applications, which we call queries. The system performs end-to-end processing of the requests, formulated as PostgreSQL declarative queries. The queries are converted into imperative descriptions, multiple imperative descriptions are merged into a single execution plan, the plan is optimized to decrease execution time via common compiler optimization techniques, and, finally, the plan is optimized to decrease memory consumption. The last two steps effectively reduce both the time and space to execute query groups, as shown in the experimental results. (UMIACS-TR-2004-14