Mass production of event simulations for the BaBar experiment using the Grid

The BaBar experiment has been taking data since 1999, investigating the violation of charge and parity (CP) symmetry in the field of High Energy Physics. Event simulation is an intensive computing task, due to the complexity of the algorithm based on the Monte Carlo method implemented using the GEANT engine. The simulation input data are stored in ROOT format, they are classified into two categories: conditions data for describing the detector status when data are recorded, and background triggers data for including the noise signal necessary to obtain a realistic simulation. In order to satisfy these requirements, in the traditional BaBar computing model events are distributed over several sites involved in the collaboration where each site manager centrally manages a private farm dedicated to simulation production. The new grid approach applied to the BaBar production framework is discussed along with the schema adopted for data deployment via Xrootd/Scalla servers, including data management using grid middleware on distributed storage facilities spread over the INFN-GRID network. A comparison between the two models is provided, describing also the custom applications developed for performing the whole production task on the grid and showing the results achieved

Scalability tests of R-GMA-based grid job monitoring system for CMS Monte Carlo data production

Copyright @ 2004 IEEEHigh-energy physics experiments, such as the compact muon solenoid (CMS) at the large hadron collider (LHC), have large-scale data processing computing requirements. The grid has been chosen as the solution. One important challenge when using the grid for large-scale data processing is the ability to monitor the large numbers of jobs that are being executed simultaneously at multiple remote sites. The relational grid monitoring architecture (R-GMA) is a monitoring and information management service for distributed resources based on the GMA of the Global Grid Forum. We report on the first measurements of R-GMA as part of a monitoring architecture to be used for batch submission of multiple Monte Carlo simulation jobs running on a CMS-specific LHC computing grid test bed. Monitoring information was transferred in real time from remote execution nodes back to the submitting host and stored in a database. In scalability tests, the job submission rates supported by successive releases of R-GMA improved significantly, approaching that expected in full-scale production

Energy-Aware Cloud Management through Progressive SLA Specification

Novel energy-aware cloud management methods dynamically reallocate computation across geographically distributed data centers to leverage regional electricity price and temperature differences. As a result, a managed VM may suffer occasional downtimes. Current cloud providers only offer high availability VMs, without enough flexibility to apply such energy-aware management. In this paper we show how to analyse past traces of dynamic cloud management actions based on electricity prices and temperatures to estimate VM availability and price values. We propose a novel SLA specification approach for offering VMs with different availability and price values guaranteed over multiple SLAs to enable flexible energy-aware cloud management. We determine the optimal number of such SLAs as well as their availability and price guaranteed values. We evaluate our approach in a user SLA selection simulation using Wikipedia and Grid'5000 workloads. The results show higher customer conversion and 39% average energy savings per VM.Comment: 14 pages, conferenc

Dynamically adaptive partition-based data distribution management

Workshop on Principles of Advanced and Distributed Simulation, PADS 2005; Monterey, CA; United States; 1 June 2005 through 3 June 2005Performance and scalability of distributed simulations depends primarily on the effectiveness of the employed data distribution management (DDM) algorithm, which aims at reducing the overall computational and messaging effort on the shared data to a necessary minimum. Existing DDM approaches, which are variations and combinations of two basic techniques, namely region-based and grid-based techniques, perform purely in the presence of load differences. We introduce the partition-based technique that allows for variable-size partitioning shared data. Based on this technique, a novel DDM algorithm is introduced that is dynamically adaptive to cluster formations in the shared data as well as in the physical location of the simulation objects. Since the re-distribution is sensitive to inter-relationships between shared data and simulation objects, a balanced constellation has the additional advantage to be of minimal messaging effort. Furthermore, dynamic system scalability is facilitated, as bottlenecks are avoided

Maintaining Replica Consistency Over Large-Scale Data Grid Using Update Propagation Technique

A Data Grid is an organized collection of nodes in a wide area network which contributes to various computation, storage data, and application. In Data Grid high numbers of users are distributed in a wide area environment which is dynamic and heterogeneous. Data management is one of the current issues where data transparency, consistency, fault-tolerance, automatic management and the performance are the user parameters in grid environment. Data management techniques must scale up while addressing autonomy, dynamicity and heterogeneity of the data resource. Data replication is a well known technique used to reduce accesses latency, improve availability and performance in a distributed computing environment. Replication introduces the problem of maintaining consistency among the replicas when files are allowed to be updated. The update information should be propagated to all replicas to guarantee correct read of the remote replicas. An asynchronous replication is a commonly agreed solution for the problem in consistency of replicas. A few studies have been done to maintain replica consistency in Data Grid. However, the introduced techniques are neither efficient nor scalable. They cannot be used in real Data Grid since the issues of large number of replica sites, large scale distribution, load balancing and site autonomy where the capability of grid site to join and leave the grid community at any time have not been addressed. This thesis proposes a new asynchronous replication protocol called Update Propagation Grid (UPG) to maintain replica consistency over a large scale data grid. In UPG the updates reach all on-line secondary replicas using a propagation technique based on nodes organized into a logical structure network in the form of two-dimensional grid structure. The proposed update propagation technique is a hybrid push-pull and dynamic technique that addresses the issues of site autonomy, efficiency, scalability, load balancing and fairness. A two performance analysis studies have been conducted to study the performance of the proposed technique in comparison with other techniques. First study involves mathematical and simulation analysis. Second study is based on Queuing Network Model. The result of the performance analysis shows that the proposed technique scales well with high number of replica sites and with high request loads. The result also shows the reduction on the average update reach time by 5% to 97%. Moreover the result shows that the proposed technique is capable of reaching load balancing while providing update propagation fairnes

Resilient Distributed MPC Algorithm for Microgrid Energy Management under Uncertainties

This paper proposes a resilient distributed energy management algorithm able to cope with different types of faults in a DC microgrid system. A distributed optimization method allows to solve the energy management problem without sharing any private data with the network and reducing the computational cost for each agent, with respect to the centralised case. A distributed MPC scheme based on distributed optimization is used to cope with uncertainty that characterizes the microgrid operation. In order to be resilient to faults that limit the amount of power available to consumers, we propose to adaptively store an amount of power in the storage systems to support the loads. A soft constraint on the minimum energy stored in each battery is introduced for feasibility and to cope with persistent faults. The effectiveness of the method is proved by extensive simulation results considering faults on three types of components: renewable generator, distribution grid and communication network

Quality of service management in service-oriented grids

Grid computing provides a robust paradigm for aggregating disparate resources in a secure and controlled environment. The emerging grid infrastructure gives rise to a class of scientific applications and services in support of collaborative and distributed resource-sharing requirements, as part of teleimmersion, visualization and simulation services. Because such applications operate in a collaborative mode, data must be stored, processed and delivered in a timely manner. Such classes of applications have collaborative and distributed resource-sharing requirements, and have stringent real-time constraints and quality-of-service (QoS) requirements. A QoS management approach is therefore essential to orchestrate and guarantee the interaction among such applications in a distributed computing environment. Grid architectures require an underpinning of QoS support to manage complex computation-intensive and data-intensive applications, as current grid middleware solutions lack QoS provision. QoS guarantees in the grid context have, however, not been given the importance they merit. To enhance its functionality, a computational grid must be overlaid with an advanced QoS architecture to best execute those applications with real-time constraints. This thesis reports on the design and implementation of a software framework, called Grid QoS Management (G-QoSm). G-QoSm incorporates a new QoS management model and provides a service-oriented QoS management approach that supports the Open Grid Service Architecture. Its novel features include grid-service discovery based on QoS attributes, immediate and advance resource reservation, service execution with QoS constraints, and techniques for QoS adaptation to compensate for resource degradation, and to optimise resource allocation while maintaining a service level agreement. The benefits of G-QoSm are demonstrated by prototype test-beds that integrate scientific grid applications and simulate grid data-transfer applications. Results show that the grid application and the data-transfer simulation have better performance when used with the proposed QoS approach. QoS abstractions are presented for building QoS-aware applications, in the context of service-oriented grids. These abstractions are application programming interfaces to facilitate application developers utilising the proposed QoS management solution.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Dynamically adaptive partition-based interest management in distributed simulation

Performance and scalability of distributed simulations depends primarily on the effectiveness of the employed interest management (IM) schema that aims at reducing the overall computational and messaging effort on the shared data to a necessary minimum. Existing IM approaches, which are based on variations or combinations of two principle data distribution techniques, namely region-based and grid-based techniques, perform poorly if the simulation develops an overloaded host. In order to facilitate distributing the processing load from overloaded areas of the shared data to less loaded hosts, the partition-based technique is introduced that allows for variable-size partitioning the shared data. Based on this data distribution technique, an IM approach is sketched that is dynamically adaptive to access latencies of simulation objects on the shared data as well as to the physical location of the objects. Since this re-distribution is decided depending on the messaging effort of the simulation objects for updating data partitions, any load balanced constellation has the additional advantage to be of minimal overall messaging effort. Hence, the IM schema dynamically resolves messaging overloading as well as overloading of hosts with simulation objects and therefore facilitates dynamic system scalability

Dynamic Grid-Based Data Distribution Management in Large Scale Distributed Simulations

Distributed simulation is an enabling concept to support the networked interaction of models and real world elements that are geographically distributed. This technology has brought a new set of challenging problems to solve, such as Data Distribution Management (DDM). The aim of DDM is to limit and control the volume of the data exchanged during a distributed simulation, and reduce the processing requirements of the simulation hosts by relaying events and state information only to those applications that require them. In this thesis, we propose a new DDM scheme, which we refer to as dynamic grid-based DDM. A lightweight UNT-RTI has been developed and implemented to investigate the performance of our DDM scheme. Our results clearly indicate that our scheme is scalable and it significantly reduces both the number of multicast groups used, and the message overhead, when compared to previous grid-based allocation schemes using large-scale and real-world scenarios

Continuous reservoir model updating by ensemble Kalman filter on Grid computing architectures

A reservoir engineering Grid computing toolkit, ResGrid and its extensions, were developed and applied to designed reservoir simulation studies and continuous reservoir model updating. The toolkit provides reservoir engineers with high performance computing capacity to complete their projects without requiring them to delve into Grid resource heterogeneity, security certification, or network protocols. Continuous and real-time reservoir model updating is an important component of closed-loop model-based reservoir management. The method must rapidly and continuously update reservoir models by assimilating production data, so that the performance predictions and the associated uncertainty are up-to-date for optimization. The ensemble Kalman filter (EnKF), a Bayesian approach for model updating, uses Monte Carlo statistics for fusing observation data with forecasts from simulations to estimate a range of plausible models. The ensemble of updated models can be used for uncertainty forecasting or optimization. Grid environments aggregate geographically distributed, heterogeneous resources. Their virtual architecture can handle many large parallel simulation runs, and is thus well suited to solving model-based reservoir management problems. In the study, the ResGrid workflow for Grid-based designed reservoir simulation and an adapted workflow provide tools for building prior model ensembles, task farming and execution, extracting simulator output results, implementing the EnKF, and using a web portal for invoking those scripts. The ResGrid workflow is demonstrated for a geostatistical study of 3-D displacements in heterogeneous reservoirs. A suite of 1920 simulations assesses the effects of geostatistical methods and model parameters. Multiple runs are simultaneously executed using parallel Grid computing. Flow response analyses indicate that efficient, widely-used sequential geostatistical simulation methods may overestimate flow response variability when compared to more rigorous but computationally costly direct methods. Although the EnKF has attracted great interest in reservoir engineering, some aspects of the EnKF remain poorly understood, and are explored in the dissertation. First, guidelines are offered to select data assimilation intervals. Second, an adaptive covariance inflation method is shown to be effective to stabilize the EnKF. Third, we show that simple truncation can correct negative effects of nonlinearity and non-Gaussianity as effectively as more complex and expensive reparameterization methods