Simulating Distributed Systems

The simulation framework developed within the "Models of Networked Analysis at Regional Centers" (MONARC) project as a design and optimization tool for large scale distributed systems is presented. The goals are to provide a realistic simulation of distributed computing systems, customized for specific physics data processing tasks and to offer a flexible and dynamic environment to evaluate the performance of a range of possible distributed computing architectures. A detailed simulation of a large system, the CMS High Level Trigger (HLT) production farm, is also presented

Implementation of Grid-computing Framework for Simulation in Multi-scale Structural Analysis

A new grid-computing framework for simulation in multi-scale structural analysis is presented. Two levels of parallel processing will be involved in this framework: multiple local distributed computing environments connected by local network to form a grid-based cluster-to-cluster distributed computing environment. To successfully perform the simulation, a large-scale structural system task is decomposed into the simulations of a simplified global model and several detailed component models using various scales. These correlated multi-scale structural system tasks are distributed among clusters and connected together in a multi-level hierarchy and then coordinated over the internet. The software framework for supporting the multi-scale structural simulation approach is also presented. The program architecture design allows the integration of several multi-scale models as clients and servers under a single platform. To check its feasibility, a prototype software system has been designed and implemented to perform the proposed concept. The simulation results show that the software framework can increase the speedup performance of the structural analysis. Based on this result, the proposed grid-computing framework is suitable to perform the simulation of the multi-scale structural analysis

STRUCTURAL SYSTEM SIMULATION USING GRID-COMPUTING FRAMEWORK

A multi-level modeling and simulation method of structural system using grid-computing framework is proposed in this paper. Two levels of parallel processing will be involved in this framework: (1) multiple locally distributed computing environments connected by the local network to form (2) a grid-based cluster-to-cluster distributed computing environment. To successfully perform the simulations, a large-scale structural system is decomposed into the simulations of a simplified global model and several detailed component models with various scales. These correlated multi-scale simulation tasks are distributed amongst clusters and connected together in a multi-level modeling and simulation method and then coordinated over the internet. This paper also presents the development of a grid-computing software framework that can support the proposed simulation approach. The architectural design of the program also allows the integration of several multi-scale models to be clients and servers under a single platform. Additionally, the comparison result between proposed method and assumed exact solution show that the proposed simulation method is appropriate to simulate the response of the structural systems

Long term nitrogen budget modelling in a small agricultural watershed: hydrological control assessment of nitrogen losses with semi-distributed (SWAT) and distributed (TNT2) models

Nitrogen exports in catchments are known to be greatly variable because nitrogen cycle in watershed is controlled by different factors such as landuse, farm management practices, climate, soil type and hydrological setting. Our aim is to study the relative importance of the processes controlling nitrogen losses at catchment scale in the long term using a modelling approach constrained by a long term record of observations. The study area is a catchment of 330 ha with 95 % of intensive agriculture in a hilly shallow soil context, in the south west of France. Historical field rotation and nitrogen river load data have been collected for a 20 year period. Two process-based and spatially distributed models have been chosen to simulate nitrogen transfer and transformation in the whole catchment. The first one is the fully distributed TNT2 model, developed and validated in a different context (farming systems in north-western France). The second one is the widely used, semi-distributed SWAT model, used and recognizedto be realistic in many studies on nitrogen transfer in river. This comparative modelling approach was used to evaluate the effect of different modelling approaches on the identification of controlling factors, and the ability of both models to simulate alternative scenarios. The discharge, especially during storm flow, is well simulated by the curve number approach and the semi-distributed hydrological parameter description used SWAT, while the Topmodel-derived approach used in TNT2 tends to underestimate some peak discharges. Nitrogen dynamic simulations are considered to be acceptable for both models for a long time period but the use of both models allows to exhibit their respective capacity and limits. TNT2 has higher potentiality to test the impact of complex agricultural scenarios because the description of management practices and the simulation of crops to management options is more detailed. It permits the assessment of spatial interactions and focussed spatial management, like the set up of grass or tree strips. SWAT can then be used to scale up change scenarios from TNT2 small catchment results to large catchments

A Critique of the Telecommunications Description Language (TeD)

TeD is an object-oriented description language designed to facilitate the modeling of large scale telecommunication networks, with simulation on parallel and distributed platforms. TeD models are mapped to the Georgia Tech Time Warp engine (GTW) for execution. In this paper we outline the features of TeD, pointing out its strengths and identifying characteristics that gave us trouble as we used TeD to model detailed networks. Our issues are motivated specifically by a model of TCP and a model of multicast resource allocation. Our intention is to illustrate by example what TeD can do, and characteristics that a potential TeD user should be aware of

Optimization and Management of Large-scale Scientific Workflows in Heterogeneous Network Environments: From Theory to Practice

Next-generation computation-intensive scientific applications feature large-scale computing workflows of various structures, which can be modeled as simple as linear pipelines or as complex as Directed Acyclic Graphs (DAGs). Supporting such computing workflows and optimizing their end-to-end network performance are crucial to the success of scientific collaborations that require fast system response, smooth data flow, and reliable distributed operation.We construct analytical cost models and formulate a class of workflow mapping problems with different mapping objectives and network constraints. The difficulty of these mapping problems essentially arises from the topological matching nature in the spatial domain, which is further compounded by the resource sharing complicacy in the temporal dimension. We provide detailed computational complexity analysis and design optimal or heuristic algorithms with rigorous correctness proof or performance analysis. We decentralize the proposed mapping algorithms and also investigate these optimization problems in unreliable network environments for fault tolerance.To examine and evaluate the performance of the workflow mapping algorithms before actual deployment and implementation, we implement a simulation program that simulates the execution dynamics of distributed computing workflows. We also develop a scientific workflow automation and management platform based on an existing workflow engine for experimentations in real environments. The performance superiority of the proposed mapping solutions are illustrated by extensive simulation-based comparisons with existing algorithms and further verified by large-scale experiments on real-life scientific workflow applications through effective system implementation and deployment in real networks

Open source interface for distribution system modeling in power system co-simulation applications and two algorithms for populating feeder models, An

2017 Spring.Includes bibliographical references.The aging electric infrastructure power system infrastructure is undergoing a transformative change mainly triggered by the large-scale integration of distributed resources such as distributed generation, hybrid loads, and home energy management systems at the end-use level. The future electric grid, also referred to as the Smart Grid, will make use of these distributed resources to intelligently manage the day to day power system operations with minimum human intervention. The proliferation of these advanced Smart Grid resources may lead to coordination problems to maintain the generation-demand balance at all times. To ensure their safe integration with the grid, extensive simulation studies need to be performed using distributed resources. Simulation studies serve as an economically viable alternative to avoid expensive failures. They also serve as an invaluable platform to study energy consumption behavior, demand response, power system stability, and power system state estimation. Traditionally, power system analysis has been performed in isolated domains using simulation tools for the transmission and distribution systems. Moreover, modeling all the power system assets using a single power system tool is difficult and inconclusive. From the Smart Grid perspective, a common simulation platform for different power systems analysis tools is essential. A co-simulation framework enables the interaction of multiple power system tools, each modeling a single domain in detail, to run simultaneously and provide a holistic power system overview. To enable the co-simulation framework, a data exchange platform between the transmission and distribution system simulators is proposed to model transmission and distribution assets on different simulation testbeds. A graphical user interface (GUI) is developed as a frontend tool for the data exchange platform and makes use of two developed algorithms that simplifies the task of: 1. modeling distribution assets consisting of diverse feeder datasets for the distribution simulator and balanced three-phase level assets for the transmission system simulator, and 2. populating the distribution system with loads having stochastic profiles for timestep simulations. The load profiles used in the distribution system models are created using concepts from one-dimensional random walk theory to mimic the energy consumption behavior of residential class of consumers. The algorithms can simulate large scale distribution system assets linked to a transmission system for co-simulation applications. The proposed algorithms are tested on the standard test system – Roy Billinton Test System (RBTS) to model detailed distribution assets linked to a selected transmission node. Two open source power system simulators—MATPOWER© and GridLAB-D© are used for the transmission and distribution simulation process. The algorithms accurately create detailed distribution topology populated with 4026 residential loads expanded from the transmission node, bus 2 in RBTS. Thus, an automated modeling of power system transmission and distribution assets is proposed along with its application using a standard test system is provided

SoL-A PV generation model for grid integration analysis in distribution networks

Photovoltaic (PV) generation systems are increasingly being integrated into distribution networks, presenting new challenges for network operators and planners. To fully quantify the impact of this distributed generation, power systems and control engineers are often required to model a large number of networks across multiple scenarios of PV uptake and allocation. However, due to the complexity of these network models and simulation methods, and the associated computational constraints, very simplified models are often adopted for solar irradiance, and PV module and inverter output power. These ad hoc models often utilise simple linear relationships, and are not validated against real data or more detailed models. The proposed model (SoL) is composed of pre-existing equations selected from the current state of the art models, as well as newly formulated equations. Each submodel is validated against pre-existing models using data from a number of meteorological sites around the world, showing good performance with reduced computational complexity. The resulting model is subsequently shown to be suitable for running large scale quasi-steady state power flow simulations and Monte Carlo analyses as required for grid integration studies.The authors would like to acknowledge the funding provided by theMinistry of Business Innovation and Employment, Transpower, and the EEA for the GREEN Grid project.Publicad

Spatial scale effect of surface routing and its parameter upscaling for urban flood simulation using a grid‐based model

Urban catchments are characterized by a wide variety of complex juxtapositions and surface compositions that are linked to multiple overland flow paths. Their extremely high spatial heterogeneity leads to great sensitivity of hydrologic simulation to the scale variation of calculation units. Although extensive efforts have been made for investigating the scale effects and indicate its significance, less is understood of how routing features vary with spatial scales and further how the variation of routing features influences the hydrological response. In this paper, a grid-based distributed urban hydrological model is applied to study spatial scale effects ranging from 10 to 250 m. Two parameters are proposed to quantitatively depict the routing features of overland flow specified for impervious and pervious areas. The results show that routing features are quite sensitive to spatial resolution. Large differences among simulations exist in the infiltration amounts attributed to the combined effects of the two routing parameters, which leads to opposite effects for both total flow volume and peak flow for various rainfall events. The relationship of the key model parameters at different spatial resolutions can be explicitly expressed by corresponding routing features. With this relationship, parameters transfer among different spatial scales can be realized to obtain consistent simulation results. This study further revealed the quantitative relationship between spatial scales, routing features and the hydrologic processes, and enabled accurate and efficient simulations required by real time flooding forecasting and land-atmosphere coupling, while fully taking the advantages of detailed surface information. Plain Language Summary Given the inherent complex underlying surface compositions and overland flow paths in urban areas, underlying high spatial resolution surface data eventually become necessary. Unfortunately, high resolution modelling in urban catchment is still challenging in terms of computational restricts, proper setting up of parameters etc., due to the high spatial heterogeneity. Practical simulation requirements often limit the use of high resolution models, as in the case of real time prediction of urban flooding, the coupling of land-atmosphere processes. Therefore, it is necessary to investigate the scale effects and its mechanism, and then to explore an accommodation approach to enable precise flooding prediction with a coarse model. For grid-based and distributed hydrologic models, the mosaic method can basically eliminate the scale effects on the runoff generation process. However, the scale effects on overland flow routing remain insufficiently understood, and to help understand the scale effects, simulations were performed under five different resolutions, ranging from 10 m to 250 m, for various rainfall events. Two physical parameters are introduced to quantify the scale effects on routing features. Three variables are concurrently calculated to assess the effects on modeling outputs. The results indicate that routing features are sensitive to changes in spatial resolution, which results in opposite effects on simulation results under different rainfall conditions. In conclusion, an accommodation approach is proposed based on the affecting mechanism