Realtime reservoir characterization and beyond: cyber-infrastructure tools and technologies

The advent of the digital oil _x000C_eld and rapidly decreasing cost of computing creates opportunities as well as challenges in simulation based reservoir studies, in particular, real-time reservoir characterization and optimization. One challenge our e_x000B_orts are directed toward is the use of real-time production data to perform live reservoir characterization using high throughput, high performance computing environments. To that end we developed the required tools of parallel reservoir simulator, parallel ensemble Kalman _x000C_lter and a scalable work ow manager. When using this collection of tools, a reservoir modeler is able to perform large scale reservoir management studies in short periods of time. This includes studies with thousands of models that are individually complex and large, involving millions of degrees of freedom. Using parallel processing, we are able to solve these models much faster than we otherwise would on a single, serial machine. This motivated the development of a fast parallel reservoir simulator. Furthermore, distributing those simulations across resources leads to a smaller total time to completion by making use of distributed processing. This allows the development of a scalable high throughput work ow manager. Finally, with thousands of models, each with millions of degrees of freedom, we end up with a super uity of model parameters. This translates directly to billions of degrees of freedom in the reservoir study. To be able to use the ensemble Kalman _x000C_lter on these models, we needed to develop a parallel implementation of the ensemble Kalman _x000C_lter. This thesis discusses the enabling tools and technologies developed to address a speci _x000C_c problem: how to accurately characterize reservoirs, using large numbers of complex detailed models. For these characterization studies to be helpful in making production decisions, the time to solution must be feasible. To that end, our work is focused on developing and extending these tools, and optimizing their performance

Many-Task Computing and Blue Waters

This report discusses many-task computing (MTC) generically and in the context of the proposed Blue Waters systems, which is planned to be the largest NSF-funded supercomputer when it begins production use in 2012. The aim of this report is to inform the BW project about MTC, including understanding aspects of MTC applications that can be used to characterize the domain and understanding the implications of these aspects to middleware and policies. Many MTC applications do not neatly fit the stereotypes of high-performance computing (HPC) or high-throughput computing (HTC) applications. Like HTC applications, by definition MTC applications are structured as graphs of discrete tasks, with explicit input and output dependencies forming the graph edges. However, MTC applications have significant features that distinguish them from typical HTC applications. In particular, different engineering constraints for hardware and software must be met in order to support these applications. HTC applications have traditionally run on platforms such as grids and clusters, through either workflow systems or parallel programming systems. MTC applications, in contrast, will often demand a short time to solution, may be communication intensive or data intensive, and may comprise very short tasks. Therefore, hardware and software for MTC must be engineered to support the additional communication and I/O and must minimize task dispatch overheads. The hardware of large-scale HPC systems, with its high degree of parallelism and support for intensive communication, is well suited for MTC applications. However, HPC systems often lack a dynamic resource-provisioning feature, are not ideal for task communication via the file system, and have an I/O system that is not optimized for MTC-style applications. Hence, additional software support is likely to be required to gain full benefit from the HPC hardware

Integrating multiple clusters for compute-intensive applications

Multicluster grids provide one promising solution to satisfying the growing computational demands of compute-intensive applications. However, it is challenging to seamlessly integrate all participating clusters in different domains into a single virtual computational platform. In order to fully utilize the capabilities of multicluster grids, computer scientists need to deal with the issue of joining together participating autonomic systems practically and efficiently to execute grid-enabled applications. Driven by several compute-intensive applications, this theses develops a multicluster grid management toolkit called Pelecanus to bridge the gap between user\u27s needs and the system\u27s heterogeneity. Application scientists will be able to conduct very large-scale execution across multiclusters with transparent QoS assurance. A novel model called DA-TC (Dynamic Assignment with Task Containers) is developed and is integrated into Pelecanus. This model uses the concept of a task container that allows one to decouple resource allocation from resource binding. It employs static load balancing for task container distribution and dynamic load balancing for task assignment. The slowest resources become useful rather than be bottlenecks in this manner. A cluster abstraction is implemented, which not only provides various cluster information for the DA-TC execution model, but also can be used as a standalone toolkit to monitor and evaluate the clusters\u27 functionality and performance. The performance of the proposed DA-TC model is evaluated both theoretically and experimentally. Results demonstrate the importance of reducing queuing time in decreasing the total turnaround time for an application. Experiments were conducted to understand the performance of various aspects of the DA-TC model. Experiments showed that our model could significantly reduce turnaround time and increase resource utilization for our targeted application scenarios. Four applications are implemented as case studies to determine the applicability of the DA-TC model. In each case the turnaround time is greatly reduced, which demonstrates that the DA-TC model is efficient for assisting application scientists in conducting their research. In addition, virtual resources were integrated into the DA-TC model for application execution. Experiments show that the execution model proposed in this thesis can work seamlessly with multiple hybrid grid/cloud resources to achieve reduced turnaround time

Akkujen käyttö ja kannattavuus sähköreservimarkkinoilla

A global trend in electricity generation has been that, in order to reduce the effect of global warming, wind and solar power will raise their share in the energy production. Consequently, there will be to more and more grid connected electricity generation that can't be designed in real time, depending on the required consumption needs, which increases the grid instability. Thus, large-scale electro-chemical battery storage systems have been considered as a potential solution for electricity imbalance challenges. On March 1st 2017, a 2 MW and 1 MWh lithium ion battery energy storage system started operating in Finland under the project name of Batcave. The purpose of this thesis is to simulate how the battery storage system in question would operate in Finnish electricity reserve markets with the support of hydropower. The simulation is conducted with a designed battery model and operated throughout the data of 2016 in the hourly market of frequency containment reserve for normal operation. For the battery simulations two principal scenarios were chosen: All-scenario and Hydro-scenario. In All-scenario the battery operates in all possible market hours that provide income and in Hydro-scenario the battery functions in the same hours when the hydroelectric power would be operated. Simulation results indicate that from all the active hours 8 % of that time the battery wasn't able to operate whereas it included over one third of the total energy flow that the battery wasn't capable of delivering. Thus, temporally the battery can operate alone rather well but when idling occurs, the hydro backup reserve is required to perform this task in a technical manner. Furthermore, the size and cost of additional energy capacities are too substantial in order to reach a completely autonomic battery with no backup reserves or additional optimizations and always carrying out the grid requirements. From the net present value calculations throughout the battery life expectancy, All-scenario has better profitability prospects than Hydro-scenario. Consequently, it is more profitable to operate the battery almost as much as possible to assure quick cash flow than spare its use. With All-scenario the project can be profitable with an investment subsidy. The optimization calculations indicate that the minimum price limit of bidding the battery in frequency containment reserve for normal operation has a desirable scope of values. Further studies should focus on systematic battery use and active data analyzing to further improve the battery model and enhance the technical and economic operation of battery storage system.Globaalisti sähköntuotannossa on havaittavissa pyrkimystä vähentää kasvihuoneilmiötä lisäämällä sekä tuuli- että aurinkovoimaa. Näin ollen sähköntuotannossa esiintyy yhä enemmän tuotantoa, jota ei voida ohjata reaaliaikaisesti kulutuksen mukaisesti, mikä puolestaan kasvattaa verkon epävakautta. Tämän seurauksena ison mittakaavan sähkökemiallisia akkuvarastointijärjestelmiä pidetään potentiaalisena ratkaisuna sähköverkon tasapainotushaasteisiin. 1. Maaliskuuta 2017 aloitti Suomessa toimintansa 2 MW:n ja 1 MWh:n kokoinen litium-ioni akkuvarastointijärjestelmä nimeltään Batcave. Tämän diplomityön tarkoituksena on simuloida, miten kyseinen akkuvarastointijärjestelmä toimisi Suomen sähköreservi-markkinoilla vesivoiman avulla. Simulointi on suoritettu akkumallilla, joka on mallinnettu operoimaan taajuusohjatun käyttöreservin tuntimarkkinalla hyödyntäen vuoden 2016 tietoja. Akkusimulointiin valittiin kaksi skenaariota: All-skenaario ja Hydro-skenaario. All-skenaariossa akku toimii jokaisella mahdollisella tuottavalla markkina-tunnilla kun taas Hydro-skenaariossa akku toimii samoilla tunneilla kuin vesivoima. Simulointitulokset osoittavat, että kaikista aktiivisista tunneista 8 % oli sellaista aikaa, jolloin akku ei pystynyt toimimaan, kun taas energiamäärällisesti se vastasi yli kolmasosaa energiavirtauksista, joita akku ei pystynyt toteuttamaan. Näin ollen ajallisesti akku pystyy operoimaan melko itsenäisesti, mutta jos tyhjäkäyntiä tapahtuu, niin vesivoimaa tarvitaan vastaamaan sen teknisiin vaatimuksiin. Tämän lisäksi ekstra-energiakapasiteetin hankkiminen olisi liian massiivista ja kallista, jotta täysin itsenäinen akkusysteemi voisi toimia ilman varareserviä tai ylimääräistä optimointia ja samalla pystyisi aina toteuttamaan pakolliset verkkovaatimukset. Nettonykyarvolaskelmat akun elinajanodotteeseen asti osoittavat, että All-skenaariolla on paremmat tuotto-odotukset kuin Hydro-skenaariolla. Näin ollen on kannattavampaa käyttää akkua miltei niin paljon kuin mahdollista taatakseen nopea rahavirta kuin säästää sen käyttöä. All-skenaariolla projekti voi olla tuottava investointitukien kanssa. Lisäksi optimointilaskut viittaavat siihen, että akulla on havaittavissa tuottoisin tarjousväli taajuusohjatun käyttöreservin tuntimarkkinalla. Jatkotutkimukset on suotavaa kohdistaa akun systemaattiseen käyttöön ja aktiiviseen data-analysointiin, jotta akkumallia voitaisiin entisestään kehittää ja akkuvarastointisysteemin teknistä- ja taloudellista toimintaa parantaa

Advances in Grid Computing

This book approaches the grid computing with a perspective on the latest achievements in the field, providing an insight into the current research trends and advances, and presenting a large range of innovative research papers. The topics covered in this book include resource and data management, grid architectures and development, and grid-enabled applications. New ideas employing heuristic methods from swarm intelligence or genetic algorithm and quantum encryption are considered in order to explain two main aspects of grid computing: resource management and data management. The book addresses also some aspects of grid computing that regard architecture and development, and includes a diverse range of applications for grid computing, including possible human grid computing system, simulation of the fusion reaction, ubiquitous healthcare service provisioning and complex water systems

Self-management for large-scale distributed systems

Autonomic computing aims at making computing systems self-managing by using autonomic managers in order to reduce obstacles caused by management complexity. This thesis presents results of research on self-management for large-scale distributed systems. This research was motivated by the increasing complexity of computing systems and their management. In the first part, we present our platform, called Niche, for programming self-managing component-based distributed applications. In our work on Niche, we have faced and addressed the following four challenges in achieving self-management in a dynamic environment characterized by volatile resources and high churn: resource discovery, robust and efficient sensing and actuation, management bottleneck, and scale. We present results of our research on addressing the above challenges. Niche implements the autonomic computing architecture, proposed by IBM, in a fully decentralized way. Niche supports a network-transparent view of the system architecture simplifying the design of distributed self-management. Niche provides a concise and expressive API for self-management. The implementation of the platform relies on the scalability and robustness of structured overlay networks. We proceed by presenting a methodology for designing the management part of a distributed self-managing application. We define design steps that include partitioning of management functions and orchestration of multiple autonomic managers. In the second part, we discuss robustness of management and data consistency, which are necessary in a distributed system. Dealing with the effect of churn on management increases the complexity of the management logic and thus makes its development time consuming and error prone. We propose the abstraction of Robust Management Elements, which are able to heal themselves under continuous churn. Our approach is based on replicating a management element using finite state machine replication with a reconfigurable replica set. Our algorithm automates the reconfiguration (migration) of the replica set in order to tolerate continuous churn. For data consistency, we propose a majority-based distributed key-value store supporting multiple consistency levels that is based on a peer-to-peer network. The store enables the tradeoff between high availability and data consistency. Using majority allows avoiding potential drawbacks of a master-based consistency control, namely, a single-point of failure and a potential performance bottleneck. In the third part, we investigate self-management for Cloud-based storage systems with the focus on elasticity control using elements of control theory and machine learning. We have conducted research on a number of different designs of an elasticity controller, including a State-Space feedback controller and a controller that combines feedback and feedforward control. We describe our experience in designing an elasticity controller for a Cloud-based key-value store using state-space model that enables to trade-off performance for cost. We describe the steps in designing an elasticity controller. We continue by presenting the design and evaluation of ElastMan, an elasticity controller for Cloud-based elastic key-value stores that combines feedforward and feedback control