Enhancing Energy Production with Exascale HPC Methods

High Performance Computing (HPC) resources have become the key actor for achieving more ambitious challenges in many disciplines. In this step beyond, an explosion on the available parallelism and the use of special purpose processors are crucial. With such a goal, the HPC4E project applies new exascale HPC techniques to energy industry simulations, customizing them if necessary, and going beyond the state-of-the-art in the required HPC exascale simulations for different energy sources. In this paper, a general overview of these methods is presented as well as some specific preliminary results.The research leading to these results has received funding from the European Union's Horizon 2020 Programme (2014-2020) under the HPC4E Project (www.hpc4e.eu), grant agreement n° 689772, the Spanish Ministry of Economy and Competitiveness under the CODEC2 project (TIN2015-63562-R), and from the Brazilian Ministry of Science, Technology and Innovation through Rede Nacional de Pesquisa (RNP). Computer time on Endeavour cluster is provided by the Intel Corporation, which enabled us to obtain the presented experimental results in uncertainty quantification in seismic imagingPostprint (author's final draft

Applying future Exascale HPC methodologies in the energy sector

The appliance of new exascale HPC techniques to energy industry simulations is absolutely needed nowadays. In this sense, the common procedure is to customize these techniques to the specific energy sector they are of interest in order to go beyond the state-of-the-art in the required HPC exascale simulations. With this aim, the HPC4E project is developing new exascale methodologies to three different energy sources that are the present and the future of energy: wind energy production and design, efficient combustion systems for biomass-derived fuels (biogas), and exploration geophysics for hydrocarbon reservoirs. In this work, the general exascale advances proposed as part of HPC4E and its outcome to specific results in different domains are presented.The research leading to these results has received funding from the European Union's Horizon 2020 Programme (2014-2020) under the HPC4E Project (www.hpc4e.eu), grant agreement n° 689772, the Spanish Ministry of Economy and Competitiveness under the CODEC2 project (TIN2015-63562-R), and from the Brazilian Ministry of Science, Technology and Innovation through Rede Nacional de Pesquisa (RNP). Computer time on Endeavour cluster is provided by the Intel Corporation, which enabled us to obtain the presented experimental results in uncertainty quantification in seismic imaging.Postprint (author's final draft

What is campus bridging and what is XSEDE doing about it?

The term “campus bridging” was first used in the charge given to an NSF Advisory Committee for Cyberinfrastructure task force. That task force developed this description of campus bridging: “Campus bridging is the seamlessly integrated use of cyberinfrastructure operated by a scientist or engineer with other cyberinfrastructure on the scientist’s campus, at other campuses, and at the regional, national, and international levels as if they were proximate to the scientist, and when working within the context of a Virtual Organization (VO) make the ‘virtual’ aspect of the organization irrelevant (or helpful) to the work of the VO.” Campus bridging is more a viewpoint and a set of approaches to usability, software, and information concerns than a particular set of tools or software. We outline here several specific use cases that have been identified as priorities for XSEDE in the next four years. These priorities include documentation, deployment of software used entirely outside of XSEDE, and software that helps bridge from individual researcher to campus to XSEDE cyberinfrastructure. We also describe early pilot tests and means by which the user community may stay informed of campus bridging activities and participate in the implementation of Campus Bridging tools created by XSEDE. Metrics are still being developed, and will include (1) the number of campuses that adopt and use Campus Bridging tools developed by XSEDE and (2) the number of and extent to which XSEDE-developed Campus Bridging tools are adopted among other CI projects.The work described here was supported by National Science Foundation Award Nos. 0932251, 0503697, 1002526, 1059812, 1040777, 0723054, 0521433, and 0504075

Development of a pattern library and a decision support system for building applications in the domain of scientific workflows for e-Science

Karastoyanova et al. created eScienceSWaT (eScience SoftWare Engineering Technique), that targets at providing a user-friendly and systematic approach for creating applications for scientific experiments in the domain of e-Science. Even though eScienceSWaT is used, still many choices about the scientific experiment model, IT experiment model and infrastructure have to be made. Therefore, a collection of best practices for building scientific experiments is required. Additionally, these best practice need to be connected and organized. Finally, a Decision Support System (DSS) that is based on the best practices and enables decisions about the various choices for e-Science solutions, needs to be developed. Hence, various e-Science applications are examined in this thesis. Best practices are recognised by abstracting from the identified problem-solution pairs in the e-Science applications. Knowledge and best practices from natural science, computer science and software engineering are stored in patterns. Furthermore, relationship types among patterns are worked out. Afterwards, relationships among the patterns are defined and the patterns are organized in a pattern library. In addition, the concept for a DSS that provisions the patterns and its prototypical implementation are presented

dispel4py: A Python framework for data-intensive scientific computing

This paper presents dispel4py, a new Python framework for describing abstract stream-based workflows for distributed data-intensive applications. These combine the familiarity of Python programming with the scalability of workflows. Data streaming is used to gain performance, rapid prototyping and applicability to live observations. dispel4py enables scientists to focus on their scientific goals, avoiding distracting details and retaining flexibility over the computing infrastructure they use. The implementation, therefore, has to map dispel4py abstract workflows optimally onto target platforms chosen dynamically. We present four dispel4py mappings: Apache Storm, message-passing interface (MPI), multi-threading and sequential, showing two major benefits: a) smooth transitions from local development on a laptop to scalable execution for production work, and b) scalable enactment on significantly different distributed computing infrastructures. Three application domains are reported and measurements on multiple infrastructures show the optimisations achieved; they have provided demanding real applications and helped us develop effective training. The dispel4py.org is an open-source project to which we invite participation. The effective mapping of dispel4py onto multiple target infrastructures demonstrates exploitation of data-intensive and high-performance computing (HPC) architectures and consistent scalability.</p

Machine Learning based Wind Power Forecasting for Operational Decision Support

To utilize renewable energy efficiently to meet the needs of mankind's living demands becomes an extremely hot topic since global warming is the most serious global environmental problem that human beings are facing today. Burning of fossil fuels, such as coal and oil directly for generating electricity leads to environment pollution and exacerbates global warning. However, large-scale development of hydropower increases greenhouse gas emissions and greenhouse effects. This research is related to knowledge of wind power forecasting (WPF) and machine learning (ML). This research is built around one central research question: How to improve the accuracy of WPF by using AI methods? A pilot conceptual system combining meteorological information and operations management has been formulated. The main contribution is visualized in a proposed new framework, named Meteorological Information Service Decision Support System, consisting of a meteorological information module, wind power prediction module and operations management module. This conceptual framework has been verified by quantitative analysis in empirical cases. This system utilizes meteorological information for decision-making based on condition-based maintenance in operations and management for the purpose of optimizing energy management. It aims to analyze and predict the variation of wind power for the next day or the following week to develop scheduling planning services for WPEs based on predicting wind speed for every six hours, which is short-term wind speed prediction, through training, validating, and testing dataset. Accurate prediction of wind speed is crucial for weather forecasting service and WPF. This study presents a carefully designed wind speed prediction model which combines fully-connected neural network (FCNN), long short-term memory (LSTM) algorithm with eXtreme Gradient Boosting (XGBoost) technique, to predict wind speed. The performance of each model is tested by using reanalysis data from European Center for Medium-Range Weather Forecasts (ECMWF) for Meteorological observatory located in Vaasa in Finland. The results show that XGBoost algorithm has similar improved prediction performance as LSTM algorithm, in terms of RMSE, MAE and R2 compared to the commonly used traditional FCNN model. On the other hand, the XGBoost algorithm has a significant advantage on training time while comparing to the other algorithms in this case study. Additionally, this sensitivity analysis indicates great potential of the optimized deep learning (DL) method, which is a subset of machine learning (ML), in improving local weather forecast on the coding platform of Python. The results indicate that, by using Meteorological Information Service Decision Support System, it is possible to support effective decision-making and create timely actions within the WPEs. Findings from this research contribute to WPF in WPEs. The main contribution of this research is achieving decision optimization on a decision support system by using ML. It was concluded that the proposed system is very promising for potential applications in wind (power) energy management