Metascheduling of HPC Jobs in Day-Ahead Electricity Markets

High performance grid computing is a key enabler of large scale collaborative computational science. With the promise of exascale computing, high performance grid systems are expected to incur electricity bills that grow super-linearly over time. In order to achieve cost effectiveness in these systems, it is essential for the scheduling algorithms to exploit electricity price variations, both in space and time, that are prevalent in the dynamic electricity price markets. In this paper, we present a metascheduling algorithm to optimize the placement of jobs in a compute grid which consumes electricity from the day-ahead wholesale market. We formulate the scheduling problem as a Minimum Cost Maximum Flow problem and leverage queue waiting time and electricity price predictions to accurately estimate the cost of job execution at a system. Using trace based simulation with real and synthetic workload traces, and real electricity price data sets, we demonstrate our approach on two currently operational grids, XSEDE and NorduGrid. Our experimental setup collectively constitute more than 433K processors spread across 58 compute systems in 17 geographically distributed locations. Experiments show that our approach simultaneously optimizes the total electricity cost and the average response time of the grid, without being unfair to users of the local batch systems.Comment: Appears in IEEE Transactions on Parallel and Distributed System

Market-Based Scheduling in Distributed Computing Systems

In verteilten Rechensystemen (bspw. im Cluster und Grid Computing) kann eine Knappheit der zur Verfügung stehenden Ressourcen auftreten. Hier haben Marktmechanismen das Potenzial, Ressourcenbedarf und -angebot durch geeignete Anreizmechanismen zu koordinieren und somit die ökonomische Effizienz des Gesamtsystems zu steigern. Diese Arbeit beschäftigt sich anhand vier spezifischer Anwendungsszenarien mit der Frage, wie Marktmechanismen für verteilte Rechensysteme ausgestaltet sein sollten

Monte Carlo-Based Tail Exponent Estimator

In this paper we study the finite sample behavior of the Hill estimator under α-stable distributions. Using large Monte Carlo simulations we show that the Hill estimator overestimates the true tail exponent and can hardly be used on samples with small length. Utilizing our results, we introduce a Monte Carlo-based method of estimation for the tail exponent. Our method is not sensitive to the choice of k and works well also on small samples. The new estimator gives unbiased results with symmetrical con_dence intervals. Finally, we demonstrate the power of our estimator on the main world stock market indices. On the two separate periods of 2002-2005 and 2006-2009 we estimate the tail exponent.Hill estimator, α-stable distributions, tail exponent estimation

Energy policy and regulatory challenges in natural gas infrastructure and supply in the energy transition in Sweden

Sweden is undergoing a major energy transition in which the present regulatory, competition and energy decisions will determine future involvement in the “oil and gas game” after decades of successful implementation of non-fossil fuel dependence policies. Contrary to major energy policies implemented since the oil crisis of the 70’s, higher natural gas investment in infrastructure – in particular regarding offshore pipelines – is not an outcome of a consented agreement between the government and private firms. The lack of clear governmental definition towards the time to phase out nuclear terminals, and how this source of energy would be replaced, is leading the country towards an energy bottleneck that could condition future energy supply, thus governance. Under these conditions, crucial decisions shall be taken in the near future regarding granting permissions to pipelines that connect to the Russian natural gas fields following an EU trend, to the Norwegian natural gas reserves on the trail of a Nordic energy path-dependence, or to both, sharing potential benefits and risks.Energy policy; Future energy supply; Natural gas infrastructure; Energy transition; Sweden; Russian natural gas fields; Norwegian natural gas reserves; Swedish energy transition.

Putting science at the heart of European policymaking: An exhibition of the European Commission’s science and knowledge service Joint Research Centre (JRC)

The Joint Research Centre (JRC) invites you on a journey to discover why and how it puts science at the heart of European policymaking. After taking in the main milestones of the JRC’s evolution from early years to maturity, immerse yourself in information. The JRC addresses the difficulty of making sense of collective scientific knowledge for policymaking and is at the forefront of the Open Data movement. The modern JRC today makes sense of knowledge, cuts through information overload, brings clarity to complexity and has become a one-stop shop for policy Directorate-Generals (DGs) in evidence-based policy making. Selected highlights offer a glimpse of the JRC’s recent contributions to a selection of 5 priority policy topics, while the ‘globe’ illustrates stories on some of the topics. A virtual visit to one of our labs is also offered, as well as a glimpse of some of the JRC science and knowledge services that are fit for the 21st centuryJRC.H.2-Knowledge for Thematic Coordinatio