Datacenter Traffic Control: Understanding Techniques and Trade-offs

Datacenters provide cost-effective and flexible access to scalable compute and storage resources necessary for today's cloud computing needs. A typical datacenter is made up of thousands of servers connected with a large network and usually managed by one operator. To provide quality access to the variety of applications and services hosted on datacenters and maximize performance, it deems necessary to use datacenter networks effectively and efficiently. Datacenter traffic is often a mix of several classes with different priorities and requirements. This includes user-generated interactive traffic, traffic with deadlines, and long-running traffic. To this end, custom transport protocols and traffic management techniques have been developed to improve datacenter network performance. In this tutorial paper, we review the general architecture of datacenter networks, various topologies proposed for them, their traffic properties, general traffic control challenges in datacenters and general traffic control objectives. The purpose of this paper is to bring out the important characteristics of traffic control in datacenters and not to survey all existing solutions (as it is virtually impossible due to massive body of existing research). We hope to provide readers with a wide range of options and factors while considering a variety of traffic control mechanisms. We discuss various characteristics of datacenter traffic control including management schemes, transmission control, traffic shaping, prioritization, load balancing, multipathing, and traffic scheduling. Next, we point to several open challenges as well as new and interesting networking paradigms. At the end of this paper, we briefly review inter-datacenter networks that connect geographically dispersed datacenters which have been receiving increasing attention recently and pose interesting and novel research problems.Comment: Accepted for Publication in IEEE Communications Surveys and Tutorial

Renewable Electric Energy Integration: Quantifying the Value of Design of Markets for International Transmission Capacity

Integrating large quantities of supply-driven renewable electricity generation remains a political and operational challenge. One of the main obstacles in Europe to installing at least 200 GWs of power from variable renewable sources is how to deal with the insufficient network capacity and the congestion that will result from new flow patterns. We model the current methodology for controlling congestion at international borders and compare its results, under varying penetrations of wind power, with a model that simulates an integrated European network that utilises nodal/localised marginal pricing. The nodal pricing simulations illustrate that congestion - and price - patterns vary considerably between wind scenarios and within countries, and that a nodal price regime could make fuller use of existing EU network capacity, introducing substantial operational cost savings and reducing marginal power prices in the majority of European countries.Power market design, renewable power integration, congestion management, transmission economics

Agglomerative Magnets and Informal Regulatory Networks: Electricity Market Design Convergence in the USA and Continental Europe

The absence of one broadly accepted design template for liberalised electricity markets induces regulatory competition and institutional diversity. Focussing on continental Europe and the USA, this analysis explores how agents and structures accelerate or impede the move to one standard market design in the electricity sector. It reveals that market design convergence in Europe is driven by the 'Florence Consensus,' a tripartite coalition between the European Commission fostering European integration and the internal market, informal regulatory networks between grid operators, standardisation authorities and regulators, who have been coordinating their actions in the 'Florence Forum,' and epistemic communities exemplified in the Florence School of Regulation. In contrast, the United States' Federal Energy Regulatory Commission lacks support among politicians, many states' public utility commissions, the neo-liberal intelligentsia and even industrial lobbying groups to effectively push for a standardised market design. However, design convergence in the USA may be induced by the gradual expansion of multi-state markets operated by regional transmission organisations.Electricity, Deregulation, Regulatory Competition, Policy Diffusion