Carbon-Intelligent Global Routing in Path-Aware Networks

The growing energy consumption of Information and Communication Technology (ICT) has raised concerns about its environmental impact. However, the carbon efficiency of data transmission over the Internet has so far received little attention. This carbon efficiency can be enhanced effectively by sending traffic over carbon-efficient inter-domain paths. However, challenges in estimating and disseminating carbon intensity of inter-domain paths have prevented carbon-aware path selection from becoming a reality. In this paper, we take advantage of path-aware network architectures to overcome these challenges. In particular, we design CIRo, a system for forecasting the carbon intensity of inter-domain paths and disseminating them across the Internet. We implement a proof of concept for CIRo on the codebase of the SCION path-aware Internet architecture and test it on the SCIONLab global research testbed. Further, we demonstrate the potential of CIRo for reducing the carbon footprint of endpoints and end domains through large-scale simulations. We show that CIRo can reduce the carbon intensity of communications by at least 47% for half of the domain pairs and the carbon footprint of Internet usage by at least 50% for 87% of end domains

Energy Efficient Ethernet on MapReduce Clusters: Packet Coalescing To Improve 10GbE Links

An important challenge of modern data centers is to reduce energy consumption, of which a substantial proportion is due to the network. Switches and NICs supporting the recent energy efficient Ethernet (EEE) standard are now available, but current practice is to disable EEE in production use, since its effect on real world application performance is poorly understood. This paper contributes to this discussion by analyzing the impact of EEE on MapReduce workloads, in terms of performance overheads and energy savings. MapReduce is the central programming model of Apache Hadoop, one of the most widely used application frameworks in modern data centers. We find that, while 1GbE links (edge links) achieve good energy savings using the standard EEE implementation, optimum energy savings in the 10 GbE links (aggregation and core links) are only possible, if these links employ packet coalescing. Packet coalescing must, however, be carefully configured in order to avoid excessive performance degradation. With our new analysis of how the static parameters of packet coalescing perform under different cluster loads, we were able to cover both idle and heavy load periods that can exist on this type of environment. Finally, we evaluate our recommendation for packet coalescing for 10 GbE links using the energy-delay metric. This paper is an extension of our previous work [1], which was published in the Proceedings of the 40th Annual IEEE Conference on Local Computer Networks (LCN 2015).This work was supported in part by the European Union’s Seventh Framework Programme (FP7/2007-2013) under Grant 610456 (EUROSERVER), in part by the Spanish Government through the Severo Ochoa programme (SEV-2011-00067 and SEV-2015-0493), in part by the Spanish Ministry of Economy a nd Competitiveness under Contract TIN2012-34557 and Contract TIN2015-65316-P, and in part by the Generalitat de Catalunya under Contract 2014-SGR-1051 and Contract 2014-SGR-1272.Peer ReviewedPostprint (author's final draft

E-EON : Energy-Efficient and Optimized Networks for Hadoop

Energy efficiency and performance improvements have been two of the major concerns of current Data Centers. With the advent of Big Data, more information is generated year after year, and even the most aggressive predictions of the largest network equipment manufacturer have been surpassed due to the non-stop growing network traffic generated by current Big Data frameworks. As, currently, one of the most famous and discussed frameworks designed to store, retrieve and process the information that is being consistently generated by users and machines, Hadoop has gained a lot of attention from the industry in recent years and presently its name describes a whole ecosystem designed to tackle the most varied requirements of today’s cloud applications. This thesis relates to Hadoop clusters, mainly focused on their interconnects, which is commonly considered to be the bottleneck of such ecosystem. We conducted research focusing on energy efficiency and also on performance optimizations as improvements on cluster throughput and network latency. Regarding the energy consumption, a significant proportion of a data center's energy consumption is caused by the network, which stands for 12% of the total system power at full load. With the non-stop growing network traffic, it is desired by industry and academic community that network energy consumption should be proportional to its utilization. Considering cluster performance, although Hadoop is a network throughput-sensitive workload with less stringent requirements for network latency, there is an increasing interest in running batch and interactive workloads concurrently on the same cluster. Doing so maximizes system utilization, to obtain the greatest benefits from the capital and operational expenditures. For this to happen, cluster throughput should not be impacted when network latency is minimized. The two biggest challenges faced during the development of this thesis were related to achieving near proportional energy consumption for the interconnects and also improving the network latency found on Hadoop clusters, while having virtually no loss on cluster throughput. Such challenges led to comparable sized opportunity: proposing new techniques that must solve such problems from the current generation of Hadoop clusters. We named E-EON the set of techniques presented in this work, which stands for Energy Efficient and Optimized Networks for Hadoop. E-EON can be used to reduce the network energy consumption and yet, to reduce network latency while cluster throughput is improved at the same time. Furthermore, such techniques are not exclusive to Hadoop and they are also expected to have similar benefits if applied to any other Big Data framework infrastructure that fits the problem characterization we presented throughout this thesis. With E-EON we were able to reduce the energy consumption by up to 80% compared to the state-of-the art technique. We were also able to reduce network latency by up to 85% and in some cases, even improve cluster throughput by 10%. Although these were the two major accomplishment from this thesis, we also present minor benefits which translate to easier configuration compared to the stat-of-the-art techniques. Finally, we enrich the discussions found in this thesis with recommendations targeting network administrators and network equipment manufacturers.La eficiencia energética y las mejoras de rendimiento han sido dos de las principales preocupaciones de los Data Centers actuales. Con el arribo del Big Data, se genera más información año con año, incluso las predicciones más agresivas de parte del mayor fabricante de dispositivos de red se han superado debido al continuo tráfico de red generado por los sistemas de Big Data. Actualmente, uno de los más famosos y discutidos frameworks desarrollado para almacenar, recuperar y procesar la información generada consistentemente por usuarios y máquinas, Hadoop acaparó la atención de la industria en los últimos años y actualmente su nombre describe a todo un ecosistema diseñado para abordar los requisitos más variados de las aplicaciones actuales de Cloud Computing. Esta tesis profundiza sobre los clusters Hadoop, principalmente enfocada a sus interconexiones, que comúnmente se consideran el cuello de botella de dicho ecosistema. Realizamos investigaciones centradas en la eficiencia energética y también en optimizaciones de rendimiento como mejoras en el throughput de la infraestructura y de latencia de la red. En cuanto al consumo de energía, una porción significativa de un Data Center es causada por la red, representada por el 12 % de la potencia total del sistema a plena carga. Con el tráfico constantemente creciente de la red, la industria y la comunidad académica busca que el consumo energético sea proporcional a su uso. Considerando las prestaciones del cluster, a pesar de que Hadoop mantiene una carga de trabajo sensible al rendimiento de red aunque con requisitos menos estrictos sobre la latencia de la misma, existe un interés creciente en ejecutar aplicaciones interactivas y secuenciales de manera simultánea sobre dicha infraestructura. Al hacerlo, se maximiza la utilización del sistema para obtener los mayores beneficios al capital y gastos operativos. Para que esto suceda, el rendimiento del sistema no puede verse afectado cuando se minimiza la latencia de la red. Los dos mayores desafíos enfrentados durante el desarrollo de esta tesis estuvieron relacionados con lograr un consumo energético cercano a la cantidad de interconexiones y también a mejorar la latencia de red encontrada en los clusters Hadoop al tiempo que la perdida del rendimiento de la infraestructura es casi nula. Dichos desafíos llevaron a una oportunidad de tamaño semejante: proponer técnicas novedosas que resuelven dichos problemas a partir de la generación actual de clusters Hadoop. Llamamos a E-EON (Energy Efficient and Optimized Networks) al conjunto de técnicas presentadas en este trabajo. E-EON se puede utilizar para reducir el consumo de energía y la latencia de la red al mismo tiempo que el rendimiento del cluster se mejora. Además tales técnicas no son exclusivas de Hadoop y también se espera que tengan beneficios similares si se aplican a cualquier otra infraestructura de Big Data que se ajuste a la caracterización del problema que presentamos a lo largo de esta tesis. Con E-EON pudimos reducir el consumo de energía hasta en un 80% en comparación con las técnicas encontradas en la literatura actual. También pudimos reducir la latencia de la red hasta en un 85% y, en algunos casos, incluso mejorar el rendimiento del cluster en un 10%. Aunque estos fueron los dos principales logros de esta tesis, también presentamos beneficios menores que se traducen en una configuración más sencilla en comparación con las técnicas más avanzadas. Finalmente, enriquecimos las discusiones encontradas en esta tesis con recomendaciones dirigidas a los administradores de red y a los fabricantes de dispositivos de red

Cross-Layer Design for Energy Efficiency on Data Center Network

Energy efficient infrastructures or green IT (Information Technology) has recently become a hot button issue for most corporations as they strive to eliminate every inefficiency from their enterprise IT systems and save capital and operational costs. Vendors of IT equipment now compete on the power efficiency of their devices, and as a result, many of the new equipment models are indeed more energy efficient. Various studies have estimated the annual electricity consumed by networking devices in the U.S. in the range of 6 - 20 Terra Watt hours. Our research has the potential to make promising solutions solve those overuses of electricity. An energy-efficient data center network architecture which can lower the energy consumption is highly desirable. First of all, we propose a fair bandwidth allocation algorithm which adopts the max-min fairness principle to decrease power consumption on packet switch fabric interconnects. Specifically, we include power aware computing factor as high power dissipation in switches which is fast turning into a key problem, owing to increasing line speeds and decreasing chip sizes. This efficient algorithm could not only reduce the convergence iterations but also lower processing power utilization on switch fabric interconnects. Secondly, we study the deployment strategy of multicast switches in hybrid mode in energy-aware data center network: a case of famous Fat-tree topology. The objective is to find the best location to deploy multicast switch not only to achieve optimal bandwidth utilization but also minimize power consumption. We show that it is possible to easily achieve nearly 50% of energy consumption after applying our proposed algorithm. Finally, although there exists a number of energy optimization solutions for DCNs, they consider only either the hosts or network, but not both. We propose a joint optimization scheme that simultaneously optimizes virtual machine (VM) placement and network flow routing to maximize energy savings. The simulation results fully demonstrate that our design outperforms existing host- or network-only optimization solutions, and well approximates the ideal but NP-complete linear program. To sum up, this study could be crucial for guiding future eco-friendly data center network that deploy our algorithm on four major layers (with reference to OSI seven layers) which are physical, data link, network and application layer to benefit power consumption in green data center

VihreäIT metriikoiden analysointi sekä mittausviitekehyksen luonti Sonera Helsinki Datakeskus (HDC) projektille.

The two objectives of this thesis were to investigate and evaluate the most suitable set of energy efficiency metrics for Sonera Helsinki Data Center (HDC), and to analyze which energy efficient technologies could be implemented and in what order to gain most impact. Sustainable IT is a complex matter, and it has two components. First and the more complex matter is the energy efficiency and energy-proportionality of the IT environment. The second is the use of renewable energy sources. Both of these need to be addressed. This thesis is a theoretical study, and it focuses on energy efficiency. The use of off-site renewables is outside of the scope of this thesis. The main aim of this thesis is to improve energy efficiency through effective metric framework. In the final metric framework, metrics that target renewable energy usage in the data center are included as they are important from CO2 emission reduction perspective. The selection of energy efficient solutions in this thesis are examples from most important data center technology categories, and do not try to cover the whole array of different solutions to improve energy efficiency in a data center. The ontological goal is to present main energy efficiency metrics available in scientific discourse, and also present examples of energy efficient solutions in most energy consuming technology domains inside the data center. Even though some of the concepts are quite abstract, realism is taken into account in every analysis. The epistemology in this thesis is based on scientific articles that include empirical validation and scientific peer review. This forms the origin of the used knowledge and the nature of this knowledge. The findings from this thesis are considered valid and reliable based on the epistemology of scientific articles, and by using the actual planning documents of Sonera HDC. The reasoning in this thesis is done in abstracto, but there are many empirical results that qualify the results also as ´in concreto´. Findings are significant for Sonera HDC but they are also applicable for any general data center project or company seeking energy efficiency in their data centers.Lopputyöllä on kaksi päätavoitetta. Ensimmäinen tavoite on löytää sopivin mittausviitekehys energiatehokkuuden osoittamiseksi Sonera Helsinki Datakeskukselle (HDC). Toisena tavoitteena on analysoida, mitä energiatehokkaita ratkaisuja tulisi implementoida ja missä järjestyksessä, saavuttaakseen mahdollisimman ison vaikutuksen. Vihreä IT on monimutkainen asia ja samalla siihen liittyy kaksi eri komponenttia. Ensimmäisenä komponenttina, ja merkityksellisempänä sekä monimutkaisempana, on energiatehokkuus ja energian kulutuksen mukautuvuus suhteessa työkuormaan. Toinen komponentti vihreän IT:n osalta on uusiutuvien energialähteiden käyttäminen. Molemmat komponentit on huomioitava. Lopputyö on teoreettinen tutkimus. Lopputyön ontologinen tavoite on esittää keskeisimmät energiatehokkuusmittarit, jotka ovat saatavilla tieteellisessä keskustelussa, ja esittää myös esimerkkejä energiatehokkaista ratkaisuista teknologia-alueisiin, jotka kuluttavat eniten energiaa data keskuksissa. Vaikka osa esitetyistä ratkaisuista on melko abstraktissa todellisuudessa, realismi on pyritty ottamaan huomioon arvioita tehdessä. Epistemologisesti tämä lopputyö perustuu tieteellisiin artikkeleihin, joissa on tehty empiiristä validointia ja tiedeyhteisön vertaisarviointia tiedon totuusarvosta. Kirjoittaja pyrkii välttämään oman arvomaailman ja subjektiivisen näkemyksen tuomista analyysiin pyrkimällä enemmänkin arvioimaan ratkaisuja perustuen päätavoitteeseen, joka on sekä lisätä energiatehokkuutta että vähentää CO2 -päästöjä datakeskuksessa. Lopputyön löydökset todetaan valideiksi ja luotettaviksi, koska ne perustuvat tieteellisten artikkeleiden epistemologiaan ja siihen, että arvioinnin pohjana on käytetty todellisia Sonera HDC -projektin suunnitteludokumentteja. Päätelmät ja analyysit ovat abstrahoituja, mutta perustuvat empiirisiin tuloksiin, jotka koskevat käytännön tekemistä sekä valintoja. Löydökset ovat merkittäviä Sonera HDC -projektin kannalta, ja myös muille datakeskuksille, jotka haluavat toimia kestävän kehityksen pohjalta

Research on grid challenges and smart grid development: the case of Sichuan grid

As the most important driving force of modern social development and a significant symbol of modern civilization, electric power is in booming demand. Furthermore, electric power is a complex system which integrates power generation, power transmission, power distribution and power utilization together and achieves generation, transmission, distribution and utilization instantaneously at the same time. It notably features with network industry and network economy. Power grid is a hub which links electricity production and electricity consumption in the power system. On the basis of basic theories of network industry and network economy, this thesis discusses the development of smart grid from the aspects of “network challenges”, resources and energy challenges and new energy access challenges encountered and counter-measures in the development of modern grid. Based on the development environment of China power, especially the Sichuan power grid, and spatial mismatching of power supply and demand (including new energy resources and distribution), this thesis analyzes and explains China (Sichuan) smart grid is strong smart grid which has UHV power grid as the backbone frame, and features information technology, and automation.Devido ao facto de ser uma força impulsionadora do desenvolvimento económico e um simbolo muito importante da civilização moderna, a procura de electricidade tem aumentado consideravelmente nas últimas décadas. Contudo, a energia elétrica é um sistema complexo que integra geração, transmissão, distribuição e implica que a oferta e a procura sejam simultâneas. A indústria da electricidade tem muitas características da economia em rede. A rede elétrica deve ser vista como um “hub” que liga a produção de electricidade ao seu consumo. Tendo por base, as teorias da indústria em rede e da economia em rede, esta tese discute o desenvolvimento das redes elétricas segundo as perspectivas dos “desafios que se colocam às redes”, dos desafios em termos de recursos e dos desafios que se colocam ao desenvolvimento da rede elétrica moderna. Esta tese estuda de uma forma detalhada os problemas relacionados com a construção de uma rede elétrica inteligente na província de Sichuan, China

Energy-efficient Communications in Cloud, Mobile Cloud and Fog Computing

This thesis studies the problem of energy efficiency of communications in distributed computing paradigms, including cloud computing, mobile cloud computing and fog/edge computing. Distributed computing paradigms have significantly changed the way of doing business. With cloud computing, companies and end users can access the vast majority services online through a virtualized environment in a pay-as-you-go basis. %Three are the main services typically consumed by cloud users are Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and Software as a Service (SaaS). Mobile cloud and fog/edge computing are the natural extension of the cloud computing paradigm for mobile and Internet of Things (IoT) devices. Based on offloading, the process of outsourcing computing tasks from mobile devices to the cloud, mobile cloud and fog/edge computing paradigms have become popular techniques to augment the capabilities of the mobile devices and to reduce their battery drain. Being equipped with a number of sensors, the proliferation of mobile and IoT devices has given rise to a new cloud-based paradigm for collecting data, which is called mobile crowdsensing as for proper operation it requires a large number of participants. A plethora of communication technologies is applicable to distributing computing paradigms. For example, cloud data centers typically implement wired technologies while mobile cloud and fog/edge environments exploit wireless technologies such as 3G/4G, WiFi and Bluetooth. Communication technologies directly impact the performance and the energy drain of the system. This Ph.D. thesis analyzes from a global perspective the efficiency in using energy of communications systems in distributed computing paradigms. In particular, the following contributions are proposed: - A new framework of performance metrics for communication systems of cloud computing data centers. The proposed framework allows a fine-grain analysis and comparison of communication systems, processes, and protocols, defining their influence on the performance of cloud applications. - A novel model for the problem of computation offloading, which describes the workflow of mobile applications through a new Directed Acyclic Graph (DAG) technique. This methodology is suitable for IoT devices working in fog computing environments and was used to design an Android application, called TreeGlass, which performs recognition of trees using Google Glass. TreeGlass is evaluated experimentally in different offloading scenarios by measuring battery drain and time of execution as key performance indicators. - In mobile crowdsensing systems, novel performance metrics and a new framework for data acquisition, which exploits a new policy for user recruitment. Performance of the framework are validated through CrowdSenSim, which is a new simulator designed for mobile crowdsensing activities in large scale urban scenarios

Interconnector Investment for a Well-Functioning Internal Market: What EU regime of regulatory incentives? Bruges European Economic Research (BEER) Papers 18/October 2010

Sufficient cross‐border electricity transmission infrastructure is a pre‐requisite for a functioning European internal market for electricity. Also, the achievement of the EU’s energy policy objectives – sustainability, competitiveness and security of supply – critically depends on adequate investment in physical interconnections between the member states. Mainly focusing on the “regulatory path”, this paper assesses different ways to achieve a sufficient level of interconnector investment. In a first step, economic analysis identifies numerous impediments to interconnector investment adding up to an “interconnector investment failure”. Reflecting on the proper regulatory design of an EU framework able to overcome the interconnector investment failure, a number of recommendations are put forward: All congestion rents should be channeled into interconnector building. Unused rents should be transferred to a European interconnector fund supervised by an EU agency. Even though inherently sub‐optimal, merchant transmission investment can be used as a means to put pressure on regulated transmission system operators (TSO) that do not deliver. An EU agency should have exclusive competence on merchant interconnector exemptions. A European TSO organization should be entrusted with supra‐national network planning, supervised by an EU agency. The agency should decide on investment cost reallocation for interconnector projects that yield strong externalities. Payments could be settled via a European interconnector fund. In case of non‐compliance with the supra‐national network plan, the EU agency should have the right to organize a tender – financed by the European interconnector fund – in order to get the “missing link” built. Assessing the existing EU regulatory framework, the efforts of the 2009 “third energy package” to fill the “regulatory gap” with new EU bodies – ACER and ENTSO‐E – are acknowledged. However, striking holes in regulatory framework are spotted, notably with regard to the use of congestion rents, interconnector cost allocation, and the distribution of decision making powers on new infrastructure exemptions A discussion of the TEN‐E interconnector funding scheme shows that massive funding can be an interim solution to the problem of insufficient interconnection capacities while overcoming the political deadlock on sensible regulatory topics such as interconnector cost allocation. The paper ends with policy recommendations