Delay Analysis of Timer-Based Frame Coalescing in Energy Efficient Ethernet

Cataloged from PDF version of article.IEEE 802.3az, also known as Energy Efficient Ethernet (EEE), aims at reducing the energy consumption of an Ethernet link by placing it in sleep mode when the link is idle. Frame coalescing mechanism proposed for EEE is an effective means to increase the average idle time of the link, thus reducing the overhead stemming from sleep/wake transitions, but at the expense of increased frame delays. Therefore, it is imperative to quantify the energy-delay trade-off while employing frame coalescing. As opposed to existing delay models that focus only on the average delays, a simple but exact queuing model is introduced for timer-based frame coalescing to find the delay distribution when the frame arrival process is Poisson and frame lengths are generally distributed. An expression for average saving in power consumption is also provided

Optimization of energy efficiency in data and WEB hosting centers

Mención Internacional en el título de doctorThis thesis tackles the optimization of energy efficiency in data centers in terms of network and server utilization. For what concerns networking utilization the work focuses on Energy Efficient Ethernet (EEE) - IEEE 802.3az standard - which is the energy-aware alternative to legacy Ethernet, and an important component of current and future green data centers. More specifically the first contribution of this thesis consists in deriving and analytical model of gigabit EEE links with coalescing using M/G/1 queues with sleep and wake-up periods. Packet coalescing has been proposed to save energy by extending the sojourn in the Low Power Idle state of EEE. The model presented in this thesis approximates with a good accuracy both the energy saving and the average packet delay by using a few significant traffic descriptors. While coalescing improves by far the energy efficiency of EEE, it is still far from achieving energy consumption proportional to traffic. Moreover, coalescing can introduce high delays. To this extend, by using sensitivity analysis the thesis evaluates the impact of coalescing timers and buffer sizes, and sheds light on the delay incurred by adopting coalescing schemes. Accordingly, the design and study of a first family of dynamic algorithms, namely measurement-based coalescing control (MBCC), is proposed. MBCC schemes tune the coalescing parameters on-the-fly, according to the instantaneous load and the coalescing delay experienced by the packets. The thesis also discusses a second family of dynamic algorithms, namely NT-policy coalescing control (NTCC), that adjusts the coalescing parameters based on the sole occurrence of timeouts and buffer fill-ups. Furthermore, the performance of static as well as dynamic coalescing schemes is investigated using real traffic traces. The results reported in this work show that, by relying on run-time delay measurements, simple and practical MBCC adaptive coalescing schemes outperform traditional static and dynamic coalescing while the adoption of NTCC coalescing schemes has practically no advantages with respect to static coalescing when delay guarantees have to be provided. Notably, MBCC schemes double the energy saving benefit of legacy EEE coalescing and allow to control the coalescing delay. For what concerns server utilization, the thesis presents an exhaustive empirical characterization of the power requirements of multiple components of data center servers. The characterization is the second key contribution of this thesis, and is achieved by devising different experiments to stress server components, taking into account the multiple available CPU frequencies and the presence of multicore servers. The described experiments, allow to measure energy consumption of server components and identify their optimal operational points. The study proves that the curve defining the minimal CPU power utilization, as a function of the load expressed in Active Cycles Per Second, is neither concave nor purely convex. Instead, it definitively shows a superlinear dependence on the load. The results illustrate how to improve the efficiency of network cards and disks. Finally, the accuracy of the model derived from the server components consumption characterization is validated by comparing the real energy consumed by two Hadoop applications - PageRank and WordCount - with the estimation from the model, obtaining errors below 4:1%, on average.This work has been partially supported by IMDEA Networks Institute and the Greek State Scholarships FoundationPrograma Oficial de Doctorado en Ingeniería TelemáticaPresidente: Marco Giuseppe Ajmone Marsan.- Secretario: Jose Luis Ayala Rodrigo.- Vocal: Gianluca Antonio Rizz

Leveraging energy saving capabilities of current EEE interfaces via pre-coalescing

The low power idle mode implemented by Energy Efficient Ethernet (EEE) allows network interfaces to save up to 90% of their nominal energy consumption when idling. There is an ample body of research that recommends the use of frame coalescing algorithms—that enter the low power mode as soon as there is no more traffic waiting to be sent, and delay the exit from this mode until there is an acceptable amount of traffic queued—to minimize energy usage while maintaining an acceptable performance. However, EEE capable hardware from several manufactures delays the entrance to the low power mode for a considerable amount of time (hysteresis). In this paper we augment existing EEE energy models to account for the hysteresis delay and show that, using the configuration ranges provided by manufacturers, most existing EEE networking devices are unable to obtain significant energy savings. To improve their energy efficiency, we propose to implement frame coalescing directly at traffic sources, before reaching the network interface. We also derive the optimum coalescing parameters to obtain a given target energy consumption at the EEE device when its configuration parameters are known in advance.Agencia Estatal de Investigación | Ref. TEC2017-85587-

POWAR: Power-Aware Routing in HPC Networks with On/Off Links

[EN] In order to save energy in HPC interconnection networks, one usual proposal is to switch idle links into a low-power mode after a certain time without any transmission, as IEEE Energy Efficient Ethernet standard proposes. Extending the low-power mode mechanism, we propose POWer-Aware Routing (POWAR), a simple power-aware routing and selection function for fat-tree and torus networks. POWAR adapts the amount of network links that can be used, taking into account the network load, and obtaining great energy savings in the network (55%-65%) and the entire system (9%-10%) with negligible performance overhead.This work has been supported by the Spanish MINECO and European Commission (FEDER funds) under project TIN2015-66972-C5-1-R. Francisco J. Andujar has been partially funded by the Spanish MICINN and by the ERDF program of the European Union: PCAS Project (TIN2017-88614-R), CAPAP-H6 (TIN2016-81840-REDT), and Junta de Castilla y Leon FEDER Grant VA082P17 (PROPHET Project).Andújar-Muñoz, FJ.; Coll, S.; Alonso Díaz, M.; López Rodríguez, PJ.; Martínez-Rubio, J. (2019). POWAR: Power-Aware Routing in HPC Networks with On/Off Links. ACM Transactions on Architecture and Code Optimization. 15(4):1-22. https://doi.org/10.1145/3293445S122154Abts, D., Marty, M. R., Wells, P. M., Klausler, P., & Liu, H. (2010). Energy proportional datacenter networks. Proceedings of the 37th annual international symposium on Computer architecture - ISCA ’10. doi:10.1145/1815961.1816004Adiga, N. R., Blumrich, M. A., Chen, D., Coteus, P., Gara, A., Giampapa, M. E., … Vranas, P. (2005). Blue Gene/L torus interconnection network. IBM Journal of Research and Development, 49(2.3), 265-276. doi:10.1147/rd.492.0265M. Alonso S. Coll J. M. Martínez V. Santonja and P. López. 2015. Power consumption management in fat-tree interconnection networks. Parallel Comput. 48 C (Oct. 2015) 59--80. 10.1016/j.parco.2015.03.007 M. Alonso S. Coll J. M. Martínez V. Santonja and P. López. 2015. Power consumption management in fat-tree interconnection networks. Parallel Comput. 48 C (Oct. 2015) 59--80. 10.1016/j.parco.2015.03.007Marina Alonso, Coll, S., Martínez, J.-M., Santonja, V., López, P., & Duato, J. (2010). Power saving in regular interconnection networks. Parallel Computing, 36(12), 696-712. doi:10.1016/j.parco.2010.08.003Bob Alverson Edwin Froese Larry Kaplan and Duncan Roweth. 2012. Cray XC series network. Cray Inc. White Paper WP-Aries01-1112 (2012). Bob Alverson Edwin Froese Larry Kaplan and Duncan Roweth. 2012. Cray XC series network. Cray Inc. White Paper WP-Aries01-1112 (2012).Anderson, T. E., Owicki, S. S., Saxe, J. B., & Thacker, C. P. (1993). High-speed switch scheduling for local-area networks. ACM Transactions on Computer Systems, 11(4), 319-352. doi:10.1145/161541.161736Andujar, F. J., Villar, J. A., Sanchez, J. L., Alfaro, F. J., & Escudero-Sahuquillo, J. (2015). VEF Traces: A Framework for Modelling MPI Traffic in Interconnection Network Simulators. 2015 IEEE International Conference on Cluster Computing. doi:10.1109/cluster.2015.141Barroso, L. A., & Hölzle, U. (2007). The Case for Energy-Proportional Computing. Computer, 40(12), 33-37. doi:10.1109/mc.2007.443Camacho, J., & Flich, J. (2011). HPC-Mesh: A Homogeneous Parallel Concentrated Mesh for Fault-Tolerance and Energy Savings. 2011 ACM/IEEE Seventh Symposium on Architectures for Networking and Communications Systems. doi:10.1109/ancs.2011.17Chen, D., Parker, J. J., Eisley, N. A., Heidelberger, P., Senger, R. M., Sugawara, Y., … Steinmacher-Burow, B. (2011). The IBM Blue Gene/Q interconnection network and message unit. Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis on - SC ’11. doi:10.1145/2063384.2063419Chen, L., & Pinkston, T. M. (2012). NoRD: Node-Router Decoupling for Effective Power-gating of On-Chip Routers. 2012 45th Annual IEEE/ACM International Symposium on Microarchitecture. doi:10.1109/micro.2012.33Christensen, K., Reviriego, P., Nordman, B., Bennett, M., Mostowfi, M., & Maestro, J. (2010). IEEE 802.3az: the road to energy efficient ethernet. IEEE Communications Magazine, 48(11), 50-56. doi:10.1109/mcom.2010.5621967Dally, & Seitz. (1987). Deadlock-Free Message Routing in Multiprocessor Interconnection Networks. IEEE Transactions on Computers, C-36(5), 547-553. doi:10.1109/tc.1987.1676939Das, R., Narayanasamy, S., Satpathy, S. K., & Dreslinski, R. G. (2013). Catnap. Proceedings of the 40th Annual International Symposium on Computer Architecture - ISCA ’13. doi:10.1145/2485922.2485950Derradji, S., Palfer-Sollier, T., Panziera, J.-P., Poudes, A., & Atos, F. W. (2015). The BXI Interconnect Architecture. 2015 IEEE 23rd Annual Symposium on High-Performance Interconnects. doi:10.1109/hoti.2015.15Jack Dongarra Hans W. Meuer and Erich Strohmaier. 2018. TOP500 Supercomputer Sites. Retrieved from https://www.top500.org. Jack Dongarra Hans W. Meuer and Erich Strohmaier. 2018. TOP500 Supercomputer Sites. Retrieved from https://www.top500.org.Duato, J. (1993). A new theory of deadlock-free adaptive routing in wormhole networks. IEEE Transactions on Parallel and Distributed Systems, 4(12), 1320-1331. doi:10.1109/71.250114José Duato Sudhakar Yalamanchili and Lionel Ni. 2003. Interconnection Networks. An Engineering Approach. Morgan Kaufmann Publishers Inc. San Francisco CA. José Duato Sudhakar Yalamanchili and Lionel Ni. 2003. Interconnection Networks. An Engineering Approach. Morgan Kaufmann Publishers Inc. San Francisco CA.GALGO 2017. GALGO—Albacete Research Institute of Informatics Supercomputer Center homepage. Retrieved from http://www.i3a.uclm.es/galgo. GALGO 2017. GALGO—Albacete Research Institute of Informatics Supercomputer Center homepage. Retrieved from http://www.i3a.uclm.es/galgo.Greenberg, A., Hamilton, J., Maltz, D. A., & Patel, P. (2008). The cost of a cloud. ACM SIGCOMM Computer Communication Review, 39(1), 68-73. doi:10.1145/1496091.1496103HPCC {n.d.}. HPC Challenge Benchmark. Retrieved from http://icl.cs.utk.edu/hpcc/index.html. HPCC {n.d.}. HPC Challenge Benchmark. Retrieved from http://icl.cs.utk.edu/hpcc/index.html.Hluchyj, M. G., & Karol, M. J. (1988). Queueing in high-performance packet switching. IEEE Journal on Selected Areas in Communications, 6(9), 1587-1597. doi:10.1109/49.12886Koibuchi, M., Otsuka, T., Hiroki Matsutani, & Amano, H. (2009). An on/off link activation method for low-power ethernet in PC clusters. 2009 IEEE International Symposium on Parallel & Distributed Processing. doi:10.1109/ipdps.2009.5161069Phillips, J. C., Braun, R., Wang, W., Gumbart, J., Tajkhorshid, E., Villa, E., … Schulten, K. (2005). Scalable molecular dynamics with NAMD. Journal of Computational Chemistry, 26(16), 1781-1802. doi:10.1002/jcc.20289Pronk, S., Páll, S., Schulz, R., Larsson, P., Bjelkmar, P., Apostolov, R., … Lindahl, E. (2013). GROMACS 4.5: a high-throughput and highly parallel open source molecular simulation toolkit. Bioinformatics, 29(7), 845-854. doi:10.1093/bioinformatics/btt055Reviriego, P., Hernandez, J., Larrabeiti, D., & Maestro, J. (2009). Performance evaluation of energy efficient ethernet. IEEE Communications Letters, 13(9), 697-699. doi:10.1109/lcomm.2009.090880K. P. Saravanan and P. Carpenter. 2018. PerfBound: Conserving energy with bounded overheads in on/off-based HPC interconnects. IEEE Trans. Comput. (2018) 1--1. 10.1109/TC.2018.2790394 K. P. Saravanan and P. Carpenter. 2018. PerfBound: Conserving energy with bounded overheads in on/off-based HPC interconnects. IEEE Trans. Comput. (2018) 1--1. 10.1109/TC.2018.2790394Saravanan, K. P., Carpenter, P. M., & Ramirez, A. (2013). Power/performance evaluation of energy efficient Ethernet (EEE) for High Performance Computing. 2013 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). doi:10.1109/ispass.2013.6557171Soteriou, V., & Li-Shiuan Peh. (s. f.). Dynamic power management for power optimization of interconnection networks using on/off links. 11th Symposium on High Performance Interconnects, 2003. Proceedings. doi:10.1109/conect.2003.1231472Totoni, E., Jain, N., & Kale, L. V. (2013). Toward Runtime Power Management of Exascale Networks by on/off Control of Links. 2013 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum. doi:10.1109/ipdpsw.2013.191VEF 2017. VEF traces homepage. Retrieved from http://www.i3a.info/VEFtraces. VEF 2017. VEF traces homepage. Retrieved from http://www.i3a.info/VEFtraces

Convergencia de tecnologías ópticas y Ethernet en LAN, MAN y SAN: nuevas arquitecturas, análisis de prestaciones y eficiencia energética

Mención Internacional en el título de doctorThe development of Information Technologies in the last decades, especially the last two, together with the introduction of computing devices to the mainstream consumer market, has had the logical consequence of the generalisation of the Internet access. The explosive development of the smartphone market has brought ubiquity to that generalisation, to the point that social interaction, content sharing and content production happens all the time. Social networks have all but increased that trend, maximising the diffusion of multimedia content: images, audio and video, which require high network capacities to be enjoyed quickly. This need for endless bandwidth and speed in information sharing brings challenges that affect mainly optical Metropolitan Area Networks (MANs) and Wide Area Networks (WANs). Furthermore, the wide spreading of Ethernet technologies has also brought the possibility to achieve economies of scale by either extending the reach of Ethernet Local Area Networks (LANs) to the MAN and WAN environment or even integrating them with Storage Area Networks (SANs). Finally, this generalisation of telecommunication technologies in every day life has as a consequence an important rise in energy consumption as well. Because of this, providing energy efficient strategies in networking is key to ensure the scalability of the whole Internet. In this thesis, the main technologies in all the fields mentioned above are reviewed, its core challenges identified and several contributions beyond the state of the art are suggested to improve today’s MANs andWANs. In the first contribution of this thesism, the integration between Metro Ethernet and Wavelength Division Multiplexion (WDM) optical transparent rings is explored by proposing an adaptation architecture to provide efficient broadcast and multicast. The second contribution explores the fusion between transparent WDM and OCDMA architectures to simplify medium access in a ring. Regarding SANs, the third contribution explores the challenges in SANs through the problems of Fibre Channel over Ethernet due to buffer design issues. In this contribution, analysis, design and validation with FCoE traces and simulation is provided to calculate buffer overflow probabilities in the absence of flow control mechanisms taking into account the bursty nature of SAN traffic. Finally, the fourth and last contribution addresses the problems of energy efficiency in Plastic Optical Fibres (POF), a new kind of optical fibre more suitable for transmission in vehicles and for home networking. This contribution suggests two packet coalescing strategies to further improve the energy effiency mechanisms in POFs.El desarrollo de las Tecnologías de la Información en las últimas décadas, especialmente las últimas dos, junto con la introducción de dispositivos informáticos al mercado de masas, ha tenido como consecuencia lógica la generalización del acceso a Internet. El explosivo desarrollo del mercado de teléfonos inteligentes ha añadido un factor de ubicuidad a tal generalización, al extremo de que la interacción social, la compartición y producción de contenidos sucede a cada instante. Las redes sociales no han hecho sino incrementar tal tendencia, maximizando la difusión de contenido multimedia: imágenes, audio y vídeo, los cuales requieren gran capacidad en las redes para poder obtenerse con rapidez. Esta necesidad de ancho de banda ilimitado y velocidad en la compartición de información trae consigo retos que afectan principalmente a las Redes de Área Metropolitana (Metropolitan Area Networks, MANs) y Redes de Área Extensa (Wide Area Networks, WANs). Además, la gran difusión de las tecnologías Ethernet ha traído la posibilidad de alcanzar economías de escala bien extendiendo el alcance de Ethernet más allá de las Redes de Área Local (Local Area Networks, LANs) al entorno de las MAN y las WAN o incluso integrándolas con Redes de Almacenamiento (Storage Area Networks, SANs). Finalmente, esta generalización de las tecnologías de la comunicación en la vida cotidiana tiene también como consecuencia un importante aumento en el consumo de energía. Por tanto, desarrollar estrategias de transmisión en red eficientes energéticamente es clave para asegurar la escalabilidad de Internet. En esta tesis, las principales tecnologías de todos los campos mencionados arriba serán estudiadas, sus más importantes retos identificados y se sugieren varias contribuciones más allá del actual estado del arte para mejorar las actuales MANs y WANs. En la primera contribución de esta tesis, se explora la integración entre Metro Ethernet y anillos ópticos transparentes por Multiplexión en Longitud de Onda (Wavelength Division Multiplex, WDM) mediante la proposición de una arquitectura de adaptación para permitir la difusión y multidifusión eficiente. La segunda contribución explora la fusión entre las arquitecturas transparentes WDM y arquitecturas por Accesso Dividido Múltiple por Códigos Ópticos (OCDMA) para simplificar el acceso en una red en anillo. En lo referente a las SANs, la tercera contribución explora los retos en SANs a través de los problemas de Fibre Channel sobre Ethernet debido a los problemas en el diseño de búferes. En esta contribución, se provee un análisis, diseño y validación con trazas FCoE para calcular las probabilidades de desbordamiento de buffer en ausencia de mecanismos de control de flujo teniendo en cuenta la naturaleza rafagosa del tráfico de SAN. Finalmente, la cuarta y última contribución aborda los problemas de eficiencia energética en Fibras Ópticas Plásticas (POF), una nueva variedad de fibra óptica más adecuada para la transmisión en vehículos y para entornos de red caseros. Esta contribución sugiere dos estrategias de agrupamiento de paquetes para mejorar los mecanismos de eficiencia energética en POFs.Programa Oficial de Posgrado en Ingeniería TelemáticaPresidente: Luca Valcarenghi.- Secretario: Ignacio Soto Campos.- Vocal: Bas Huiszoo

Optimal Design Strategies for Survivable Carrier Ethernet Networks

Ethernet technologies have evolved through enormous standardization efforts over the past two decades to achieve carrier-grade functionalities, leading to carrier Ethernet. Carrier Ethernet is expected to dominate next generation backbone networks due to its low-cost and simplicity. Ethernet's ability to provide carrier-grade Layer-2 protection switching with SONET/SDH-like fast restoration time is achieved by a new protection switching protocol, Ethernet Ring Protection (ERP). In this thesis, we address two important design aspects of carrier Ethernet networks, namely, survivable design of ERP-based Ethernet transport networks together with energy efficient network design. For the former, we address the problem of optimal resource allocation while designing logical ERP for deployment and model the combinatorially complex problem of joint Ring Protection Link (RPL) placements and ring hierarchies selection as an optimization problem. We develop several Mixed Integer Linear Programming (MILP) model to solve the problem optimally considering both single link failure and concurrent dual link failure scenarios. We also present a traffic engineering based ERP design approach and develop corresponding MILP design models for configuring either single or multiple logical ERP instances over one underlying physical ring. For the latter, we propose two novel architectures of energy efficient Ethernet switches using passive optical correlators for optical bypassing as well as using energy efficient Ethernet (EEE) ports for traffic aggregation and forwarding. We develop an optimal frame scheduling model for EEE ports to ensure minimal energy consumption by using packet coalescing and efficient scheduling