Classically the data produced by Big Data applications is transferred through the access and core networks to be processed in data centers where the resulting data is stored. In this work we investigate improving the energy efficiency of transporting Big Data by processing the data in processing nodes of limited processing and storage capacity along its journey through the core network to the data center. The amount of data transported over the core network will be significantly reduced each time the data is processed therefore we refer to such a network as an Energy Efficient Tapered Data Network. The results of a Mixed Integer linear Programming (MILP), developed to optimize the processing of Big Data in the Energy Efficient Tapered Data Networks, show significant reduction in network power consumption up to 76%

Al-Salim, AMA

El-Gorashi, TEH

Elmirghani, JMH

Lawey, AQ

White Rose Research Online

This is a repository copy of Energy Efficient Tapered Data Networks for Big Data Processing in IP/WDM Networks.White Rose Research Online URL for this paper:http://eprints.whiterose.ac.uk/88009/Version: Accepted VersionProceedings Paper:Al-Salim, AMA, Lawey, AQ, El-Gorashi, TEH et al. (1 more author) (2015) Energy Efficient Tapered Data Networks for Big Data Processing in IP/WDM Networks. In: 2015 17th International Conference on Transparent Optical Networks (ICTON). 17th International Conference on Transparent Optical Networks (ICTON 2015), 05-09 Jul 2015, Budapest, Hungary. IEEE . ISBN 978-1-4673-7880-2 https://doi.org/10.1109/ICTON.2015.7193536eprints@whiterose.ac.ukhttps://eprints.whiterose.ac.uk/Reuse Unless indicated otherwise, fulltext items are protected by copyright with all rights reserved. The copyright exception in section 29 of the Copyright, Designs and Patents Act 1988 allows the making of a single copy solely for the purpose of non-commercial research or private study within the limits of fair dealing. The publisher or other rights-holder may allow further reproduction and re-use of this version - refer to the White Rose Research Online record for this item. Where records identify the publisher as the copyright holder, users can verify any specific terms of use on the publisher’s website. Takedown If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request. Energy Efficient Tapered Data Networks for Big Data Processingin IP/WDM NetworksAli M. Al-Salim, Ahmed Q. Lawey, Taisir El-Gorashi and Jaafar M. H. ElmirghaniSchool of Electronic and Electrical Engineering, University of Leeds, LS2 9JT, United KingdomE-mail: {elamas, elaql, t.e.h.elgorashi, j.m.h.elmirghani}@leeds.ac.ukABSTRACTClassically the data produced by Big Data applications is transferred through the access and core networks to beprocessed in data centers where the resulting data is stored. In this work we investigate improving the energyefficiency of transporting Big Data by processing the data in processing nodes of limited processing and storagecapacity along its journey through the core network to the data center. The amount of data transported over the corenetwork will be significantly reduced each time the data is processed therefore we refer to such a network as anEnergy Efficient Tapered Data Network. The results of a Mixed Integer linear Programming (MILP), developed tooptimize the processing of Big Data in the Energy Efficient Tapered Data Networks, show significant reduction innetwork power consumption up to 76%.Keywords: Big Data, IP/WDM, energy consumption, routing, distributed processing.1. INTRODUCTIONThe amount of data created by humanity from the dawn civilisation to 2003 is estimated to be five Exabyte.Currently, the same amount of data is created every two days [1], resulting in what we refer to as Big Data. Seriousconcerns are raised about the power consumption of processing and transporting such huge amounts of data [2].Efficient management of processing and transmitting is needed to avoid overwhelming data centres and networkswith such large volumes of data. The authors in [3] introduced a processing system for executing a sequence ofMapReduce jobs on Geo-distributed data centers where job processing is optimized according to time cost orfinancial cost. Recently, the authors in [4] designed an efficient dynamic bulk data transfer service in geo-distributeddata canters and engineered its design and algorithms based on Software Defined Network (SDN) architecture. In [5]the authors study the minimization of overall cost for Big Data placement, processing, and movement across geo-distributed data centers. In [6] the authors introduced a framework for designing energy efficient cloud computingservices over IP/WDM core networks.In this work we investigate limiting the impact of Big Data on communication networks by processing data at thesource and/or along its journey through the core network to the data centre. The amount of data transported overthe core network is significantly reduced each time the data is processed. Therefore we refer to such a network as anEnergy Efficient Tapered Data Network. We developed a Mixed Integer Linear Programming (MILP) model tooptimize the processing of Big Data in energy efficient tapered data networks so that the network powerconsumption is reduced.The remainder of this paper is organized as follows: In Section 2, we further explain the concept of EnergyEfficient Tapered Data Networks and briefly describe the MILP model developed. In Section 3, we present andanalyze the results of the model. We conclude the paper in Section 4.2. ENERGY EFFICIENT BIG DATA PROCESSINGThe concept of Energy Efficient Tapered Data Networks is illustrated in Fig.1. Fig.1 (a) shows the classical approachof proccessing Big Data where all the processing is performed in a data centre. In energy efficient tapered datanetworks, see Fig.1 (b), Processing Nodes (PNs) are attached to the ISP network centers that host the IP/WDM nodes.A PN comprises a set of servers, storage, and internal LAN switches and routers and can process data generated inthe local core node or data generated by other nodes on its way to the data centre. The capacity of a PN is limited bythe avaliable space to build the the PN inside the network center. We refere to the raw unprocessed data as chunkswhile the knowledge extracted from the chunks after processing is refered to as info. Note in Fig.1 (b) that the dataproduced by sources can either be chunks or info. The latter is the case if the source has processing capability. Thistype of source is referred to as a Source PN and depending on its processing capability and input data volume mayproduce info only or info and chunks. The processing capability placed at intermediate nodes is refererd to asIntermediate PN.The data chunks can be processed locally, in the PN attached to the core node where the data is generated oralternatively they can be processed in the PNs attached to any the intermediate nodes on the way to the central datacenter. The processing of a data chunk can also be performed in multiple stages in muiltple PNs or in one or morePNs and in a data centre. The decision of where to process the data chunks depends on the avaliable resources inPNs.We developed an MILP model to optimize the processing Big Data in bypass IP/WDM networks (see [7]-[15] fordetails of MILP in IP over WDM networks) so that the total power consumption is minimized. However, in thispaper we only give a brief introduction to the model due to paper length limitation. The total power consumption isgiven as: 劇剣建欠健鶏剣拳結堅系剣券嫌憲兼喧建件剣券 噺【券結建拳剣堅倦喧剣拳結堅潔剣券嫌憲兼喧建件剣券 髪 鶏軽嫌喧剣拳結堅潔剣券嫌憲兼喧建件 髪 経系嫌喧剣拳結堅潔剣券嫌憲兼喧建件剣券岫な岻The total network power consumption is composed of the power consumption of router ports, transponders,EDFAs, regenerators, and optical switches. On the other hand, the PNs and DCs power consumption is composedof the power consumption of the servers, storages, switches and routers.The PNs are of limited processing and storage capacity while the internal LAN switches and routers capacity isassumed to be large enough to avoid blocking of chunks. The data centres are assumed to be unlimited in capacity.The model is subject to a number of consraints including constraints to ensure that the processing and storagecapacity of the PNs is not exceeded.Fig. 1(a) Delivery of Big Data to be Processed and Stored in Data Centers.Fig. 1(b) Energy Efficient Tapered Data Networks.3. RESULTS AND EVALUATIONThe MILP model is evaluated on the NSFNET network, shown in Fig. 2, under the IP/WDM bypass approach. TheNSFNET network consists of 14 nodes with 21 bidirectional links [10]. We consider a network that contains twodata centres hosted in Nodes 6 and 10. All the other nodes are connected to a PN. The processing and storagecapacities of PNs are assumed to be randomly distributed between 60-120 GHz and 10-70 PB, respectively. Themodel is solved using the CPLEX 12.5 solver on a PC with Intel® Core™ i5-2500 CPU @ 3.3 GHz and 8 GBRAM. Table 1 summarises the input parameters to the model [6], [9]-[15].Fig. 2 The NSFNET Network.Table (1) Input data for the modelPNs processing capacity 60 GHz - 140 GHzPNs storage capacity 10 PB - 70 PBServer processing capacity 4 GHzMax server power consumption 300 W [6]Idle server power consumption 200 W [6]Energy per bit of the switch 11.875 W/Gbps [11]-[15]Energy per bit of the router 7.727 W/Gbps [11]-[15]Storage power consumption 0.008 W/Gb [11]-[15]IP/WDM router power consumption 440 W [6]IP/WDM transponder power consumption 148 W [6]IP/WDM optical switch power consumption 85 W [6]IP/WDM EFDA power consumption 52 W [6]Wavelength bit rate 40 Gbps [11]-[15]Number of wavelengths per fiber 32 [11]-[15]We evaluated our new proposed approach in a scenario where we optimize the processing of the Aggregate FastBig Data type of the very well-known Hadoop-MapReduce Big Data platform [16]-[18]. The input data is obtainedfrom the 6 month MapReduce Facebook trace file in [19]. Table (2) shows the size of the input chunk and the outputinfo for the Aggregate Fast. For simplicity, we assume that each chunk is transmitted at a data rate equal to its size persecond.Table (2): Chunk Size, Info Size, and CPU WorkloadData type Chunk size in Gb Info size in Gb CPU Workload in GHzAggregate Fast 1840 3.68 1Fig. 3 compares the energy efficient tapered data network to the classical approach of processing Big Data interms of network power consumption. We consider a number of chunks generated per node varying between 25-200chunks. For 25 and 50 chunks, the capacity of PNs is enough to perform all the processing locally reducing thenetwork traffic to the minimum, i.e. only infos are sent to the data centres. As the number of chunks increases theresources of some of the PNs become insufficient to completely process local chunks. Chunks that are not fullyprocessed in local PNs will be processed in intermediate PNs and/or data centres and therefore the network trafficincreases. The PNs of the considered network can completely/partially process up to 100 chunks. Any more chunkswill be completely processed in the data centre. The power consumption savings achieved by PNs compared toprocessing in data centres varies between 95% for 25 chunks and 85% for 100 chunks.Fig. 4 further explains the behaviour of the energy efficient data tapered network. It shows the utilization of theprocessing capacity percentage at the different PNs. For up to 50 chunks, the utilization of the different PNs is lessthan 100% which means, as discussed above, that the resources are enough to process all chunks locally. As thenumber of chunks increases the resources of some PNs are depleted, i.e. utilization reaches 100%. For a number ofchunks between 75 and 100, the PNs resources are enough to process all chunks either locally or intermediately andno unprocessed chunks are forwarded to the data centres and therefore the data centres resources utilization is stillidle.The resources of PNs are fully utilized when the number of chunks exceeds 100 and any extra chunk thereafter isoptimally transmitted to the data centers to be processed there. Note that in Fig. 4, utilization is referenced to thecapability of a PN node. Therefore, nodes 6 and 10 which have data centres show utilizations above 100%. Alsonote that a data centre has unlimited capacity in our case as mentioned earlier. Therefore as Fig. 4 shows the largestdata centre capability needed is just under 8 PNs processing capacity.Fig. 3 Classical Big Data Network Power Consumption and Energy Efficient Data Tapered Network Power Consumption withDifferent Number of Chunks per Node for Data Type Aggregate Fast* Data center processing capacity is 8 times PNs processing capacity.Fig. 4 Processing Capacity Utilization with Different Number of Chunks per Node for the Data Type Aggregate Fast05010015020025025 50 75 100 125 150 175 200NetworkpowerconsumptioninMWNumber of chunks per nodeClassical Big Data NetworkEnergy Efficient Data Tapered NetworkCONCLUSIONSIn this work we introduced a MILP model to study Energy Efficient Tapered Data Network for Big Data Processingin IP/WDM Networks. We proposed to distribute the processing jobs of data chunks generated at a given sourcenode among different nodes connecting the source to the intended data centers. Any node that is capable of storingand processing a chunk is called a Processing Node (PN). The bigger the storage and processing capacity of PNs themore is the data that will be optimally processed either locally or in the intermediate PNs, and the less is the dataprocessed at data centers, and the less the traffic in the network (ie between PNs and data centers). Having a givenPN that exhausts its processing and/or storage capacity, additional chunks are forwarded to the best PN that is notfully utilized yet. When all PNs reach their maximum capacity, no more chunks can be processed on route and all thechunks are forwarded to the data centers. The results show that an average network power saving of 76% is achievedwhen our Tapered Data Networks concept is used to optimize the processing location of Big Data chunks indifferent stages along the path from the source to the data centers.ACKNOWLEDGEMENTThe authors would like to acknowledge funding from the Engineering and Physical Sciences Research Council(EPSRC), INTERNET (EP/H040536/1) and STAR (EP/K016873/1). Mr. Ali Al-Salim would like to acknowledgeThe Higher Committee for Education Development in Iraq (HCED/IRAQ) for funding his scholarship.REFERENCES[1] http://europa.eu/rapid/press-release_SPEECH-13-893_en.htm[2] http://www.lightwaveonline.com/articles/print/volume-31/issue-6/features/impact-of-big-data-transport-on-network-innovation.html[3] C. Jayalath, J. Stephen, and P. Eugster, From the Cloud to the Atmosphere: Running MapReduce across DataCenters, IEEE Trans. Comput., vol. 63, no. 1, pp. 7487, Jan. 2014.[4] Y. Wu, Z. Zhang, C. Wu, C. Guo, Z. Li, and F. C. M. Lau, Orchestrating Bulk Data Transfers across Geo-Distributed Datacenters, IEEE Trans. Cloud Comput., vol. 7161, no. c, pp. 11, 2015.[5] L. Gu, D. Zeng, P. Li, and S. Guo, Cost Minimization for Big Data Processing in Geo-Distributed DataCenters, IEEE Trans. Emerg. Top. Comput., vol. 6750, no. c, pp. 11, 2014.[6] A. Q. Lawey, T. E. H. El-Gorashi, and J. M. H. Elmirghani, Distributed Energy Efficient Clouds Over CoreNetworks, J. Light. Technol., vol. 32, no. 7, pp. 12611281, 2014.[7] X. Xiang, C. Lin, F. Chen, and X. Chen, Greening Geo-Distributed Data Centers by Joint Optimization ofRequest Routing and Virtual Machine Scheduling, pp. 110, 2014.[8] B. Rajagopalan, J. Luciani, D. Awduche, B. Cain, B. Jamoussi, and D. Saha, "IP over optical networks: Aframework," Work in Progress, 2003.[9] X. Dong, T. El-Gorashi, and J. M. H. Elmirghani, Green IP Over WDM Networks With Data Centers, J.Light. Technol., vol. 29, no. 12, pp. 18611880, Jun. 2011.[10] G. Shen and R. Tucker, Energy-minimized design for IP over WDM networks, IEEE /OSA J. Opt.Commun. Netw., vol. 1, no. 1, pp. 176186, Jun. 2009.[11] Dong, X., El-Gorashi, T.E.H. and Elmirghani, J.M.H., IP Over WDM Networks Employing RenewableEnergy Sources, IEEE/OSA Journal of Lightwave Technology, vol. 27, No. 1, pp. 3-14, 2011.[12] Dong, X., El-Gorashi, T.E.H. and Elmirghani, J.M.H., On the Energy Efficiency of Physical TopologyDesign for IP over WDM Networks, IEEE/OSA Journal of Lightwave Technology, vol. 30, pp.1931-1942,2012.[13] Osman, N. I., El-Gorashi, T.E.H., Krug, L. and Elmirghani, Energy-Efficient Future High-Definition TV,IEEE/OSA Journal of Lightwave Technology, vol. 32, No. 13, pp. 2364  2381, 2014.[14] Lawey, A., El-Gorashi, T.E.H. and Elmirghani, J.M.H., BitTorrent Content Distribution in OpticalNetworks, IEEE/OSA Journal of Lightwave Technology, vol. 32, No. 21, pp. 3607  3623, 2014.[15] Nonde, L., El-Gorashi, T.E.H. and Elmirghani, J.M.H., Energy Efficient Virtual Network Embedding forCloud Networks, IEEE/OSA Journal of Lightwave Technology, vol. 33, No. 9, pp. 1828-1849, 2015.[16] J. Baliga, R.W. A. Ayre, K. Hinton, and R. S. Tucker, Green cloud com- puting: Balancing energy inprocessing, storage, and transport, Proc. IEEE, vol. 99, no. 1, pp. 149167, 2011.[17] K. Singh, Hadoop: Addressing Challenges of Big Data, pp. 686689, 2014.[18] N. Tiwari, Scheduling and Energy Efficiency Improvement Techniques for Hadoop Map-reduce: State of Artand Directions for Future Research.[19] Y. Chen, A. Ganapathi, R. Griffith, and R. Katz, The Case for Evaluating MapReduce Performance UsingWorkload Suites, 2011 IEEE 19th Annu. Int. Symp. Model. Anal. Simul. Comput. Telecommun. Syst., pp.390399, Jul. 2011.

Energy Efficient Tapered Data Networks for Big Data Processing in IP/WDM Networks

Ali M. Al-Salim

Ahmed Q. Lawey

Taisir El-Gorashi

Jaafar M. H. Elmirghani

Crossref

Energy Efficient Tapered Data Networks for Big Data processing in IP/WDM networks

https://eprints.whiterose.ac.uk/88009/1/Energy%20Efficient%20Tapered%20Data%20Networks%20for%20Big%20Data%20Processing%20in%20IPWDM%20Networks%20submitted.pdf

Energy Efficient Tapered Data Networks for Big Data Processing in IP/WDM Networks

Abstract

Similar works

Full text

Available Versions

White Rose Research Online

Crossref