The increase of the computing capacity of IoT devices and the appearance of lightweight machine learning frameworks have led to the situation that machine learning can nowadays be run in IoT applications at the network edge. There is an opportunity to implement machine learning algorithms with the more and more computationally powerful edge nodes and using the ever increasing amount of local data coming from nearby sensors. For this purpose, federated learning becomes a promising machine learning approach, where a machine learning model is trained by various nodes using their local data. For performing practical federated learning experiments, we have built a testbed deployed within a wireless city mesh network with geographically distributed low capacity devices. We describe the testbed implementation and show its potential to experimentally study federated learning protocols and algorithms in real edge environments.This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 871582 — NGIatlantic.eu and was partially supported by the Spanish Government under contracts PID2019-106774RBC21, PCI2019-111850-2 (DiPET CHIST-ERA), PCI2019-111851-2 (LeadingEdge CHIST-ERA). The work of C.-H. Liu was supported in part by the U.S. National Science Foundation (NSF) under Award CNS-2006453 and in part by Mississippi State University under Grant ORED 253551-060702. The work of L. Wei is supported in part by the U.S. National Science Foundation (#2006612 and #2150486).Peer ReviewedPostprint (published version

Freitag, Fèlix

Liu, Chun-Hung

Selimi, Mennan

Vilchez Blanco, Pedro

Wei, Lu

English

UPCommons. Portal del coneixement obert de la UPC

Poster: Testbed in Wireless City Mesh Network with Applicationto Federated Learning ExperimentsFelix FreitagPedro Vilchez{felix.freitag,pedro.vilchez}@upc.eduComputer Architecture Department, UPC BarcelonaTechSpainLu WeiDepartment of Computer ScienceTexas Tech UniversityLubbock TX, USAluwei@ttu.eduChun-Hung LiuDepartment of Electrical and Computer EngineeringMississippi State UniversityStarkville MS, USAchliu@ece.msstate.eduMennan SelimiMax van der Stoel InstituteSouth East European UniversityNorth Macedoniam.selimi@seeu.edu.mkABSTRACTThe increase of the computing capacity of IoT devices and the ap-pearance of lightweight machine learning frameworks have ledto the situation that machine learning can nowadays be run inIoT applications at the network edge. There is an opportunity toimplement machine learning algorithms with the more and morecomputationally powerful edge nodes and using the ever increas-ing amount of local data coming from nearby sensors. For thispurpose, federated learning becomes a promising machine learningapproach, where a machine learning model is trained by variousnodes using their local data. For performing practical federatedlearning experiments, we have built a testbed deployed within awireless city mesh network with geographically distributed lowcapacity devices. We describe the testbed implementation and showits potential to experimentally study federated learning protocolsand algorithms in real edge environments.KEYWORDStestbeds, federated learning, IoT devicesACM Reference Format:Felix Freitag, Pedro Vilchez, Lu Wei, Chun-Hung Liu, and Mennan Selimi.2021. Poster: Testbed in Wireless City Mesh Network with Application toFederated Learning Experiments. In 11th International Conference on theInternet of Things (IoT ’21), November 8–12, 2021, St.Gallen, Switzerland.ACM,New York, NY, USA, 4 pages. https://doi.org/10.1145/3494322.34943531 INTRODUCTIONThere is a strong tendency to run machine learning (ML) models onever smaller computing devices. For inference with trained models,simple recognition tasks can nowadays be performed even withdevices as tiny as microcontroller boards [6].This work is licensed under a Creative Commons Attribution International4.0 License.IoT ’21, November 8–12, 2021, St.Gallen, Switzerland© 2021 Copyright held by the owner/author(s).ACM ISBN 978-1-4503-8566-4/21/11.https://doi.org/10.1145/3494322.3494353The training of the machine learning models, however, is a morecompute-intensive task than the inference, and imposes challengesfor being done with low capacity devices. GPUs instead of CPUsmay be used for an improved training performance. However, inlow-capacity devices, such as mini-PCs or single-board-computers(SBCs), often GPUs are either not available or the GPU is not sup-ported by the machine learning binary. As a consequence, duringthe training process there is a high load on the CPU. Training takesmuch longer time in these low-capacity devices.Federated learning (FL) is a recent approach to training machinelearning models in a distributed fashion in which many client nodesparticipate in the model training [8]. Different to the centralizedtraining approach requiring all training data available at a singlelocation, in FL many client nodes train a machine learning modelonly with their local data. One important advantage is the privacypreservation of the data, since only the trained machine learningmodel is shared, but the local data does not have to leave the node.For low-capacity IoT devices, the split of the training effortamong several nodes done in federated learning is another im-portant feature, since potentially less energy and storage is neededat each node to perform the model training. Furthermore, in FL thelocal dataset at each client node is considered to be smaller, as itmay consist, for instance, only of the sensor data collected at thisnode.There are a few works that address the resource consumptionof federated learning in IoT scenarios. In [9], a federated learn-ing framework for the IoT is proposed. The specific applicationis to detect anomalies in the network traffic of IoT devices. Thescenario is motivated by cybersecurity requirements, in which thecommunication overhead of a centralized over-the-cloud approachis unfeasible. For the anomaly detection, a deep autoencoder modelis trained with federated learning at each node. The evaluationis performed on real devices, namely Raspberry Pi model 4 andNVIDIA Jetson Nano. While the evaluation focuses more on theaccuracy of the detection but not on the resource consumption as-pects, it was observed that only a small fraction of the 4GB memoryof the Raspberry Pi was used.In thework of Y. Gao et al [4] an empirical evaluation of two state-of-art machine learning techniques is presented, namely the split169IoT ’21, November 8–12, 2021, St.Gallen, Switzerland Freitag and Vilchez, et al.neural networks (SplitNN) and federated learning. For an end-to-end evaluation, a variety of datasets, different model architectures,multiple clients, and various performance metrics were consid-ered. Model training was done on Raspberry Pi devices, where theCPU consumption, memory usage, communication overhead, andtraining time was measured. From the experiments, the authorsconclude that FL overall outperforms SplitNN, because of the lowercommunication overhead.In [7], an adaptive federated learning approach is proposed. Thefocus is on the global aggregation frequency parameter. Specifically,a control algorithm is proposed to determine in real time after howmany local training epochs at a node the model data is sent back tothe aggregator node. This approach is different to the typically usedfixed global aggregation frequency. The evaluation is performed bysimulations and with a few experiments in real nodes consisting ofthree Raspberry Pi and two laptop computers.It can be seen that there are several works that propose waysfor reducing the computational resource consumption of federatedlearning, ranging from changes of the machine leaning model train-ing up to off-loading of specific services to other platforms. There is,however, a lack of studies that test federated learning in real envi-ronments. In this paper we present a testbed with which a federatedlearning implementation can be practically experimented. For thiswe deployed low-capacity devices as testbed nodes in a wireless citymesh network. With initial experiments we show the potential ofthe testbed to conduct experimental research on federated learningprotocols and algorithms.The main contributions of the paper are:• We present a testbed consisting of low capacity devices de-ployed within a wireless city mesh network for the evalua-tion of distributed services such as federated learning.• We present initial performance measurements of federatedlearning using the testbed nodes.2 GUIFISANTS WIRELESS CITY MESHNETWORKThe choice for the environment to deploy the testbed was theGuifiSants1 wireless mesh network. One one hand, our universitycampus is already connected to GuifiSants through a few Guifinodes. On the other hand, this wireless mesh network offers a realand dynamic edge network environment, which provides a realisticsetting for the targeted practical federated learning experiments.GuifiSants began to operate in 2009 in the Sants quarter of thecity of Barcelona, Spain. Technically it is based on the mesh net-work technology developed in the Quick Mesh Project (QMP)2. TheGuifiSants network is an urban mesh network and is a subset of thelarger Guifi.net community network3. The Guifi.net network is acommunication infrastructure of more then 30, 000 interconnected(wired and wireless) heterogeneous networking devices (routers),belonging to the thousands of community network members [2].At the time of writing, the GuifiSants wireless mesh networkhas around 54 routers4. There are four gateways (i.e., proxies)1http://sants.guifi.net/2http://qmp.cat3guifi.net: Commons Telecommunication Network Open Free Neutralhttp://guifi.net/4http://dsg.ac.upc.edu/qmpsu/index.phpFigure 1: Nodes and connctivity of the GuifiSants wirelessmesh network in the city of Barcelona.distributed in the network that connect GuifiSants to the rest ofGuifi.net and the Internet (proxies highlighted in Figure 1). A de-tailed study of the GuifiSants mesh network can be found in [3].Edge computing services started around 2015 within Guifi.net[1].For this community network members installed edge devices attheir premises. These devices typically host the device owner’sspecific application services, sometimes shared with other com-munity network members, but also community-oriented servicesuch as network monitoring. Within this context, the testbed nodesrepresent additional edge computing infrastructure, connected torouters within GuifiSants.3 TESTBED3.1 HardwareAs hardware for the testbed nodes we use Pc Engine’s APU2 andMinix mini-PCs. This hardware is already successfully used by somecommunity network members of Guifi.net to host local services forwhich the devices have sufficient computing capacity. Furthermore,these devices are low-energy consuming and users are willing to runthem in a 24/7 modes. Another reason for the choice of these devicesas testbed nodes was to be conformant to the typical computinghardware used in Guifi.net, such that any findings and insightsgained from the testbed preparation and experimentation can beeasily transferred and applied in Guifi.net.The Minix mini-PC is the NEO Z83-4 with Intel Atom x5-Z8350processor and 4GB DDR3 RAM5 . The PC Engine’s model used isthe APU2 with an AMD Embedded G series GX-412TC processorand 4 GB DDR3 RAM6 . In both type of devices we install Debian 10Buster. It is worth mentioning that within these two device types,the Minix NEO Z83-4APU2 integrates an Intel GPU while the PcEngine’s APU2 does not have any GPU.5https://minix.com.hk/products/neo-z83-4-pro6https://pcengines.ch/apu2e4.htm170Poster: Testbed in Wireless City Mesh Network with Application to Federated Learning Experiments IoT ’21, November 8–12, 2021, St.Gallen, SwitzerlandMinix Cluster C6e208APU Cluster C6e104BellvitgePisuergaMinix Cluster C6e104Testbed nodes in GuifiSantsCluster nodeSingle node  Minix deviceManacorMesh NetworkFigure 2: Testbed nodes within GuifiSants deployed in the city of Barcelona.3.2 Testbed DeploymentSeveral of the previously described APU and Minix nodes weredeployed as testbed nodes in the GuifiSants wireless city meshnetwork. The devices are connected to a Guifi node, i.e., a routertypically located at the rooftop7. The router assigns a static Guifi-wide routable IP address to the testbed device, such that the devicecan host services and connect to other nodes of the Guifi communitynetwork.Figure 2 illustrates the testbed in the area of Barcelona. Thetestbed has two types of nodes, clusters of devices and individualnodes. The clusters of devices (named Minix cluster C6e104, Minixcluster C6e208 and APU cluster C6e104) consist of several devicesat the university campus, which are connected through Guifi.netnodes to the GuifiSants wireless mesh network. The individualnodes of the testbed are at different locations in GuifiSants, either atthe premises of a community network member or at a municipalityinstallation, where a Guifi.net node could be installed.Table 1 shows the testbed heterogeneity in terms of comput-ing devices (APU2 and Minix) and network (e.g., connectivity) atthe different location, where low bandwidth corresponds to linkswith typically less than 10 MBits/sec to other nodes, medium band-width to links with typically 10-100 MBits/sec bandwidth to othernodes, and high bandwidth to links which can have more than 100MBits/sec.3.3 ExperimentationFor doing experimentation with a distributed federated learning net-work, we use a Python-based implementation. The implementationconsists of twomajor components, the code for the FL client and the7An example of such routers are the Ubiquity NanoStation. https://www.ui.com/airmax/nanostationm/Table 1: Testbed nodes.Location name Devices Node connectivyMinix cluster C6e104 4 Minix high bandwidthMinix cluster C6e208 5 Minix medium bandwidthAPU cluster C6e104 7 APU2 high bandwidthPisuerga 1 Minix low bandwidthBellvitge 1 Minix medium bandwidthManacor 1 Minix low bandwidthFL server. Additional information on the software implementationof the federated learning network can be found in [5].For running the federated learning components on the distributednodes of the testbed, we create Docker images for the server andclient. Since the processor of the Minix and APU2 devices are both64 bit x86 instruction set-based, it could be expected that a singleDocker image could be used for both types of devices. We noticed,however, that the default Tensorflow binary (at time of writing thev2.5), which is integrated in the client of the federated learningsoftware for the machine learning part, leveraged the AdvancedVector Extensions (AVX) instruction set, which is supported by theCPU of the APU2 device, but not by the CPU on the Minix device.For this reason, a specific binary of Tensorflow was used to buildthe Docker images for the Minix devices.We use the nodes from the testbed to run an experiment. Thegoal is to measure the resource consumption of federated learningdevices and observe the effect of the heterogeneity of the wirelessmesh network on federated learning training. We use an APU2device from the APU cluster C6e104 to run the FL server. As FLclients we chose one Minix device from the Minix cluster C6e104,which has a high bandwidth to the FL server, and the Minx device171IoT ’21, November 8–12, 2021, St.Gallen, Switzerland Freitag and Vilchez, et al.from the Pisuerga node, which has a low bandwidth to the FL server.Both clients train locally a CNN machine learning model with thesame number of samples and same number of epochs. We conductthree training rounds during a duration of around 3 minutes. Wemeasure the resource consumption at the FL server in terms of CPU,memory and bandwidth consumed. In Figure 3 it can be seen thatthe FL server has peaks for the CPU consumption when sending themachine learning model to the clients, as well as when receivingthe model from the two clients. The 2nd, 5th and 8th peak in theCPU consumption corresponds to the model reception of the highbandwidth client, the 3rd, 6th and 9th peak to the low bandwidthclient. The memory consumption of the server during the trainingprocess is fairly stable. Figure 4 shows the peaks of bandwidthconsumption corresponding to the sending and reception of modelsby the server. It can be observed that it takes longer to send themodel from the FL server to the low bandwidth FL client, and alsothat the server receives the model later from this client.Figure 3: CPU andmemory consumption of federated learn-ing server in three training rounds with two clients.Figure 4: Bandwidth consumption of federated learningserver in three training rounds with two clients.4 CONCLUSIONS AND OUTLOOKThis paper presented a testbed of distributed low-capacity devicesdeployed geographically within a wireless city mesh network, tar-geting at the experimentation with distributed services running atedge nodes. Specifically, the usage of the testbed for a federatedlearning experiment was exemplified. The testbed has been shownto be a tool that can help researchers to better understand the practi-cal design of federated learning applications for edge environmentsof the IoT, and experimentally study federated learning protocolsand algorithms in real settings.Our next steps will focus on extending the usage of this testbedfor assessing different design options of federated learning withregards to their resource usages patterns. There seems to be anopportunity for more adaptive federated learning designs, whichwill be critical to making this machine learning training paradigmsuitable for resource-constraint IoT environments.ACKNOWLEDGMENTSThis project has received funding from the European Union’s Hori-zon 2020 research and innovation programme under grant agree-ment No 871582 — NGIatlantic.eu and was partially supportedby the Spanish Government under contracts PID2019-106774RB-C21, PCI2019-111850-2 (DiPET CHIST-ERA), PCI2019-111851-2(LeadingEdge CHIST-ERA). The work of C.-H. Liu was supportedin part by the U.S. National Science Foundation (NSF) under AwardCNS-2006453 and in part by Mississippi State University underGrant ORED 253551-060702. The work of L. Wei is supported in partby the U.S. National Science Foundation (#2006612 and #2150486).REFERENCES[1] Roger Baig, Felix Freitag, and Leandro Navarro. 2018. Cloudy in guifi.net: Estab-lishing and sustaining a community cloud as open commons. Future GenerationComputer Systems 87 (2018), 868–887. https://doi.org/10.1016/j.future.2017.12.017[2] Roger Baig, Ramon Roca, Felix Freitag, and Leandro Navarro. 2015. Guifi.Net, aCrowdsourced Network Infrastructure Held in Common. Comput. Netw. 90, C(Oct. 2015), 150–165. https://doi.org/10.1016/j.comnet.2015.07.009[3] Llorenç Cerdà-Alabern, Axel Neumann, and Pau Escrich. 2013. ExperimentalEvaluation of a Wireless Community Mesh Network. In Proceedings of the 16thACM International Conference on Modeling, Analysis and Simulation of Wirelessand Mobile Systems (Barcelona, Spain) (MSWiM ’13). ACM, New York, NY, USA,23–30. https://doi.org/10.1145/2507924.2507960[4] Y. Gao, M. Kim, S. Abuadbba, Y. Kim, C. Thapa, K. Kim, S. A. Camtep, H. Kim, andS. Nepal. 2020. End-to-End Evaluation of Federated Learning and Split Learningfor Internet of Things. In 2020 International Symposium on Reliable DistributedSystems (SRDS). 91–100. https://doi.org/10.1109/SRDS51746.2020.00017[5] Eduardo Yanez Parareda. 2021. Federated learning network: Training distributedmachine learning models with the federated learning paradigm. (2021). http://hdl.handle.net/10609/126546[6] Fouad Sakr, Francesco Bellotti, Riccardo Berta, and Alessandro De Gloria. 2020.Machine Learning on Mainstream Microcontrollers. Sensors 20, 9 (May 2020), 2638.https://doi.org/10.3390/s20092638[7] Shiqiang Wang, Tiffany Tuor, Theodoros Salonidis, Kin K. Leung, ChristianMakaya, Ting He, and Kevin Chan. 2019. Adaptive Federated Learning in ResourceConstrained Edge Computing Systems. IEEE Journal on Selected Areas in Commu-nications 37, 6 (2019), 1205–1221. https://doi.org/10.1109/JSAC.2019.2904348[8] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. 2019. Federated MachineLearning: Concept and Applications. ACM Trans. Intell. Syst. Technol. 10, 2, Article12 (Jan. 2019), 19 pages. https://doi.org/10.1145/3298981[9] Tuo Zhang, Chaoyang He, Tianhao Ma, Mark Ma, and Salman Avestimehr. 2021.Federated Learning for Internet of Things: A Federated Learning Framework forOn-device Anomaly Data Detection. arXiv:2106.07976 [cs.LG]172

Poster: Testbed in wireless city mesh network with application to federated learning experiments

https://upcommons.upc.edu/bitstream/2117/367302/1/3494322.3494353.pdf

Poster: Testbed in wireless city mesh network with application to federated learning experiments

Abstract

Similar works

Full text

Available Versions

UPCommons. Portal del coneixement obert de la UPC