Split and Migrate: Resource-Driven Placement and Discovery of Microservices at the Edge

Microservices architectures combine the use of fine-grained and independently-scalable services with lightweight communication protocols, such as REST calls over HTTP. Microservices bring flexibility to the development and deployment of application back-ends in the cloud. Applications such as collaborative editing tools require frequent interactions between the front-end running on users\u27 machines and a back-end formed of multiple microservices. User-perceived latencies depend on their connection to microservices, but also on the interaction patterns between these services and their databases. Placing services at the edge of the network, closer to the users, is necessary to reduce user-perceived latencies. It is however difficult to decide on the placement of complete stateful microservices at one specific core or edge location without trading between a latency reduction for some users and a latency increase for the others. We present how to dynamically deploy microservices on a combination of core and edge resources to systematically reduce user-perceived latencies. Our approach enables the split of stateful microservices, and the placement of the resulting splits on appropriate core and edge sites. Koala, a decentralized and resource-driven service discovery middleware, enables REST calls to reach and use the appropriate split, with only minimal changes to a legacy microservices application. Locality awareness using network coordinates further enables to automatically migrate services split and follow the location of the users. We confirm the effectiveness of our approach with a full prototype and an application to ShareLatex, a microservices-based collaborative editing application

Small-world networks, distributed hash tables and the e-resource discovery problem

Resource discovery is one of the most important underpinning problems behind producing a scalable, robust and efficient global infrastructure for e-Science. A number of approaches to the resource discovery and management problem have been made in various computational grid environments and prototypes over the last decade. Computational resources and services in modern grid and cloud environments can be modelled as an overlay network superposed on the physical network structure of the Internet and World Wide Web. We discuss some of the main approaches to resource discovery in the context of the general properties of such an overlay network. We present some performance data and predicted properties based on algorithmic approaches such as distributed hash table resource discovery and management. We describe a prototype system and use its model to explore some of the known key graph aspects of the global resource overlay network - including small-world and scale-free properties

A Novel Locality Algorithm and Peer-to-Peer Communication Infrastructure for Optimizing Network Performance in Smart Microgrids

[EN] Peer-to-Peer (P2P) overlay communications networks have emerged as a new paradigm for implementing distributed services in microgrids due to their potential benefits: they are robust, scalable, fault-tolerant, and they can route messages even with a large number of nodes which are frequently entering or leaving from the network. However, current P2P systems have been mainly developed for file sharing or cycle sharing applications where the processes of searching and managing resources are not optimized. Locality algorithms have gained a lot of attention due to their potential to provide an optimized path to groups with similar interests for routing messages in order to get better network performance. This paper develops a fully functional decentralized communication architecture with a new P2P locality algorithm and a specific protocol for monitoring and control of microgrids. Experimental results show that the proposed locality algorithm reduces the number of lookup messages and the lookup delay time. Moreover, the proposed communication architecture heavily depends of the lookup used algorithm as well as the placement of the communication layers within the architecture. Experimental results will show that the proposed techniques meet the network requirements of smart microgrids even with a large number of nodes on stream.This work is supported by the Spanish Ministry of Economy and Competitiveness (MINECO) and the European Regional Development Fund (ERDF) under Grant ENE2015-64087-C2-2R. This work is supported by the Spanish Ministry of Economy and Competitiveness (MINECO) under BES-2013-064539.Marzal-Romeu, S.; González-Medina, R.; Salas-Puente, RA.; Figueres Amorós, E.; Garcerá, G. (2017). A Novel Locality Algorithm and Peer-to-Peer Communication Infrastructure for Optimizing Network Performance in Smart Microgrids. Energies. 10(9):1-25. https://doi.org/10.3390/en10091275S125109Khan, R. H., & Khan, J. Y. (2013). A comprehensive review of the application characteristics and traffic requirements of a smart grid communications network. Computer Networks, 57(3), 825-845. doi:10.1016/j.comnet.2012.11.002Dada, J. O. (2014). Towards understanding the benefits and challenges of Smart/Micro-Grid for electricity supply system in Nigeria. Renewable and Sustainable Energy Reviews, 38, 1003-1014. doi:10.1016/j.rser.2014.07.077Lidula, N. W. A., & Rajapakse, A. D. (2011). Microgrids research: A review of experimental microgrids and test systems. Renewable and Sustainable Energy Reviews, 15(1), 186-202. doi:10.1016/j.rser.2010.09.041Hussain, A., Arif, S. M., Aslam, M., & Shah, S. D. A. (2017). Optimal siting and sizing of tri-generation equipment for developing an autonomous community microgrid considering uncertainties. Sustainable Cities and Society, 32, 318-330. doi:10.1016/j.scs.2017.04.004Dehghanpour, K., Colson, C., & Nehrir, H. (2017). A Survey on Smart Agent-Based Microgrids for Resilient/Self-Healing Grids. Energies, 10(5), 620. doi:10.3390/en10050620Palizban, O., Kauhaniemi, K., & Guerrero, J. M. (2014). Microgrids in active network management – part II: System operation, power quality and protection. Renewable and Sustainable Energy Reviews, 36, 440-451. doi:10.1016/j.rser.2014.04.048Shi, W., Li, N., Chu, C.-C., & Gadh, R. (2017). Real-Time Energy Management in Microgrids. IEEE Transactions on Smart Grid, 8(1), 228-238. doi:10.1109/tsg.2015.2462294Deng, R., Yang, Z., Chow, M.-Y., & Chen, J. (2015). A Survey on Demand Response in Smart Grids: Mathematical Models and Approaches. IEEE Transactions on Industrial Informatics, 11(3), 570-582. doi:10.1109/tii.2015.2414719Moazami Goodarzi, H., & Kazemi, M. (2017). A Novel Optimal Control Method for Islanded Microgrids Based on Droop Control Using the ICA-GA Algorithm. Energies, 10(4), 485. doi:10.3390/en10040485Erol-Kantarci, M., Kantarci, B., & Mouftah, H. (2011). Reliable overlay topology design for the smart microgrid network. IEEE Network, 25(5), 38-43. doi:10.1109/mnet.2011.6033034Hassan Youssef, K. (2016). Optimal management of unbalanced smart microgrids for scheduled and unscheduled multiple transitions between grid-connected and islanded modes. Electric Power Systems Research, 141, 104-113. doi:10.1016/j.epsr.2016.07.015Giotitsas, C., Pazaitis, A., & Kostakis, V. (2015). A peer-to-peer approach to energy production. Technology in Society, 42, 28-38. doi:10.1016/j.techsoc.2015.02.002Kazmi, S. A. A., Shahzad, M. K., Khan, A. Z., & Shin, D. R. (2017). Smart Distribution Networks: A Review of Modern Distribution Concepts from a Planning Perspective. Energies, 10(4), 501. doi:10.3390/en10040501Werth, A., Andre, A., Kawamoto, D., Morita, T., Tajima, S., Tokoro, M., … Tanaka, K. (2018). Peer-to-Peer Control System for DC Microgrids. IEEE Transactions on Smart Grid, 9(4), 3667-3675. doi:10.1109/tsg.2016.2638462Deconinck, G., Vanthournout, K., Beitollahi, H., Qui, Z., Duan, R., Nauwelaers, B., … Belmans, R. (2008). A Robust Semantic Overlay Network for Microgrid Control Applications. Architecting Dependable Systems V, 101-123. doi:10.1007/978-3-540-85571-2_5Bandara, H. M. N. D., & Jayasumana, A. P. (2012). Collaborative applications over peer-to-peer systems–challenges and solutions. Peer-to-Peer Networking and Applications, 6(3), 257-276. doi:10.1007/s12083-012-0157-3Palizban, O., & Kauhaniemi, K. (2015). Hierarchical control structure in microgrids with distributed generation: Island and grid-connected mode. Renewable and Sustainable Energy Reviews, 44, 797-813. doi:10.1016/j.rser.2015.01.008Khatibzadeh, A., Besmi, M., Mahabadi, A., & Reza Haghifam, M. (2017). Multi-Agent-Based Controller for Voltage Enhancement in AC/DC Hybrid Microgrid Using Energy Storages. Energies, 10(2), 169. doi:10.3390/en10020169Planas, E., Gil-de-Muro, A., Andreu, J., Kortabarria, I., & Martínez de Alegría, I. (2013). General aspects, hierarchical controls and droop methods in microgrids: A review. Renewable and Sustainable Energy Reviews, 17, 147-159. doi:10.1016/j.rser.2012.09.032Olivares, D. E., Mehrizi-Sani, A., Etemadi, A. H., Canizares, C. A., Iravani, R., Kazerani, M., … Hatziargyriou, N. D. (2014). Trends in Microgrid Control. IEEE Transactions on Smart Grid, 5(4), 1905-1919. doi:10.1109/tsg.2013.2295514Vandoorn, T. L., Vasquez, J. C., De Kooning, J., Guerrero, J. M., & Vandevelde, L. (2013). Microgrids: Hierarchical Control and an Overview of the Control and Reserve Management Strategies. IEEE Industrial Electronics Magazine, 7(4), 42-55. doi:10.1109/mie.2013.2279306Zhou, B., Li, W., Chan, K. W., Cao, Y., Kuang, Y., Liu, X., & Wang, X. (2016). Smart home energy management systems: Concept, configurations, and scheduling strategies. Renewable and Sustainable Energy Reviews, 61, 30-40. doi:10.1016/j.rser.2016.03.047Ancillotti, E., Bruno, R., & Conti, M. (2013). The role of communication systems in smart grids: Architectures, technical solutions and research challenges. Computer Communications, 36(17-18), 1665-1697. doi:10.1016/j.comcom.2013.09.004Llaria, A., Terrasson, G., Curea, O., & Jiménez, J. (2016). Application of Wireless Sensor and Actuator Networks to Achieve Intelligent Microgrids: A Promising Approach towards a Global Smart Grid Deployment. Applied Sciences, 6(3), 61. doi:10.3390/app6030061Luna, A. C., Diaz, N. L., Graells, M., Vasquez, J. C., & Guerrero, J. M. (2016). Cooperative energy management for a cluster of households prosumers. IEEE Transactions on Consumer Electronics, 62(3), 235-242. doi:10.1109/tce.2016.7613189Gungor, V. C., Lu, B., & Hancke, G. P. (2010). Opportunities and Challenges of Wireless Sensor Networks in Smart Grid. IEEE Transactions on Industrial Electronics, 57(10), 3557-3564. doi:10.1109/tie.2009.2039455Zhao, C., He, J., Cheng, P., & Chen, J. (2017). Consensus-Based Energy Management in Smart Grid With Transmission Losses and Directed Communication. IEEE Transactions on Smart Grid, 8(5), 2049-2061. doi:10.1109/tsg.2015.2513772Lo, C.-H., & Ansari, N. (2013). Decentralized Controls and Communications for Autonomous Distribution Networks in Smart Grid. IEEE Transactions on Smart Grid, 4(1), 66-77. doi:10.1109/tsg.2012.2228282Li, C., Savaghebi, M., Guerrero, J., Coelho, E., & Vasquez, J. (2016). Operation Cost Minimization of Droop-Controlled AC Microgrids Using Multiagent-Based Distributed Control. Energies, 9(9), 717. doi:10.3390/en9090717Wu, X., Jiang, P., & Lu, J. (2014). Multiagent-Based Distributed Load Shedding for Islanded Microgrids. Energies, 7(9), 6050-6062. doi:10.3390/en7096050Kantamneni, A., Brown, L. E., Parker, G., & Weaver, W. W. (2015). Survey of multi-agent systems for microgrid control. Engineering Applications of Artificial Intelligence, 45, 192-203. doi:10.1016/j.engappai.2015.07.005Lopes, A. L., & Botelho, L. M. (2008). Improving Multi-Agent Based Resource Coordination in Peer-to-Peer Networks. Journal of Networks, 3(2). doi:10.4304/jnw.3.2.38-47Cameron, A., Stumptner, M., Nandagopal, N., Mayer, W., & Mansell, T. (2015). Rule-based peer-to-peer framework for decentralised real-time service oriented architectures. Science of Computer Programming, 97, 202-234. doi:10.1016/j.scico.2014.06.005Zhang, C., Wu, J., Cheng, M., Zhou, Y., & Long, C. (2016). A Bidding System for Peer-to-Peer Energy Trading in a Grid-connected Microgrid. Energy Procedia, 103, 147-152. doi:10.1016/j.egypro.2016.11.264Malatras, A. (2015). State-of-the-art survey on P2P overlay networks in pervasive computing environments. Journal of Network and Computer Applications, 55, 1-23. doi:10.1016/j.jnca.2015.04.014Eng Keong Lua, Crowcroft, J., Pias, M., Sharma, R., & Lim, S. (2005). A survey and comparison of peer-to-peer overlay network schemes. IEEE Communications Surveys & Tutorials, 7(2), 72-93. doi:10.1109/comst.2005.1610546Xu, J., Kumar, A., & Yu, X. (2004). On the Fundamental Tradeoffs Between Routing Table Size and Network Diameter in Peer-to-Peer Networks. IEEE Journal on Selected Areas in Communications, 22(1), 151-163. doi:10.1109/jsac.2003.818805Stoica, I., Morris, R., Karger, D., Kaashoek, M. F., & Balakrishnan, H. (2001). Chord. ACM SIGCOMM Computer Communication Review, 31(4), 149-160. doi:10.1145/964723.383071Rowstron, A., & Druschel, P. (2001). Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems. Lecture Notes in Computer Science, 329-350. doi:10.1007/3-540-45518-3_18Yuh-Jzer Joung, Li-Wei Yang, & Chien-Tse Fang. (2007). Keyword search in DHT-based peer-to-peer networks. IEEE Journal on Selected Areas in Communications, 25(1), 46-61. doi:10.1109/jsac.2007.070106Stoica, I., Morris, R., Liben-Nowell, D., Karger, D. R., Kaashoek, M. F., Dabek, F., & Balakrishnan, H. (2003). Chord: a scalable peer-to-peer lookup protocol for internet applications. IEEE/ACM Transactions on Networking, 11(1), 17-32. doi:10.1109/tnet.2002.808407Gottron, C., König, A., & Steinmetz, R. (2010). A Survey on Security in Mobile Peer-to-Peer Architectures—Overlay-Based vs. Underlay-Based Approaches. Future Internet, 2(4), 505-532. doi:10.3390/fi2040505Seyedi, Y., Karimi, H., & Guerrero, J. M. (2017). Centralized Disturbance Detection in Smart Microgrids With Noisy and Intermittent Synchrophasor Data. IEEE Transactions on Smart Grid, 8(6), 2775-2783. doi:10.1109/tsg.2016.2539947Youssef, T., Elsayed, A., & Mohammed, O. (2016). Data Distribution Service-Based Interoperability Framework for Smart Grid Testbed Infrastructure. Energies, 9(3), 150. doi:10.3390/en9030150Liu, X., Xia, H., & Chien, A. A. (2004). Validating and Scaling the MicroGrid: A Scientific Instrument for Grid Dynamics. Journal of Grid Computing, 2(2), 141-161. doi:10.1007/s10723-004-4200-3Kansal, P., & Bose, A. (2012). Bandwidth and Latency Requirements for Smart Transmission Grid Applications. IEEE Transactions on Smart Grid, 3(3), 1344-1352. doi:10.1109/tsg.2012.2197229Kuo, M.-T., & Lu, S.-D. (2013). Design and Implementation of Real-Time Intelligent Control and Structure Based on Multi-Agent Systems in Microgrids. Energies, 6(11), 6045-6059. doi:10.3390/en6116045Del Val, E., Rebollo, M., & Botti, V. (2012). Enhancing decentralized service discovery in open service-oriented multi-agent systems. Autonomous Agents and Multi-Agent Systems, 28(1), 1-30. doi:10.1007/s10458-012-9210-0Howell, S., Rezgui, Y., Hippolyte, J.-L., Jayan, B., & Li, H. (2017). Towards the next generation of smart grids: Semantic and holonic multi-agent management of distributed energy resources. Renewable and Sustainable Energy Reviews, 77, 193-214. doi:10.1016/j.rser.2017.03.107Frey, S., Diaconescu, A., Menga, D., & Demeure, I. (2015). A Generic Holonic Control Architecture for Heterogeneous Multiscale and Multiobjective Smart Microgrids. ACM Transactions on Autonomous and Adaptive Systems, 10(2), 1-21. doi:10.1145/2700326Miers, C., Simplicio, M., Gallo, D., Carvalho, T., Bressan, G., Souza, V., … Damola, A. (2010). A Taxonomy for Locality Algorithms on Peer-to-Peer Networks. IEEE Latin America Transactions, 8(4), 323-331. doi:10.1109/tla.2010.5595121Porsinger, T., Janik, P., Leonowicz, Z., & Gono, R. (2017). Modelling and Optimization in Microgrids. Energies, 10(4), 523. doi:10.3390/en10040523Ali, M., Zakariya, M., Asif, M., & Ullah, A. (2012). TCP/IP Based Intelligent Load Management System in Micro-Grids Network Using MATLAB/Simulink. Energy and Power Engineering, 04(04), 283-289. doi:10.4236/epe.2012.44038Shin, I.-J., Song, B.-K., & Eom, D.-S. (2017). International Electronical Committee (IEC) 61850 Mapping with Constrained Application Protocol (CoAP) in Smart Grids Based European Telecommunications Standard Institute Machine-to-Machine (M2M) Environment. Energies, 10(3), 393. doi:10.3390/en10030393Loh, P. C., Li, D., Chai, Y. K., & Blaabjerg, F. (2013). Autonomous Operation of Hybrid Microgrid With AC and DC Subgrids. IEEE Transactions on Power Electronics, 28(5), 2214-2223. doi:10.1109/tpel.2012.2214792Overlay networks for smart gridshttp://users.atlantis.ugent.be/cdvelder/papers/2013/wauters2013sgv.pdfEugster, P. T., Felber, P. A., Guerraoui, R., & Kermarrec, A.-M. (2003). The many faces of publish/subscribe. ACM Computing Surveys, 35(2), 114-131. doi:10.1145/857076.857078Ali, I. (2012). High-speed Peer-to-peer Communication based Protection Scheme Implementation and Testing in Laboratory. International Journal of Computer Applications, 38(4), 16-24. doi:10.5120/4596-6793Yoo, B.-K., Yang, S.-H., Yang, H.-S., Kim, W.-Y., Jeong, Y.-S., Han, B.-M., & Jang, K.-S. (2011). Communication Architecture of the IEC 61850-based Micro Grid System. Journal of Electrical Engineering and Technology, 6(5), 605-612. doi:10.5370/jeet.2011.6.5.605Dou, X., Quan, X., Wu, Z., Hu, M., Yang, K., Yuan, J., & Wang, M. (2014). Hybrid Multi-Agent Control in Microgrids: Framework, Models and Implementations Based on IEC 61850. Energies, 8(1), 31-58. doi:10.3390/en801003

Proof-of-Concept Application - Annual Report Year 1

In this document the Cat-COVITE Application for use in the CATNETS Project is introduced and motivated. Furthermore an introduction to the catallactic middleware and Web Services Agreement (WS-Agreement) concepts is given as a basis for the future work. Requirements for the application of Cat-COVITE with in catallactic systems are analysed. Finally the integration of the Cat-COVITE application and the catallactic middleware is described. --Grid Computing

Analysis of current middleware used in peer-to-peer and grid implementations for enhancement by catallactic mechanisms

This deliverable describes the work done in task 3.1, Middleware analysis: Analysis of current middleware used in peer-to-peer and grid implementations for enhancement by catallactic mechanisms from work package 3, Middleware Implementation. The document is divided in four parts: The introduction with application scenarios and middleware requirements, Catnets middleware architecture, evaluation of existing middleware toolkits, and conclusions. -- Die Arbeit definiert Anforderungen an Grid und Peer-to-Peer Middleware Architekturen und analysiert diese auf ihre Eignung für die prototypische Umsetzung der Katallaxie. Eine Middleware-Architektur für die Umsetzung der Katallaxie in Application Layer Netzwerken wird vorgestellt.Grid Computing