Tight Lower Bounds for Greedy Routing in Higher-Dimensional Small-World Grids

We consider Kleinberg's celebrated small world graph model (Kleinberg, 2000), in which a D-dimensional grid {0,...,n-1}^D is augmented with a constant number of additional unidirectional edges leaving each node. These long range edges are determined at random according to a probability distribution (the augmenting distribution), which is the same for each node. Kleinberg suggested using the inverse D-th power distribution, in which node v is the long range contact of node u with a probability proportional to ||u-v||^(-D). He showed that such an augmenting distribution allows to route a message efficiently in the resulting random graph: The greedy algorithm, where in each intermediate node the message travels over a link that brings the message closest to the target w.r.t. the Manhattan distance, finds a path of expected length O(log^2 n) between any two nodes. In this paper we prove that greedy routing does not perform asymptotically better for any uniform and isotropic augmenting distribution, i.e., the probability that node u has a particular long range contact v is independent of the labels of u and v and only a function of ||u-v||. In order to obtain the result, we introduce a novel proof technique: We define a budget game, in which a token travels over a game board, while the player manages a "probability budget". In each round, the player bets part of her remaining probability budget on step sizes. A step size is chosen at random according to a probability distribution of the player's bet. The token then makes progress as determined by the chosen step size, while some of the player's bet is removed from her probability budget. We prove a tight lower bound for such a budget game, and then obtain a lower bound for greedy routing in the D-dimensional grid by a reduction

On the Fundamental Tradeoffs between Routing Table Size and Network Diameter in Peer-to-Peer Networks

In this work, we study a fundamental tradeoff issue in designing dynamic hash table (DHT) in peer-to-peer networks: the size of the routing table v.s. the network diameter. It was observed in Ratnasamy et al. that existing DHT schemes either (a) have a routing table of size of O(log₂n) and network diameter of O(log₂n), or (b) have a routing table of size d and network diameter of O(n [superscript 1/d]). They asked whether this represents the best asymptotic "state-efficiency" tradeoffs. Our first major result is to show that there are routing algorithms which achieve better asymptotic tradeoffs. However, such algorithms all cause severe congestion on certain network nodes, which is undesirable in a P2P network. We then define the notion of "congestion-free" and conjecture that the above tradeoffs are asymptotically optimal for a congestion-free network. Though we are not able to prove (or disprove) this conjecture in full generality, our rigorous formulation of the problem and techniques introduced in proving slightly weaker results serve as the basis for further exploration of this problem. Our second major result is to prove that, if the routing algorithms are symmetric, the aforementioned tradeoffs are asymptotically optimal. Furthermore, for symmetric algorithms, we find that O(log₂n) is a magic threshold point for routing table size as follows. The "congestion" factor dominates the "reachability" factor in determining the minimum network diameter when the routing table size is asymptotically smaller than or equal to O(log₂ n), and it is the other way around when the routing table size is asymptotically larger than O(log₂n). Our third and final major result is to study the exact (instead of asymptotic) optimal tradeoffs. We propose a new routing algorithm that reduces the routing table size and the network diameter of Chord both by 21.4% without introducing any other overhead, based on a novel number-theoretical technique

X.: On the fundamental tradeoffs between routing table size and network diameter in peer-to-peer networks

In this work, we study a fundamental tradeoff issue in designing distributed hash table (DHT) in peer-to-peer networks: the size of the routing table v.s. the network diameter. It was observed by Ratnasamy et al. that existing DHT schemes either (a) have a routing table of size and network diameter of, or (b) have a routing table of size and network diameter of. They asked whether this represents the best asymptotic “state-efficiency ” tradeoffs. Our first major result is to show that there are straightforward routing algorithms which achieve better asymptotic tradeoffs. However, such algorithms all cause severe congestion on certain network nodes, which is undesirable in a P2P network. We then rigorously define the notion of “congestion ” and conjecture that the above tradeoffs are asymptotically optimal for a congestion-free network. We show that the answer to this conjecture is negative in the strict sense. However, the answer becomes positive if the routing algorithm is required to eliminate congestion in a “natural ” way by being uniform. Our second major result is to prove that the aforementioned tradeoffs are asymptotically optimal for uniform algorithms. Furthermore, for uniform algorithms, we find that the routing table size of is a magic threshold point that separates two different “state-efficiency ” regions. Our third result is to study the exact (instead of asymptotic) optimal tradeoffs for uniform algorithms. We propose a new routing algorithm that reduces the routing table size and the network diameter of Chord both by 21.4 % without introducing any other protocol overhead, based on a novel number-theoretical technique. Our fourth and final result is to present Ulysses, a congestion-free non-uniform algorithm that achieves a better asymptotic “state-efficiency ” tradeoff than existing schemes in the probabilistic sense, even under dynamic node joins/leaves. I

Designs and Analyses in Structured Peer-To-Peer Systems

Peer-to-Peer (P2P) computing is a recent hot topic in the areas of networking and distributed systems. Work on P2P computing was triggered by a number of ad-hoc systems that made the concept popular. Later, academic research efforts started to investigate P2P computing issues based on scientific principles. Some of that research produced a number of structured P2P systems that were collectively referred to by the term "Distributed Hash Tables" (DHTs). However, the research occurred in a diversified way leading to the appearance of similar concepts yet lacking a common perspective and not heavily analyzed. In this thesis we present a number of papers representing our research results in the area of structured P2P systems grouped as two sets labeled respectively "Designs" and "Analyses". The contribution of the first set of papers is as follows. First, we present the princi- ple of distributed k-ary search and argue that it serves as a framework for most of the recent P2P systems known as DHTs. That is, given this framework, understanding existing DHT systems is done simply by seeing how they are instances of that frame- work. We argue that by perceiving systems as instances of that framework, one can optimize some of them. We illustrate that by applying the framework to the Chord system, one of the most established DHT systems. Second, we show how the frame- work helps in the design of P2P algorithms by two examples: (a) The DKS(n; k; f) system which is a system designed from the beginning on the principles of distributed k-ary search. (b) Two broadcast algorithms that take advantage of the distributed k-ary search tree. The contribution of the second set of papers is as follows. We account for two approaches that we used to evaluate the performance of a particular class of DHTs, namely the one adopting periodic stabilization for topology maintenance. The first approach was of an intrinsic empirical nature. In this approach, we tried to perceive a DHT as a physical system and account for its properties in a size-independent manner. The second approach was of a more analytical nature. In this approach, we applied the technique of Master Equations, which is a widely used technique in the analysis of natural systems. The application of the technique lead to a highly accurate description of the behavior of structured overlays. Additionally, the thesis contains a primer on structured P2P systems that tries to capture the main ideas prevailing in the field

An Emergent Architecture for Scaling Decentralized Communication Systems (DCS)

With recent technological advancements now accelerating the mobile and wireless Internet solution space, a ubiquitous computing Internet is well within the research and industrial community's design reach - a decentralized system design, which is not solely driven by static physical models and sound engineering principals, but more dynamically, perhaps sub-optimally at initial deployment and socially-influenced in its evolution. To complement today's Internet system, this thesis proposes a Decentralized Communication System (DCS) architecture with the following characteristics: flat physical topologies with numerous compute oriented and communication intensive nodes in the network with many of these nodes operating in multiple functional roles; self-organizing virtual structures formed through alternative mobility scenarios and capable of serving ad hoc networking formations; emergent operations and control with limited dependency on centralized control and management administration. Today, decentralized systems are not commercially scalable or viable for broad adoption in the same way we have to come to rely on the Internet or telephony systems. The premise in this thesis is that DCS can reach high levels of resilience, usefulness, scale that the industry has come to experience with traditional centralized systems by exploiting the following properties: (i.) network density and topological diversity; (ii.) self-organization and emergent attributes; (iii.) cooperative and dynamic infrastructure; and (iv.) node role diversity. This thesis delivers key contributions towards advancing the current state of the art in decentralized systems. First, we present the vision and a conceptual framework for DCS. Second, the thesis demonstrates that such a framework and concept architecture is feasible by prototyping a DCS platform that exhibits the above properties or minimally, demonstrates that these properties are feasible through prototyped network services. Third, this work expands on an alternative approach to network clustering using hierarchical virtual clusters (HVC) to facilitate self-organizing network structures. With increasing network complexity, decentralized systems can generally lead to unreliable and irregular service quality, especially given unpredictable node mobility and traffic dynamics. The HVC framework is an architectural strategy to address organizational disorder associated with traditional decentralized systems. The proposed HVC architecture along with the associated promotional methodology organizes distributed control and management services by leveraging alternative organizational models (e.g., peer-to-peer (P2P), centralized or tiered) in hierarchical and virtual fashion. Through simulation and analytical modeling, we demonstrate HVC efficiencies in DCS structural scalability and resilience by comparing static and dynamic HVC node configurations against traditional physical configurations based on P2P, centralized or tiered structures. Next, an emergent management architecture for DCS exploiting HVC for self-organization, introduces emergence as an operational approach to scaling DCS services for state management and policy control. In this thesis, emergence scales in hierarchical fashion using virtual clustering to create multiple tiers of local and global separation for aggregation, distribution and network control. Emergence is an architectural objective, which HVC introduces into the proposed self-management design for scaling and stability purposes. Since HVC expands the clustering model hierarchically and virtually, a clusterhead (CH) node, positioned as a proxy for a specific cluster or grouped DCS nodes, can also operate in a micro-capacity as a peer member of an organized cluster in a higher tier. As the HVC promotional process continues through the hierarchy, each tier of the hierarchy exhibits emergent behavior. With HVC as the self-organizing structural framework, a multi-tiered, emergent architecture enables the decentralized management strategy to improve scaling objectives that traditionally challenge decentralized systems. The HVC organizational concept and the emergence properties align with and the view of the human brain's neocortex layering structure of sensory storage, prediction and intelligence. It is the position in this thesis, that for DCS to scale and maintain broad stability, network control and management must strive towards an emergent or natural approach. While today's models for network control and management have proven to lack scalability and responsiveness based on pure centralized models, it is unlikely that singular organizational models can withstand the operational complexities associated with DCS. In this work, we integrate emergence and learning-based methods in a cooperative computing manner towards realizing DCS self-management. However, unlike many existing work in these areas which break down with increased network complexity and dynamics, the proposed HVC framework is utilized to offset these issues through effective separation, aggregation and asynchronous processing of both distributed state and policy. Using modeling techniques, we demonstrate that such architecture is feasible and can improve the operational robustness of DCS. The modeling emphasis focuses on demonstrating the operational advantages of an HVC-based organizational strategy for emergent management services (i.e., reachability, availability or performance). By integrating the two approaches, the DCS architecture forms a scalable system to address the challenges associated with traditional decentralized systems. The hypothesis is that the emergent management system architecture will improve the operational scaling properties of DCS-based applications and services. Additionally, we demonstrate structural flexibility of HVC as an underlying service infrastructure to build and deploy DCS applications and layered services. The modeling results demonstrate that an HVC-based emergent management and control system operationally outperforms traditional structural organizational models. In summary, this thesis brings together the above contributions towards delivering a scalable, decentralized system for Internet mobile computing and communications

A Novel Locality Algorithm and Peer-to-Peer Communication Infrastructure for Optimizing Network Performance in Smart Microgrids

[EN] Peer-to-Peer (P2P) overlay communications networks have emerged as a new paradigm for implementing distributed services in microgrids due to their potential benefits: they are robust, scalable, fault-tolerant, and they can route messages even with a large number of nodes which are frequently entering or leaving from the network. However, current P2P systems have been mainly developed for file sharing or cycle sharing applications where the processes of searching and managing resources are not optimized. Locality algorithms have gained a lot of attention due to their potential to provide an optimized path to groups with similar interests for routing messages in order to get better network performance. This paper develops a fully functional decentralized communication architecture with a new P2P locality algorithm and a specific protocol for monitoring and control of microgrids. Experimental results show that the proposed locality algorithm reduces the number of lookup messages and the lookup delay time. Moreover, the proposed communication architecture heavily depends of the lookup used algorithm as well as the placement of the communication layers within the architecture. Experimental results will show that the proposed techniques meet the network requirements of smart microgrids even with a large number of nodes on stream.This work is supported by the Spanish Ministry of Economy and Competitiveness (MINECO) and the European Regional Development Fund (ERDF) under Grant ENE2015-64087-C2-2R. This work is supported by the Spanish Ministry of Economy and Competitiveness (MINECO) under BES-2013-064539.Marzal-Romeu, S.; González-Medina, R.; Salas-Puente, RA.; Figueres Amorós, E.; Garcerá, G. (2017). A Novel Locality Algorithm and Peer-to-Peer Communication Infrastructure for Optimizing Network Performance in Smart Microgrids. Energies. 10(9):1-25. https://doi.org/10.3390/en10091275S125109Khan, R. H., & Khan, J. Y. (2013). A comprehensive review of the application characteristics and traffic requirements of a smart grid communications network. Computer Networks, 57(3), 825-845. doi:10.1016/j.comnet.2012.11.002Dada, J. O. (2014). Towards understanding the benefits and challenges of Smart/Micro-Grid for electricity supply system in Nigeria. Renewable and Sustainable Energy Reviews, 38, 1003-1014. doi:10.1016/j.rser.2014.07.077Lidula, N. W. A., & Rajapakse, A. D. (2011). Microgrids research: A review of experimental microgrids and test systems. Renewable and Sustainable Energy Reviews, 15(1), 186-202. doi:10.1016/j.rser.2010.09.041Hussain, A., Arif, S. M., Aslam, M., & Shah, S. D. A. (2017). Optimal siting and sizing of tri-generation equipment for developing an autonomous community microgrid considering uncertainties. Sustainable Cities and Society, 32, 318-330. doi:10.1016/j.scs.2017.04.004Dehghanpour, K., Colson, C., & Nehrir, H. (2017). A Survey on Smart Agent-Based Microgrids for Resilient/Self-Healing Grids. Energies, 10(5), 620. doi:10.3390/en10050620Palizban, O., Kauhaniemi, K., & Guerrero, J. M. (2014). Microgrids in active network management – part II: System operation, power quality and protection. Renewable and Sustainable Energy Reviews, 36, 440-451. doi:10.1016/j.rser.2014.04.048Shi, W., Li, N., Chu, C.-C., & Gadh, R. (2017). Real-Time Energy Management in Microgrids. IEEE Transactions on Smart Grid, 8(1), 228-238. doi:10.1109/tsg.2015.2462294Deng, R., Yang, Z., Chow, M.-Y., & Chen, J. (2015). A Survey on Demand Response in Smart Grids: Mathematical Models and Approaches. IEEE Transactions on Industrial Informatics, 11(3), 570-582. doi:10.1109/tii.2015.2414719Moazami Goodarzi, H., & Kazemi, M. (2017). A Novel Optimal Control Method for Islanded Microgrids Based on Droop Control Using the ICA-GA Algorithm. Energies, 10(4), 485. doi:10.3390/en10040485Erol-Kantarci, M., Kantarci, B., & Mouftah, H. (2011). Reliable overlay topology design for the smart microgrid network. IEEE Network, 25(5), 38-43. doi:10.1109/mnet.2011.6033034Hassan Youssef, K. (2016). Optimal management of unbalanced smart microgrids for scheduled and unscheduled multiple transitions between grid-connected and islanded modes. Electric Power Systems Research, 141, 104-113. doi:10.1016/j.epsr.2016.07.015Giotitsas, C., Pazaitis, A., & Kostakis, V. (2015). A peer-to-peer approach to energy production. Technology in Society, 42, 28-38. doi:10.1016/j.techsoc.2015.02.002Kazmi, S. A. A., Shahzad, M. K., Khan, A. Z., & Shin, D. R. (2017). Smart Distribution Networks: A Review of Modern Distribution Concepts from a Planning Perspective. Energies, 10(4), 501. doi:10.3390/en10040501Werth, A., Andre, A., Kawamoto, D., Morita, T., Tajima, S., Tokoro, M., … Tanaka, K. (2018). Peer-to-Peer Control System for DC Microgrids. IEEE Transactions on Smart Grid, 9(4), 3667-3675. doi:10.1109/tsg.2016.2638462Deconinck, G., Vanthournout, K., Beitollahi, H., Qui, Z., Duan, R., Nauwelaers, B., … Belmans, R. (2008). A Robust Semantic Overlay Network for Microgrid Control Applications. Architecting Dependable Systems V, 101-123. doi:10.1007/978-3-540-85571-2_5Bandara, H. M. N. D., & Jayasumana, A. P. (2012). Collaborative applications over peer-to-peer systems–challenges and solutions. Peer-to-Peer Networking and Applications, 6(3), 257-276. doi:10.1007/s12083-012-0157-3Palizban, O., & Kauhaniemi, K. (2015). Hierarchical control structure in microgrids with distributed generation: Island and grid-connected mode. Renewable and Sustainable Energy Reviews, 44, 797-813. doi:10.1016/j.rser.2015.01.008Khatibzadeh, A., Besmi, M., Mahabadi, A., & Reza Haghifam, M. (2017). Multi-Agent-Based Controller for Voltage Enhancement in AC/DC Hybrid Microgrid Using Energy Storages. Energies, 10(2), 169. doi:10.3390/en10020169Planas, E., Gil-de-Muro, A., Andreu, J., Kortabarria, I., & Martínez de Alegría, I. (2013). General aspects, hierarchical controls and droop methods in microgrids: A review. Renewable and Sustainable Energy Reviews, 17, 147-159. doi:10.1016/j.rser.2012.09.032Olivares, D. E., Mehrizi-Sani, A., Etemadi, A. H., Canizares, C. A., Iravani, R., Kazerani, M., … Hatziargyriou, N. D. (2014). Trends in Microgrid Control. IEEE Transactions on Smart Grid, 5(4), 1905-1919. doi:10.1109/tsg.2013.2295514Vandoorn, T. L., Vasquez, J. C., De Kooning, J., Guerrero, J. M., & Vandevelde, L. (2013). Microgrids: Hierarchical Control and an Overview of the Control and Reserve Management Strategies. IEEE Industrial Electronics Magazine, 7(4), 42-55. doi:10.1109/mie.2013.2279306Zhou, B., Li, W., Chan, K. W., Cao, Y., Kuang, Y., Liu, X., & Wang, X. (2016). Smart home energy management systems: Concept, configurations, and scheduling strategies. Renewable and Sustainable Energy Reviews, 61, 30-40. doi:10.1016/j.rser.2016.03.047Ancillotti, E., Bruno, R., & Conti, M. (2013). The role of communication systems in smart grids: Architectures, technical solutions and research challenges. Computer Communications, 36(17-18), 1665-1697. doi:10.1016/j.comcom.2013.09.004Llaria, A., Terrasson, G., Curea, O., & Jiménez, J. (2016). Application of Wireless Sensor and Actuator Networks to Achieve Intelligent Microgrids: A Promising Approach towards a Global Smart Grid Deployment. Applied Sciences, 6(3), 61. doi:10.3390/app6030061Luna, A. C., Diaz, N. L., Graells, M., Vasquez, J. C., & Guerrero, J. M. (2016). Cooperative energy management for a cluster of households prosumers. IEEE Transactions on Consumer Electronics, 62(3), 235-242. doi:10.1109/tce.2016.7613189Gungor, V. C., Lu, B., & Hancke, G. P. (2010). Opportunities and Challenges of Wireless Sensor Networks in Smart Grid. IEEE Transactions on Industrial Electronics, 57(10), 3557-3564. doi:10.1109/tie.2009.2039455Zhao, C., He, J., Cheng, P., & Chen, J. (2017). Consensus-Based Energy Management in Smart Grid With Transmission Losses and Directed Communication. IEEE Transactions on Smart Grid, 8(5), 2049-2061. doi:10.1109/tsg.2015.2513772Lo, C.-H., & Ansari, N. (2013). Decentralized Controls and Communications for Autonomous Distribution Networks in Smart Grid. IEEE Transactions on Smart Grid, 4(1), 66-77. doi:10.1109/tsg.2012.2228282Li, C., Savaghebi, M., Guerrero, J., Coelho, E., & Vasquez, J. (2016). Operation Cost Minimization of Droop-Controlled AC Microgrids Using Multiagent-Based Distributed Control. Energies, 9(9), 717. doi:10.3390/en9090717Wu, X., Jiang, P., & Lu, J. (2014). Multiagent-Based Distributed Load Shedding for Islanded Microgrids. Energies, 7(9), 6050-6062. doi:10.3390/en7096050Kantamneni, A., Brown, L. E., Parker, G., & Weaver, W. W. (2015). Survey of multi-agent systems for microgrid control. Engineering Applications of Artificial Intelligence, 45, 192-203. doi:10.1016/j.engappai.2015.07.005Lopes, A. L., & Botelho, L. M. (2008). Improving Multi-Agent Based Resource Coordination in Peer-to-Peer Networks. Journal of Networks, 3(2). doi:10.4304/jnw.3.2.38-47Cameron, A., Stumptner, M., Nandagopal, N., Mayer, W., & Mansell, T. (2015). Rule-based peer-to-peer framework for decentralised real-time service oriented architectures. Science of Computer Programming, 97, 202-234. doi:10.1016/j.scico.2014.06.005Zhang, C., Wu, J., Cheng, M., Zhou, Y., & Long, C. (2016). A Bidding System for Peer-to-Peer Energy Trading in a Grid-connected Microgrid. Energy Procedia, 103, 147-152. doi:10.1016/j.egypro.2016.11.264Malatras, A. (2015). State-of-the-art survey on P2P overlay networks in pervasive computing environments. Journal of Network and Computer Applications, 55, 1-23. doi:10.1016/j.jnca.2015.04.014Eng Keong Lua, Crowcroft, J., Pias, M., Sharma, R., & Lim, S. (2005). A survey and comparison of peer-to-peer overlay network schemes. IEEE Communications Surveys & Tutorials, 7(2), 72-93. doi:10.1109/comst.2005.1610546Xu, J., Kumar, A., & Yu, X. (2004). On the Fundamental Tradeoffs Between Routing Table Size and Network Diameter in Peer-to-Peer Networks. IEEE Journal on Selected Areas in Communications, 22(1), 151-163. doi:10.1109/jsac.2003.818805Stoica, I., Morris, R., Karger, D., Kaashoek, M. F., & Balakrishnan, H. (2001). Chord. ACM SIGCOMM Computer Communication Review, 31(4), 149-160. doi:10.1145/964723.383071Rowstron, A., & Druschel, P. (2001). Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems. Lecture Notes in Computer Science, 329-350. doi:10.1007/3-540-45518-3_18Yuh-Jzer Joung, Li-Wei Yang, & Chien-Tse Fang. (2007). Keyword search in DHT-based peer-to-peer networks. IEEE Journal on Selected Areas in Communications, 25(1), 46-61. doi:10.1109/jsac.2007.070106Stoica, I., Morris, R., Liben-Nowell, D., Karger, D. R., Kaashoek, M. F., Dabek, F., & Balakrishnan, H. (2003). Chord: a scalable peer-to-peer lookup protocol for internet applications. IEEE/ACM Transactions on Networking, 11(1), 17-32. doi:10.1109/tnet.2002.808407Gottron, C., König, A., & Steinmetz, R. (2010). A Survey on Security in Mobile Peer-to-Peer Architectures—Overlay-Based vs. Underlay-Based Approaches. Future Internet, 2(4), 505-532. doi:10.3390/fi2040505Seyedi, Y., Karimi, H., & Guerrero, J. M. (2017). Centralized Disturbance Detection in Smart Microgrids With Noisy and Intermittent Synchrophasor Data. IEEE Transactions on Smart Grid, 8(6), 2775-2783. doi:10.1109/tsg.2016.2539947Youssef, T., Elsayed, A., & Mohammed, O. (2016). Data Distribution Service-Based Interoperability Framework for Smart Grid Testbed Infrastructure. Energies, 9(3), 150. doi:10.3390/en9030150Liu, X., Xia, H., & Chien, A. A. (2004). Validating and Scaling the MicroGrid: A Scientific Instrument for Grid Dynamics. Journal of Grid Computing, 2(2), 141-161. doi:10.1007/s10723-004-4200-3Kansal, P., & Bose, A. (2012). Bandwidth and Latency Requirements for Smart Transmission Grid Applications. IEEE Transactions on Smart Grid, 3(3), 1344-1352. doi:10.1109/tsg.2012.2197229Kuo, M.-T., & Lu, S.-D. (2013). Design and Implementation of Real-Time Intelligent Control and Structure Based on Multi-Agent Systems in Microgrids. Energies, 6(11), 6045-6059. doi:10.3390/en6116045Del Val, E., Rebollo, M., & Botti, V. (2012). Enhancing decentralized service discovery in open service-oriented multi-agent systems. Autonomous Agents and Multi-Agent Systems, 28(1), 1-30. doi:10.1007/s10458-012-9210-0Howell, S., Rezgui, Y., Hippolyte, J.-L., Jayan, B., & Li, H. (2017). Towards the next generation of smart grids: Semantic and holonic multi-agent management of distributed energy resources. Renewable and Sustainable Energy Reviews, 77, 193-214. doi:10.1016/j.rser.2017.03.107Frey, S., Diaconescu, A., Menga, D., & Demeure, I. (2015). A Generic Holonic Control Architecture for Heterogeneous Multiscale and Multiobjective Smart Microgrids. ACM Transactions on Autonomous and Adaptive Systems, 10(2), 1-21. doi:10.1145/2700326Miers, C., Simplicio, M., Gallo, D., Carvalho, T., Bressan, G., Souza, V., … Damola, A. (2010). A Taxonomy for Locality Algorithms on Peer-to-Peer Networks. IEEE Latin America Transactions, 8(4), 323-331. doi:10.1109/tla.2010.5595121Porsinger, T., Janik, P., Leonowicz, Z., & Gono, R. (2017). Modelling and Optimization in Microgrids. Energies, 10(4), 523. doi:10.3390/en10040523Ali, M., Zakariya, M., Asif, M., & Ullah, A. (2012). TCP/IP Based Intelligent Load Management System in Micro-Grids Network Using MATLAB/Simulink. Energy and Power Engineering, 04(04), 283-289. doi:10.4236/epe.2012.44038Shin, I.-J., Song, B.-K., & Eom, D.-S. (2017). International Electronical Committee (IEC) 61850 Mapping with Constrained Application Protocol (CoAP) in Smart Grids Based European Telecommunications Standard Institute Machine-to-Machine (M2M) Environment. Energies, 10(3), 393. doi:10.3390/en10030393Loh, P. C., Li, D., Chai, Y. K., & Blaabjerg, F. (2013). Autonomous Operation of Hybrid Microgrid With AC and DC Subgrids. IEEE Transactions on Power Electronics, 28(5), 2214-2223. doi:10.1109/tpel.2012.2214792Overlay networks for smart gridshttp://users.atlantis.ugent.be/cdvelder/papers/2013/wauters2013sgv.pdfEugster, P. T., Felber, P. A., Guerraoui, R., & Kermarrec, A.-M. (2003). The many faces of publish/subscribe. ACM Computing Surveys, 35(2), 114-131. doi:10.1145/857076.857078Ali, I. (2012). High-speed Peer-to-peer Communication based Protection Scheme Implementation and Testing in Laboratory. International Journal of Computer Applications, 38(4), 16-24. doi:10.5120/4596-6793Yoo, B.-K., Yang, S.-H., Yang, H.-S., Kim, W.-Y., Jeong, Y.-S., Han, B.-M., & Jang, K.-S. (2011). Communication Architecture of the IEC 61850-based Micro Grid System. Journal of Electrical Engineering and Technology, 6(5), 605-612. doi:10.5370/jeet.2011.6.5.605Dou, X., Quan, X., Wu, Z., Hu, M., Yang, K., Yuan, J., & Wang, M. (2014). Hybrid Multi-Agent Control in Microgrids: Framework, Models and Implementations Based on IEC 61850. Energies, 8(1), 31-58. doi:10.3390/en801003

A framework for the dynamic management of Peer-to-Peer overlays

Peer-to-Peer (P2P) applications have been associated with inefficient operation, interference with other network services and large operational costs for network providers. This thesis presents a framework which can help ISPs address these issues by means of intelligent management of peer behaviour. The proposed approach involves limited control of P2P overlays without interfering with the fundamental characteristics of peer autonomy and decentralised operation. At the core of the management framework lays the Active Virtual Peer (AVP). Essentially intelligent peers operated by the network providers, the AVPs interact with the overlay from within, minimising redundant or inefficient traffic, enhancing overlay stability and facilitating the efficient and balanced use of available peer and network resources. They offer an “insider‟s” view of the overlay and permit the management of P2P functions in a compatible and non-intrusive manner. AVPs can support multiple P2P protocols and coordinate to perform functions collectively. To account for the multi-faceted nature of P2P applications and allow the incorporation of modern techniques and protocols as they appear, the framework is based on a modular architecture. Core modules for overlay control and transit traffic minimisation are presented. Towards the latter, a number of suitable P2P content caching strategies are proposed. Using a purpose-built P2P network simulator and small-scale experiments, it is demonstrated that the introduction of AVPs inside the network can significantly reduce inter-AS traffic, minimise costly multi-hop flows, increase overlay stability and load-balancing and offer improved peer transfer performance

SoS: self-organizing substrates

Large-scale networked systems often, both by design or chance exhibit self-organizing properties. Understanding self-organization using tools from cybernetics, particularly modeling them as Markov processes is a first step towards a formal framework which can be used in (decentralized) systems research and design.Interesting aspects to look for include the time evolution of a system and to investigate if and when a system converges to some absorbing states or stabilizes into a dynamic (and stable) equilibrium and how it performs under such an equilibrium state. Such a formal framework brings in objectivity in systems research, helping discern facts from artefacts as well as providing tools for quantitative evaluation of such systems. This thesis introduces such formalism in analyzing and evaluating peer-to-peer (P2P) systems in order to better understand the dynamics of such systems which in turn helps in better designs. In particular this thesis develops and studies the fundamental building blocks for a P2P storage system. In the process the design and evaluation methodology we pursue illustrate the typical methodological approaches in studying and designing self-organizing systems, and how the analysis methodology influences the design of the algorithms themselves to meet system design goals (preferably with quantifiable guarantees). These goals include efficiency, availability and durability, load-balance, high fault-tolerance and self-maintenance even in adversarial conditions like arbitrarily skewed and dynamic load and high membership dynamics (churn), apart of-course the specific functionalities that the system is supposed to provide. The functionalities we study here are some of the fundamental building blocks for various P2P applications and systems including P2P storage systems, and hence we call them substrates or base infrastructure. These elemental functionalities include: (i) Reliable and efficient discovery of resources distributed over the network in a decentralized manner; (ii) Communication among participants in an address independent manner, i.e., even when peers change their physical addresses; (iii) Availability and persistence of stored objects in the network, irrespective of availability or departure of individual participants from the system at any time; and (iv) Freshness of the objects/resources' (up-to-date replicas). Internet-scale distributed index structures (often termed as structured overlays) are used for discovery and access of resources in a decentralized setting. We propose a rapid construction from scratch and maintenance of the P-Grid overlay network in a self-organized manner so as to provide efficient search of both individual keys as well as a whole range of keys, doing so providing good load-balancing characteristics for diverse kind of arbitrarily skewed loads - storage and replication, query forwarding and query answering loads. For fast overlay construction we employ recursive partitioning of the key-space so that the resulting partitions are balanced with respect to storage load and replication. The proper algorithmic parameters for such partitioning is derived from a transient analysis of the partitioning process which has Markov property. Preservation of ordering information in P-Grid such that queries other than exact queries, like range queries can be efficiently and rather trivially handled makes P-Grid suitable for data-oriented applications. Fast overlay construction is analogous to building an index on a new set of keys making P-Grid suitable as the underlying indexing mechanism for peer-to-peer information retrieval applications among other potential applications which may require frequent indexing of new attributes apart regular updates to an existing index. In order to deal with membership dynamics, in particular changing physical address of peers across sessions, the overlay itself is used as a (self-referential) directory service for maintaining the participating peers' physical addresses across sessions. Exploiting this self-referential directory, a family of overlay maintenance scheme has been designed with lower communication overhead than other overlay maintenance strategies. The notion of dynamic equilibrium study for overlays under continuous churn and repairs, modeled as a Markov process, was introduced in order to evaluate and compare the overlay maintenance schemes. While the self-referential directory was originally invented to realize overlay maintenance schemes with lower overheads than existing overlay maintenance schemes, the self-referential directory is generic in nature and can be used for various other purposes, e.g., as a decentralized public key infrastructure. Persistence of peer identity across sessions, in spite of changes in physical address, provides a logical independence of the overlay network from the underlying physical network. This has many other potential usages, for example, efficient maintenance mechanisms for P2P storage systems and P2P trust and reputation management. We specifically look into the dynamics of maintaining redundancy for storage systems and design a novel lazy maintenance strategy. This strategy is algorithmically a simple variant of existing maintenance strategies which adapts to the system dynamics. This randomized lazy maintenance strategy thus explores the cost-performance trade-offs of the storage maintenance operations in a self-organizing manner. We model the storage system (redundancy), under churn and maintenance, as a Markov process. We perform an equilibrium study to show that the system operates in a more stable dynamic equilibrium with our strategy than for the existing maintenance scheme for comparable overheads. Particularly, we show that our maintenance scheme provides substantial performance gains in terms of maintenance overhead and system's resilience in presence of churn and correlated failures. Finally, we propose a gossip mechanism which works with lower communication overhead than existing approaches for communication among a relatively large set of unreliable peers without assuming any specific structure for their mutual connectivity. We use such a communication primitive for propagating replica updates in P2P systems, facilitating management of mutable content in P2P systems. The peer population affected by a gossip can be modeled as a Markov process. Studying the transient spread of gossips help in choosing proper algorithm parameters to reduce communication overhead while guaranteeing coverage of online peers. Each of these substrates in themselves were developed to find practical solutions for real problems. Put together, these can be used in other applications, including a P2P storage system with support for efficient lookup and inserts, membership dynamics, content mutation and updates, persistence and availability. Many of the ideas have already been implemented in real systems and several others are in the way to be integrated into the implementations. There are two principal contributions of this dissertation. It provides design of the P2P systems which are useful for end-users as well as other application developers who can build upon these existing systems. Secondly, it adapts and introduces the methodology of analysis of a system's time-evolution (tools typically used in diverse domains including physics and cybernetics) to study the long run behavior of P2P systems, and uses this methodology to (re-)design appropriate algorithms and evaluate them. We observed that studying P2P systems from the perspective of complex systems reveals their inner dynamics and hence ways to exploit such dynamics for suitable or better algorithms. In other words, the analysis methodology in itself strongly influences and inspires the way we design such systems. We believe that such an approach of orchestrating self-organization in internet-scale systems, where the algorithms and the analysis methodology have strong mutual influence will significantly change the way future such systems are developed and evaluated. We envision that such an approach will particularly serve as an important tool for the nascent but fast moving P2P systems research and development community