Resource discovery for distributed computing systems: A comprehensive survey

Large-scale distributed computing environments provide a vast amount of heterogeneous computing resources from different sources for resource sharing and distributed computing. Discovering appropriate resources in such environments is a challenge which involves several different subjects. In this paper, we provide an investigation on the current state of resource discovery protocols, mechanisms, and platforms for large-scale distributed environments, focusing on the design aspects. We classify all related aspects, general steps, and requirements to construct a novel resource discovery solution in three categories consisting of structures, methods, and issues. Accordingly, we review the literature, analyzing various aspects for each category

Utility-Based Mechanism for Structural Self-Organization in Service-Oriented MAS

Structural relations established among agents influence the performance of decentralized service discovery process in multiagent systems. Moreover, distributed systems should be able to adapt their structural relations to changes in environmental conditions. In this article, we present a service-oriented multiagent systems, where agents initially self-organize their structural relations based on the similarity of their services. During the service discovery process, agents integrate a mechanism that facilitates the self-organization of their structural relations to adapt the structure of the system to the service demand. This mechanism facilitates the task of decentralized service discovery and improves its performance. Each agent has local knowledge about its direct neighbors and the queries received during discovery processes. With this information, an agent is able to analyze its structural relations and decide when it is more appropriate to modify its direct neighbors and select the most suitable acquaintances to replace them. The experimental evaluation shows how this self-organization mechanism improves the overall performance of the service discovery process in the system when the service demand changesThis work is partially supported by the Spanish Ministry of Science and Innovation through grants CSD2007-0022 (CONSOLIDER-INGENIO 2010), TIN2012-36586-C03-01, TIN2012-36586-C03-01, TIN2012-36586-C03-02, PROMETEOII/2013/019, and FPU grant AP-2008-00601 awarded to E. Del Val.Del Val Noguera, E.; Rebollo Pedruelo, M.; Vasirani, M.; Fernández, A. (2014). Utility-Based Mechanism for Structural Self-Organization in Service-Oriented MAS. ACM Transactions on Autonomous and Adaptive Systems. 9(3):1-24. https://doi.org/10.1145/2651423S12493Sherief Abdallah and Victor Lesser. 2007. Multiagent reinforcement learning and self-organization in a network of agents. In Proceedings of the 6th International Conference on Autonomous Agents and Multiagent Systems. 172--179.Lada A. Adamic and Bernardo A. Huberman. 2002. Zipf’s law and the Internet. Glottometrics 3, 143--150.Muntasir Al-Asfoor, Brendan Neville, and Maria Fasli. 2012. Heuristic resource search in a self-organised distributed multi agent system. In Proceedings of the 6th International Workshop on Self-Organizing Systems. 84--89.Mathieu Aquin, Salman Elahi, and Enrico Motta. 2010. Personal monitoring of Web information exchange: Towards Web lifelogging. In Proceedings of the Web Science Conference.Ulrich Basters and Matthias Klusch. 2006. RS2D: Fast adaptive search for semantic Web services in unstructured p2p networks. In Proceedings of the International Semantic Web Conference. 87--100.Umesh Bellur and Roshan Kulkarni. 2007. Improved matchmaking algorithm for semantic Web services based on bipartite graph matching. In Proceedings of the International Semantic Web Conference. 86--93.Devis Bianchini, Valeria De Antonellis, and Michele Melchiori. 2009. Service-based semantic search in p2p systems. In Proceedings of the European Conference on Web Services. 7--16.Bartosz Biskupski, Jim Dowling, and Jan Sacha. 2007. Properties and mechanisms of self-organizing MANET and P2P systems. ACM Transactions on Autonomous and Adaptive Systems 2, 1, 1--34.Alberto Blanc, Yi-Kai Liu, and Amin Vahdat. 2005. Designing incentives for peer-to-peer routing. In Proceedings of the 24th Annual Joint Conference of the IEEE Computer and Communications Societies. 374--385.Michael Bowling and Manuela Veloso. 2002. Multiagent learning using a variable learning rate. Artificial Intelligence 136, 215--250.Frances M. T. Brazier, Jeffrey O. Kephart, H. Van Dyke Parunak, and Michael N. Huhns. 2009. Agents and service-oriented computing for autonomic computing: A research agenda. IEEE Internet Computing 13, 3, 82--87.Tyson Condie, Sepandar D. Kamvar, and Hector Garcia-Molina. 2004. Adaptive peer-to-peer topologies. In Proceedings of the 4th International Conference on Peer-to-Peer Computing. 53--62.Arturo Crespo and Hector Garcia-Molina. 2002. Routing indices for peer-to-peer systems. In Proceedings of the 22nd International Conference on Distributed Computing Systems. 23--32.Elena Del Val, Natalia Criado, Carlos Carrascosa, Vicente Julian, Miguel Rebollo, Estefania Argente, and Vicente Botti. 2010. THOMAS: A service-oriented framework for virtual organizations. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems (AAMAS’10). 1631--1632.Elena Del Val, Miguel Rebollo, and Vicente Botti. 2011. Introducing homophily to improve semantic service search in a self-adaptive system. In Proceedings of the 10th International Conference on Autonomous Agents and Multiagent Systems. 1241--1242.Elena Del Val, Miguel Rebollo, and Vicente Botti. 2012a. Enhancing decentralized service discovery in open service-oriented multi-agent systems. Autonomous Agents and Multi-Agent Systems 28, 1, 1--30.Elena Del Val, Miguel Rebollo, and Vicente Botti. 2012b. Promoting cooperation in service-oriented MAS through social plasticity and incentives. Journal of Systems and Software 86, 2, 520--537.Gianni Di Caro, Frederick Ducatelle, and Luca Maria Gambardella. 2005. AntHocNet: An adaptive nature-inspired algorithm for routing in mobile ad hoc networks. European Transactions on Telecommunications 16, 443--455.Ding Ding, Lei Liu, and Hartmut Schmeck. 2010. Service discovery in self-organizing service-oriented environments. In Proceedings of the 2010 IEEE Asia-Pacific Services Computing Conference. 717--724.Sergey N. Dorogovtsev and Jose F. F. Mendes. 2003. Evolution of Networks: From Biological Nets to the Internet and WWW. Oxford University Press.Giovanna Di Marzo Serugendo, Marie-Pierre Gleizes, and Anthony Karageorgos. 2011. Self-Organizing Software: From Natural to Artificial Adaptation. Natural Computing Series.Erik Einhorn and Andreas Mitschele-Thiel. 2008. RLTE: Reinforcement learning for traffic-engineering. In Proceedings of the 2nd International Conference on Autonomous Infrastructure, Management, and Security. 120--133.Nelson Fernandez, Carlos Maldonado, and Carlos Gershenson. 2014. Information measures of complexity, emergence, self-organization, homeostasis, and autopoiesis. In Guided Self-Organization: Inception. Emergence, Complexity and Computation, Vol. 9. Springer, 19--51. DOI: http://dx.doi.org/10.1007/978-3-642-53734-9_2Jose Luis Fernandez-Marquez, Josep Lluis Arcos, and Giovanna Di Marzo Serugendo. 2012. A decentralized approach for detecting dynamically changing diffuse event sources in noisy WSN environments. Applied Artificial Intelligence 26, 4, 376--397. DOI: http://dx.doi.org/10.1080/08839514.2012.653659Agostino Forestiero, Carlo Mastroianni, and Michela Meo. 2009. Self-Chord: A bio-inspired algorithm for structured P2P systems. In Proceedings of the 9th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing. 44--51.Matthew E. Gaston and Marie des Jardins. 2005. Agent-organized networks for multi-agent production and exchange. In Proceedings of the 20th AAAI Conference on Artificial Intelligence. 77--82.Nathan Griffiths and Michael Luck. 2010. Changing neighbours: Improving tag-based cooperation. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems. 249--256.Peter Haase, Ronny Siebes, and Frank van Harmelen. 2008. Expertise-based peer selection in peer-to-peer networks. Knowledge and Information Systems 15, 1, 75--107.Philip N. Howard, Lee Rainee, and Steve Jones. 2001. Days and nights on the Internet. American Behavioural Scientist, 383--404.Bernardo A. Huberman and Lada A. Adamic. 2000. The nature of markets in the WWW. Quarterly Journal of Electronic Commerce 1, 5--12.Michael N. Huhns et al. 2005. Research directions for service-oriented multiagent systems. IEEE Internet Computing 9, 6, 65--70.Tomoko Itao, Tatsuya Suda, Tetsuya Nakamura, Miyuki Imada, Masato Matsuo, and Tomonori Aoyama. 2001. Jack-in-the-Net: Adaptive networking architecture for service emergence. In Proceedings of the Asian-Pacific Conference on Communications. 9.Emily M. Jin, Michelle Girvan, and Mark E. J. Newman. 2001. Structure of growing social networks. Physical Review E 64, 4, 046132.Sachin Kamboj and Keith S. Decker. 2007. Organizational self-design in semi-dynamic environments. In Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems. 335--337.Rahamatullah Khondoker, S. M. Taslim Arif, Nathan Kerr, and Dennis Schwerdel. 2011. Self-organizing communication services in future network architectures. In Proceedings of the 5th International Workshop on Self-Organizing Systems.Matthias Klusch, Benedikt Fries, and Katia Sycara. 2009. OWLS-MX: A hybrid Semantic Web service matchmaker for OWL-S services. Web Semantics Science Services and Agents on the World Wide Web 7, 2, 121--133.Dionisis Kontominas, Paraskevi Raftopoulou, Christos Tryfonopoulos, and Euripides G. M. Petrakis. 2013. DS4: A distributed social and semantic search system. Advances in Information Retrieval 7814, 832--836.Ramachandra Kota, Nicholas Gibbins, and Nicholas R. Jennings. 2012. Decentralized approaches for self-adaptation in agent organizations. ACM Transactions on Autonomous and Adaptive Systems 7, 1, Article No. 1.Paul Lazarsfeld. 1954. Friendship as a social process: A substantive and methodological analysis. In Freedom and Control in Modern Society. Van Nostrand, New York, NY.Paulo Leito. 2013. Towards self-organized service-oriented multi-agent systems. In Service Orientation in Holonic and Multi Agent Manufacturing and Robotics. Studies in Computational Intelligence, Vol. 472. Springer, 41--56.W. Sabrina Lin, Hong Vikcy Zhao, and K. J. Ray Liu. 2009. Incentive cooperation strategies for peer-to-peer live multimedia streaming social networks. IEEE Transactions on Multimedia 11, 3, 396--412.Sheila A. McIlraith, Tran Cao Son, and Honglei Zeng. 2001. Semantic Web services. IEEE Intelligent Systems 16, 2, 46--53.Miller McPherson, Lynn Smith-Lovin, and James Cook. 2001. Birds of a feather: Homophily in social networks. Annual Review of Sociology 27, 415--444.Vivek Nallur and Rami Bahsoon. 2012. A decentralized self-adaptation mechanism for service-based applications in the cloud. IEEE Transactions on Software Engineering 99, 591--612.Aris Ouksel, Yair Babad, and Thomas Tesch. 2004. Matchmaking software agents in B2B markets. In Proceedings of the 37th Annual Hawaii International Conference on System Sciences. 1--9.Massimo Paolucci, Takahiro Kawamura, Terry R. Payne, and Katia P. Sycara. 2002. Semantic matching of Web services capabilities. In Proceedings of the 1st International Semantic Web Conference. 333--347.Leonid Peshkin and Virginia Savova. 2002. Reinforcement learning for adaptive routing. In Proceedings of the 2002 International Conference on Neural Networks (IJCNN’02). 1825--1830.Paraskevi Raftopoulou and Euripides G. M. Petrakis. 2008. iCluster: A self-organizing overlay network for P2P information retrieval. In Proceedings of the 30th European Conference on Advances in Information Retrieval (ECIR’08). 65--76.Sharmila Savarimuthu, Maryam Purvis, Martin Purvis, and Bastin Tony Roy Savarimuthu. 2011. Mechanisms for the self-organization of peer groups in agent societies. In Multi-Agent-Based Simulation XI. Lecture Notes in Computer Science, Vol. 6532. Springer, 93--107.Giovanna Di Marzo Serugendo, Marie-Pierre Gleizes, and Anthony Karageorgos. 2005. Self-organization in multi-agent systems. Knowledge Engineering Review 20, 2, 165--189.Abdul Khalique Shaikh, Saadat M. Alhashmi, and Rajendran Parthiban. 2012. A semantic impact in decentralized resource discovery mechanism for grid computing environments. In Algorithms and Architectures for Parallel Processing. Lecture Notes in Computer Science, Vol. 7440. Springer, 206--216.Qixiang Sun and Hector Garcia-Molina. 2004. SLIC: A selfish link-based incentive mechanism for unstructured peer-to-peer networks. In Proceedings of the 24th International Conference on Distributed Computing Systems (ICDCS’04). 506--515.Mirko Viroli and Franco Zambonelli. 2010. A biochemical approach to adaptive service ecosystems. Information Sciences 180, 10, 1876--1892. DOI: http://dx.doi.org/10.1016/j.ins.2009.11.021Li Wang. 2011. SoFA: An expert-driven, self-organization peer-to-peer semantic communities for network resource management. Expert Systems with Applications 38, 1, 94--105.Kevin Werbach. 2000. Syndication—the emerging model for business in the Internet era. Harvard Business Review 78, 3, 84--93, 214.Tom Wolf and Tom Holvoet. 2005. Emergence versus self-organisation: Different concepts but promising when combined. In Engineering Self-Organising Systems. Lecture Notes in Computer Science, Vol. 3464. Springer, 1--15.Haizheng Zhang, W. Bruce Croft, Brian Levine, and Victor Lesser. 2004. A multi-agent approach for peer-to-peer based information retrieval system. In Proceedings of the 3rd International Conference on Autonomous Agents and Multiagent Systems, Vol. 1. 456--463.Ming Zhong. 2006. Popularity-biased random walks for peer-to-peer search under the square-root principle. In Proceedings of the 5th International Workshop on Peer-to-Peer Systems

Semantic search and composition in unstructured peer-to-peer networks

This dissertation focuses on several research questions in the area of semantic search and composition in unstructured peer-to-peer (P2P) networks. Going beyond the state of the art, the proposed semantic-based search strategy S2P2P offers a novel path-suggestion based query routing mechanism, providing a reasonable tradeoff between search performance and network traffic overhead. In addition, the first semantic-based data replication scheme DSDR is proposed. It enables peers to use semantic information to select replica numbers and target peers to address predicted future demands. With DSDR, k-random search can achieve better precision and recall than it can with a near-optimal non-semantic replication strategy. Further, this thesis introduces a functional automatic semantic service composition method, SPSC. Distinctively, it enables peers to jointly compose complex workflows with high cumulative recall but low network traffic overhead, using heuristic-based bidirectional haining and service memorization mechanisms. Its query branching method helps to handle dead-ends in a pruned search space. SPSC is proved to be sound and a lower bound of is completeness is given. Finally, this thesis presents iRep3D for semantic-index based 3D scene selection in P2P search. Its efficient retrieval scales to answer hybrid queries involving conceptual, functional and geometric aspects. iRep3D outperforms previous representative efforts in terms of search precision and efficiency.Diese Dissertation bearbeitet Forschungsfragen zur semantischen Suche und Komposition in unstrukturierten Peer-to-Peer Netzen(P2P). Die semantische Suchstrategie S2P2P verwendet eine neuartige Methode zur Anfrageweiterleitung basierend auf Pfadvorschlägen, welche den Stand der Wissenschaft übertrifft. Sie bietet angemessene Balance zwischen Suchleistung und Kommunikationsbelastung im Netzwerk. Außerdem wird das erste semantische System zur Datenreplikation genannt DSDR vorgestellt, welche semantische Informationen berücksichtigt vorhergesagten zukünftigen Bedarf optimal im P2P zu decken. Hierdurch erzielt k-random-Suche bessere Präzision und Ausbeute als mit nahezu optimaler nicht-semantischer Replikation. SPSC, ein automatisches Verfahren zur funktional korrekten Komposition semantischer Dienste, ermöglicht es Peers, gemeinsam komplexe Ablaufpläne zu komponieren. Mechanismen zur heuristischen bidirektionalen Verkettung und Rückstellung von Diensten ermöglichen hohe Ausbeute bei geringer Belastung des Netzes. Eine Methode zur Anfrageverzweigung vermeidet das Feststecken in Sackgassen im beschnittenen Suchraum. Beweise zur Korrektheit und unteren Schranke der Vollständigkeit von SPSC sind gegeben. iRep3D ist ein neuer semantischer Selektionsmechanismus für 3D-Modelle in P2P. iRep3D beantwortet effizient hybride Anfragen unter Berücksichtigung konzeptioneller, funktionaler und geometrischer Aspekte. Der Ansatz übertrifft vorherige Arbeiten bezüglich Präzision und Effizienz

Combination of self-organization mechanisms to enhance service discovery in open systems

Decentralized systems have emerged as an alternative to centralized approaches for dealing with dynamic requirements in new business models. These systems should provide mechanisms that contribute to flexibility and facilitate adaptation to changes in the environment. In this paper, we present two self-organization mechanisms for a decentralized service discovery system in order to improve its performance. These mechanisms are based on local actions of agents that only consider local information about queries they forward during the discovery process. The self-organization actions are chosen by each agent individually when the agent considers them to be appropriate. The actions are: remaining in the system, leaving the system, cloning, and changing structural relations with other agents. We have evaluated each self-organization mechanism separately but also the combination of the two as the environmental conditions in the service demand change. The results show that the proposed self-organization mechanisms considerably improve the performance of the service discovery systemDel Val Noguera, E.; Rebollo Pedruelo, M.; Botti Navarro, VJ. (2014). Combination of self-organization mechanisms to enhance service discovery in open systems. Information Sciences. 279:138-162. doi:10.1016/j.ins.2014.03.109S13816227