Distributed Information Retrieval using Keyword Auctions

This report motivates the need for large-scale distributed approaches to information retrieval, and proposes solutions based on keyword auctions

Organic Design of Massively Distributed Systems: A Complex Networks Perspective

The vision of Organic Computing addresses challenges that arise in the design of future information systems that are comprised of numerous, heterogeneous, resource-constrained and error-prone components or devices. Here, the notion organic particularly highlights the idea that, in order to be manageable, such systems should exhibit self-organization, self-adaptation and self-healing characteristics similar to those of biological systems. In recent years, the principles underlying many of the interesting characteristics of natural systems have been investigated from the perspective of complex systems science, particularly using the conceptual framework of statistical physics and statistical mechanics. In this article, we review some of the interesting relations between statistical physics and networked systems and discuss applications in the engineering of organic networked computing systems with predictable, quantifiable and controllable self-* properties.Comment: 17 pages, 14 figures, preprint of submission to Informatik-Spektrum published by Springe

Distributed top-k aggregation queries at large

Top-k query processing is a fundamental building block for efficient ranking in a large number of applications. Efficiency is a central issue, especially for distributed settings, when the data is spread across different nodes in a network. This paper introduces novel optimization methods for top-k aggregation queries in such distributed environments. The optimizations can be applied to all algorithms that fall into the frameworks of the prior TPUT and KLEE methods. The optimizations address three degrees of freedom: 1) hierarchically grouping input lists into top-k operator trees and optimizing the tree structure, 2) computing data-adaptive scan depths for different input sources, and 3) data-adaptive sampling of a small subset of input sources in scenarios with hundreds or thousands of query-relevant network nodes. All optimizations are based on a statistical cost model that utilizes local synopses, e.g., in the form of histograms, efficiently computed convolutions, and estimators based on order statistics. The paper presents comprehensive experiments, with three different real-life datasets and using the ns-2 network simulator for a packet-level simulation of a large Internet-style network

Utility-Based Mechanism for Structural Self-Organization in Service-Oriented MAS

Structural relations established among agents influence the performance of decentralized service discovery process in multiagent systems. Moreover, distributed systems should be able to adapt their structural relations to changes in environmental conditions. In this article, we present a service-oriented multiagent systems, where agents initially self-organize their structural relations based on the similarity of their services. During the service discovery process, agents integrate a mechanism that facilitates the self-organization of their structural relations to adapt the structure of the system to the service demand. This mechanism facilitates the task of decentralized service discovery and improves its performance. Each agent has local knowledge about its direct neighbors and the queries received during discovery processes. With this information, an agent is able to analyze its structural relations and decide when it is more appropriate to modify its direct neighbors and select the most suitable acquaintances to replace them. The experimental evaluation shows how this self-organization mechanism improves the overall performance of the service discovery process in the system when the service demand changesThis work is partially supported by the Spanish Ministry of Science and Innovation through grants CSD2007-0022 (CONSOLIDER-INGENIO 2010), TIN2012-36586-C03-01, TIN2012-36586-C03-01, TIN2012-36586-C03-02, PROMETEOII/2013/019, and FPU grant AP-2008-00601 awarded to E. Del Val.Del Val Noguera, E.; Rebollo Pedruelo, M.; Vasirani, M.; Fernández, A. (2014). Utility-Based Mechanism for Structural Self-Organization in Service-Oriented MAS. ACM Transactions on Autonomous and Adaptive Systems. 9(3):1-24. https://doi.org/10.1145/2651423S12493Sherief Abdallah and Victor Lesser. 2007. Multiagent reinforcement learning and self-organization in a network of agents. In Proceedings of the 6th International Conference on Autonomous Agents and Multiagent Systems. 172--179.Lada A. Adamic and Bernardo A. Huberman. 2002. Zipf’s law and the Internet. Glottometrics 3, 143--150.Muntasir Al-Asfoor, Brendan Neville, and Maria Fasli. 2012. Heuristic resource search in a self-organised distributed multi agent system. In Proceedings of the 6th International Workshop on Self-Organizing Systems. 84--89.Mathieu Aquin, Salman Elahi, and Enrico Motta. 2010. Personal monitoring of Web information exchange: Towards Web lifelogging. In Proceedings of the Web Science Conference.Ulrich Basters and Matthias Klusch. 2006. RS2D: Fast adaptive search for semantic Web services in unstructured p2p networks. In Proceedings of the International Semantic Web Conference. 87--100.Umesh Bellur and Roshan Kulkarni. 2007. Improved matchmaking algorithm for semantic Web services based on bipartite graph matching. In Proceedings of the International Semantic Web Conference. 86--93.Devis Bianchini, Valeria De Antonellis, and Michele Melchiori. 2009. Service-based semantic search in p2p systems. In Proceedings of the European Conference on Web Services. 7--16.Bartosz Biskupski, Jim Dowling, and Jan Sacha. 2007. Properties and mechanisms of self-organizing MANET and P2P systems. ACM Transactions on Autonomous and Adaptive Systems 2, 1, 1--34.Alberto Blanc, Yi-Kai Liu, and Amin Vahdat. 2005. Designing incentives for peer-to-peer routing. In Proceedings of the 24th Annual Joint Conference of the IEEE Computer and Communications Societies. 374--385.Michael Bowling and Manuela Veloso. 2002. Multiagent learning using a variable learning rate. Artificial Intelligence 136, 215--250.Frances M. T. Brazier, Jeffrey O. Kephart, H. Van Dyke Parunak, and Michael N. Huhns. 2009. Agents and service-oriented computing for autonomic computing: A research agenda. IEEE Internet Computing 13, 3, 82--87.Tyson Condie, Sepandar D. Kamvar, and Hector Garcia-Molina. 2004. Adaptive peer-to-peer topologies. In Proceedings of the 4th International Conference on Peer-to-Peer Computing. 53--62.Arturo Crespo and Hector Garcia-Molina. 2002. Routing indices for peer-to-peer systems. In Proceedings of the 22nd International Conference on Distributed Computing Systems. 23--32.Elena Del Val, Natalia Criado, Carlos Carrascosa, Vicente Julian, Miguel Rebollo, Estefania Argente, and Vicente Botti. 2010. THOMAS: A service-oriented framework for virtual organizations. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems (AAMAS’10). 1631--1632.Elena Del Val, Miguel Rebollo, and Vicente Botti. 2011. Introducing homophily to improve semantic service search in a self-adaptive system. In Proceedings of the 10th International Conference on Autonomous Agents and Multiagent Systems. 1241--1242.Elena Del Val, Miguel Rebollo, and Vicente Botti. 2012a. Enhancing decentralized service discovery in open service-oriented multi-agent systems. Autonomous Agents and Multi-Agent Systems 28, 1, 1--30.Elena Del Val, Miguel Rebollo, and Vicente Botti. 2012b. Promoting cooperation in service-oriented MAS through social plasticity and incentives. Journal of Systems and Software 86, 2, 520--537.Gianni Di Caro, Frederick Ducatelle, and Luca Maria Gambardella. 2005. AntHocNet: An adaptive nature-inspired algorithm for routing in mobile ad hoc networks. European Transactions on Telecommunications 16, 443--455.Ding Ding, Lei Liu, and Hartmut Schmeck. 2010. Service discovery in self-organizing service-oriented environments. In Proceedings of the 2010 IEEE Asia-Pacific Services Computing Conference. 717--724.Sergey N. Dorogovtsev and Jose F. F. Mendes. 2003. Evolution of Networks: From Biological Nets to the Internet and WWW. Oxford University Press.Giovanna Di Marzo Serugendo, Marie-Pierre Gleizes, and Anthony Karageorgos. 2011. Self-Organizing Software: From Natural to Artificial Adaptation. Natural Computing Series.Erik Einhorn and Andreas Mitschele-Thiel. 2008. RLTE: Reinforcement learning for traffic-engineering. In Proceedings of the 2nd International Conference on Autonomous Infrastructure, Management, and Security. 120--133.Nelson Fernandez, Carlos Maldonado, and Carlos Gershenson. 2014. Information measures of complexity, emergence, self-organization, homeostasis, and autopoiesis. In Guided Self-Organization: Inception. Emergence, Complexity and Computation, Vol. 9. Springer, 19--51. DOI: http://dx.doi.org/10.1007/978-3-642-53734-9_2Jose Luis Fernandez-Marquez, Josep Lluis Arcos, and Giovanna Di Marzo Serugendo. 2012. A decentralized approach for detecting dynamically changing diffuse event sources in noisy WSN environments. Applied Artificial Intelligence 26, 4, 376--397. DOI: http://dx.doi.org/10.1080/08839514.2012.653659Agostino Forestiero, Carlo Mastroianni, and Michela Meo. 2009. Self-Chord: A bio-inspired algorithm for structured P2P systems. In Proceedings of the 9th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing. 44--51.Matthew E. Gaston and Marie des Jardins. 2005. Agent-organized networks for multi-agent production and exchange. In Proceedings of the 20th AAAI Conference on Artificial Intelligence. 77--82.Nathan Griffiths and Michael Luck. 2010. Changing neighbours: Improving tag-based cooperation. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems. 249--256.Peter Haase, Ronny Siebes, and Frank van Harmelen. 2008. Expertise-based peer selection in peer-to-peer networks. Knowledge and Information Systems 15, 1, 75--107.Philip N. Howard, Lee Rainee, and Steve Jones. 2001. Days and nights on the Internet. American Behavioural Scientist, 383--404.Bernardo A. Huberman and Lada A. Adamic. 2000. The nature of markets in the WWW. Quarterly Journal of Electronic Commerce 1, 5--12.Michael N. Huhns et al. 2005. Research directions for service-oriented multiagent systems. IEEE Internet Computing 9, 6, 65--70.Tomoko Itao, Tatsuya Suda, Tetsuya Nakamura, Miyuki Imada, Masato Matsuo, and Tomonori Aoyama. 2001. Jack-in-the-Net: Adaptive networking architecture for service emergence. In Proceedings of the Asian-Pacific Conference on Communications. 9.Emily M. Jin, Michelle Girvan, and Mark E. J. Newman. 2001. Structure of growing social networks. Physical Review E 64, 4, 046132.Sachin Kamboj and Keith S. Decker. 2007. Organizational self-design in semi-dynamic environments. In Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems. 335--337.Rahamatullah Khondoker, S. M. Taslim Arif, Nathan Kerr, and Dennis Schwerdel. 2011. Self-organizing communication services in future network architectures. In Proceedings of the 5th International Workshop on Self-Organizing Systems.Matthias Klusch, Benedikt Fries, and Katia Sycara. 2009. OWLS-MX: A hybrid Semantic Web service matchmaker for OWL-S services. Web Semantics Science Services and Agents on the World Wide Web 7, 2, 121--133.Dionisis Kontominas, Paraskevi Raftopoulou, Christos Tryfonopoulos, and Euripides G. M. Petrakis. 2013. DS4: A distributed social and semantic search system. Advances in Information Retrieval 7814, 832--836.Ramachandra Kota, Nicholas Gibbins, and Nicholas R. Jennings. 2012. Decentralized approaches for self-adaptation in agent organizations. ACM Transactions on Autonomous and Adaptive Systems 7, 1, Article No. 1.Paul Lazarsfeld. 1954. Friendship as a social process: A substantive and methodological analysis. In Freedom and Control in Modern Society. Van Nostrand, New York, NY.Paulo Leito. 2013. Towards self-organized service-oriented multi-agent systems. In Service Orientation in Holonic and Multi Agent Manufacturing and Robotics. Studies in Computational Intelligence, Vol. 472. Springer, 41--56.W. Sabrina Lin, Hong Vikcy Zhao, and K. J. Ray Liu. 2009. Incentive cooperation strategies for peer-to-peer live multimedia streaming social networks. IEEE Transactions on Multimedia 11, 3, 396--412.Sheila A. McIlraith, Tran Cao Son, and Honglei Zeng. 2001. Semantic Web services. IEEE Intelligent Systems 16, 2, 46--53.Miller McPherson, Lynn Smith-Lovin, and James Cook. 2001. Birds of a feather: Homophily in social networks. Annual Review of Sociology 27, 415--444.Vivek Nallur and Rami Bahsoon. 2012. A decentralized self-adaptation mechanism for service-based applications in the cloud. IEEE Transactions on Software Engineering 99, 591--612.Aris Ouksel, Yair Babad, and Thomas Tesch. 2004. Matchmaking software agents in B2B markets. In Proceedings of the 37th Annual Hawaii International Conference on System Sciences. 1--9.Massimo Paolucci, Takahiro Kawamura, Terry R. Payne, and Katia P. Sycara. 2002. Semantic matching of Web services capabilities. In Proceedings of the 1st International Semantic Web Conference. 333--347.Leonid Peshkin and Virginia Savova. 2002. Reinforcement learning for adaptive routing. In Proceedings of the 2002 International Conference on Neural Networks (IJCNN’02). 1825--1830.Paraskevi Raftopoulou and Euripides G. M. Petrakis. 2008. iCluster: A self-organizing overlay network for P2P information retrieval. In Proceedings of the 30th European Conference on Advances in Information Retrieval (ECIR’08). 65--76.Sharmila Savarimuthu, Maryam Purvis, Martin Purvis, and Bastin Tony Roy Savarimuthu. 2011. Mechanisms for the self-organization of peer groups in agent societies. In Multi-Agent-Based Simulation XI. Lecture Notes in Computer Science, Vol. 6532. Springer, 93--107.Giovanna Di Marzo Serugendo, Marie-Pierre Gleizes, and Anthony Karageorgos. 2005. Self-organization in multi-agent systems. Knowledge Engineering Review 20, 2, 165--189.Abdul Khalique Shaikh, Saadat M. Alhashmi, and Rajendran Parthiban. 2012. A semantic impact in decentralized resource discovery mechanism for grid computing environments. In Algorithms and Architectures for Parallel Processing. Lecture Notes in Computer Science, Vol. 7440. Springer, 206--216.Qixiang Sun and Hector Garcia-Molina. 2004. SLIC: A selfish link-based incentive mechanism for unstructured peer-to-peer networks. In Proceedings of the 24th International Conference on Distributed Computing Systems (ICDCS’04). 506--515.Mirko Viroli and Franco Zambonelli. 2010. A biochemical approach to adaptive service ecosystems. Information Sciences 180, 10, 1876--1892. DOI: http://dx.doi.org/10.1016/j.ins.2009.11.021Li Wang. 2011. SoFA: An expert-driven, self-organization peer-to-peer semantic communities for network resource management. Expert Systems with Applications 38, 1, 94--105.Kevin Werbach. 2000. Syndication—the emerging model for business in the Internet era. Harvard Business Review 78, 3, 84--93, 214.Tom Wolf and Tom Holvoet. 2005. Emergence versus self-organisation: Different concepts but promising when combined. In Engineering Self-Organising Systems. Lecture Notes in Computer Science, Vol. 3464. Springer, 1--15.Haizheng Zhang, W. Bruce Croft, Brian Levine, and Victor Lesser. 2004. A multi-agent approach for peer-to-peer based information retrieval system. In Proceedings of the 3rd International Conference on Autonomous Agents and Multiagent Systems, Vol. 1. 456--463.Ming Zhong. 2006. Popularity-biased random walks for peer-to-peer search under the square-root principle. In Proceedings of the 5th International Workshop on Peer-to-Peer Systems

Peer to Peer Information Retrieval: An Overview

Peer-to-peer technology is widely used for file sharing. In the past decade a number of prototype peer-to-peer information retrieval systems have been developed. Unfortunately, none of these have seen widespread real- world adoption and thus, in contrast with file sharing, information retrieval is still dominated by centralised solutions. In this paper we provide an overview of the key challenges for peer-to-peer information retrieval and the work done so far. We want to stimulate and inspire further research to overcome these challenges. This will open the door to the development and large-scale deployment of real-world peer-to-peer information retrieval systems that rival existing centralised client-server solutions in terms of scalability, performance, user satisfaction and freedom

Physics-inspired Performace Evaluation of a Structured Peer-to-Peer Overlay Network

In the majority of structured peer-to-peer overlay networks a graph with a desirable topology is constructed. In most cases, the graph is maintained by a periodic activity performed by each node in the graph to preserve the desirable structure in face of the continuous change of the set of nodes. The interaction of the autonomous periodic activities of the nodes renders the performance analysis of such systems complex and simulation of scales of interest can be prohibitive. Physicists, however, are accustomed to dealing with scale by characterizing a system using intensive variables, i.e. variables that are size independent. The approach has proved its usefulness when applied to satisfiability theory. This work is the first attempt to apply it in the area of distributed systems. The contribution of this paper is two-fold. First, we describe a methodology to be used for analyzing the performance of large scale distributed systems. Second, we show how we applied the methodology to find an intensive variable that describe the characteristic behavior of the Chord overlay network, namely, the ratio of the magnitude of perturbation of the network (joins/failures) to the magnitude of periodic stabilization of the network