Enabling Internet-Scale Publish/Subscribe In Overlay Networks

As the amount of data in todays Internet is growing larger, users are exposed to too much information, which becomes increasingly more difficult to comprehend. Publish/subscribe systems leverage this problem by providing loosely-coupled communications between producers and consumers of data in a network. Data consumers, i.e., subscribers, are provided with a subscription mechanism, to express their interests in a subset of data, in order to be notified only when some data that matches their subscription is generated by the producers, i.e., publishers. Most publish/subscribe systems today, are based on the client/server architectural model. However, to provide the publish/subscribe service in large scale, companies either have to invest huge amount of money for over-provisioning the resources, or are prone to frequent service failures. Peer-to-peer overlay networks are attractive alternative solutions for building Internet-scale publish/subscribe systems. However, scalability comes with a cost: a published message often needs to traverse a large number of uninterested (unsubscribed) nodes before reaching all its subscribers. We refer to this undesirable traffic, as relay overhead. Without careful considerations, the relay overhead might sharply increase resource consumption for the relay nodes (in terms of bandwidth transmission cost, CPU, etc) and could ultimately lead to rapid deterioration of the system’s performance once the relay nodes start dropping the messages or choose to permanently abandon the system. To mitigate this problem, some solutions use unbounded number of connections per node, while some other limit the expressiveness of the subscription scheme. In this thesis work, we introduce two systems called Vitis and Vinifera, for topic-based and content-based publish/subscribe models, respectively. Both these systems are gossip-based and significantly decrease the relay overhead. We utilize novel techniques to cluster together nodes that exhibit similar subscriptions. In the topic-based model, distinct clusters for each topic are constructed, while clusters in the content-based model are fuzzy and do not have explicit boundaries. We augment these clustered overlays by links that facilitate routing in the network. We construct a hybrid system by injecting structure into an otherwise unstructured network. The resulting structures resemble navigable small-world networks, which spans along clusters of nodes that have similar subscriptions. The properties of such overlays make them an ideal platform for efficient data dissemination in large-scale systems. The systems requires only a bounded node degree and as we show, through simulations, they scale well with the number of nodes and subscriptions and remain efficient under highly complex subscription patterns, high publication rates, and even in the presence of failures in the network. We also compare both systems against some state-of-the-art publish/subscribe systems. Our measurements show that both Vitis and Vinifera significantly outperform their counterparts on various subscription and churn scenarios, under both synthetic workloads and real-world traces

Offloading content routing cost from routers

The publish/subscribe paradigm has lately received much attention. In publish/subscribe systems, a specialized event-based middleware delivers notifications of events created by producers (publishers) to consumers (subscribers) interested in that particular event. It is considered a good approach for implementing Internet-wide distributed systems as it provides full decoupling of the communicating parties in time, space and synchronization. One flavor of the paradigm is content-based publish/subscribe which allows the subscribers to express their interests very accurately. In order to implement a content-based publish/subscribe middleware in way suitable for Internet scale, its underlying architecture must be organized as a peer-to-peer network of content-based routers that take care of forwarding the event notifications to all interested subscribers. A communication infrastructure that provides such service is called a content-based network. A content-based network is an application-level overlay network. Unfortunately, the expressiveness of the content-based interaction scheme comes with a price - compiling and maintaining the content-based forwarding and routing tables is very expensive when the amount of nodes in the network is large. The routing tables are usually partially-ordered set (poset) -based data structures. In this work, we present an algorithm that aims to improve scalability in content-based networks by reducing the workload of content-based routers by offloading some of their content routing cost to clients. We also provide experimental results of the performance of the algorithm. Additionally, we give an introduction to the publish/subscribe paradigm and content-based networking and discuss alternative ways of improving scalability in content-based networks. ACM Computing Classification System (CCS): C.2.4 [Computer-Communication Networks]: Distributed Systems - Distributed application

Using publish/subscribe for message routing in mobile environments

Publish/subscribe is a mature communication paradigm to route and deliver events from publishers to interested subscribers. Initially conceived for large scale systems, e.g., the Internet, it has been used more recently in new scenarios, e.g., wireless sensor networks and the Internet of Things (IoT), where mobility and dynamicity are the norm. The loose-coupling and asynchronicity of publish/subscribe makes it an interesting choice for IoT scenarios, i.e., each node in an IoT network can choose a different role depending on its location, capabilities, etc. This paper presents MFT-PubSub, a fully mobile and fault tolerant content-based publish/subscribe protocol. Our proposal is a purely reactive solution for mobility in a publish/subscribe system without any kind of limits on the mobility patterns of the nodes. A wireless ad hoc network is created without the need of any previous connections or knowledge on the nodes. Handling the mobility, be it physical or logical, of both clients and brokers. We prove the validity of our solution by experimentation, and compare it with AODV, a routing protocol for mobile ad hoc networking. The simulations show an improvement on message delivery rate over previously used protocols.Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. Research supported by grant TIN2016-79897-P funded by MCIN/AEI/10.13039/501100011033 and by the European Union, and by the Department of Education, Universities and Research of the Basque Government, grant IT-1437-22 (ADIAN)

Efficient Event Notification Middleware for Smart Microgrids over P2P Networks

© 2018 IEEE. Personal use of this material is permitted. Permissíon from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertisíng or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.[EN] Microgrids are moving towards large-scale smart distributed networks which demand an efficient and reliable communication infrastructure to manage, control and monitor energy resources. With regard to this, publisher/subscriber eventbased middleware has become relevant for large-scale distributed time applications because it allows decouple time and space between senders and receivers. Particularly the content publish/subscribe systems over structured peer-to-peer (P2P) networks has emerged to enhance scalability and dynamism of notification middleware systems. However, this type of systems use multicast routing schemes that still generate much network traffic and as a consequence an overload of the communication channel is produced. This results in inefficient network utilization and rapid depletion of network resources leading to unreliable operations, degradation of system performance and even instability of the microgrid. In this paper, a new content-based publish/subscribe notification middleware over structured P2P systems is proposed, such that smart microgrid communication requirements are met. This proposed system organizes the publications and subscriptions in a one dimensional representation using the Hilbert space filling curve. Through this representation, an innovative routing and matching algorithms are developed. Experimental results demonstrate that the proposed publisher/subscribe system significantly enhance efficiency of the system, network performance and the use of computational resources.This work is supported by the Spanish Ministry of Economy and Competitiveness (MINECO) and the European Regional Development Fund (ERDF) under Grant ENE2015- 64087- C2- 2 R. This work is supported by the Spanish Ministry of Economy and Competitiveness (MINECO) under BES- 2013- 064539.Marzal-Romeu, S.; Salas-Puente, RA.; González-Medina, R.; Garcerá, G.; Figueres Amorós, E. (2018). Efficient Event Notification Middleware for Smart Microgrids over P2P Networks. IEEE Transactions on Smart Grid. https://doi.org/10.1109/TSG.2018.2865432

GRIDKIT: Pluggable overlay networks for Grid computing

A `second generation' approach to the provision of Grid middleware is now emerging which is built on service-oriented architecture and web services standards and technologies. However, advanced Grid applications have significant demands that are not addressed by present-day web services platforms. As one prime example, current platforms do not support the rich diversity of communication `interaction types' that are demanded by advanced applications (e.g. publish-subscribe, media streaming, peer-to-peer interaction). In the paper we describe the Gridkit middleware which augments the basic service-oriented architecture to address this particular deficiency. We particularly focus on the communications infrastructure support required to support multiple interaction types in a unified, principled and extensible manner-which we present in terms of the novel concept of pluggable overlay networks

Automatic subscriptions in publish-subscribe systems

In this paper, we describe how to automate the process of subscribing to complex publish-subscribe systems. We present a proof-of-concept prototype, in which we analyze Web browsing history to generate zero-click subscriptions to Web feeds and video news stories. Our experience so far indicates that user attention data is a promising source of data for automating the subscription process

Quality-constrained routing in publish/subscribe systems

Routing in publish/subscribe (pub/sub) features a communication model where messages are not given explicit destination addresses, but destinations are determined by matching the subscription declared by subscribers. For a dynamic computing environment with applications that have quality demands, this is not sufficient. Routing decision should, in such environments, not only depend on the subscription predicate, but should also take the quality-constraints of applications and characteristics of network paths into account. We identified three abstraction levels of these quality constraints: functional, middleware and network. The main contribution of the paper is the concept of the integration of these constraints into the pub/sub routing. This is done by extending the syntax of pub/sub system and applying four generic, proposed by us, guidelines. The added values of quality-constrained routing concept are: message delivery satisfying quality demands of applications, improvement of system scalability and more optimise use of the network resources. We discuss the use case that shows the practical value of our concept