A Hybrid Multicast-Unicast Infrastructure for Efficient Publish-Subscribe in Enterprise Networks

One of the main challenges in building a large scale publish-subscribe infrastructure in an enterprise network, is to provide the subscribers with the required information, while minimizing the consumed host and network resources. Typically, previous approaches utilize either IP multicast or point-to-point unicast for efficient dissemination of the information. In this work, we propose a novel hybrid framework, which is a combination of both multicast and unicast data dissemination. Our hybrid framework allows us to take the advantages of both multicast and unicast, while avoiding their drawbacks. We investigate several algorithms for computing the best mapping of publishers' transmissions into multicast and unicast transport. Using extensive simulations, we show that our hybrid framework reduces consumed host and network resources, outperforming traditional solutions. To insure the subscribers interests closely resemble those of real-world settings, our simulations are based on stock market data and on recorded IBM WebShpere subscriptions

Middleware’s message : the financial technics of codata

In this paper, I will argue for the relevance of certain distinctive features of messaging systems, namely those in which data (a) can be sent and received asynchronously, (b) can be sent to multiple simultaneous recipients and (c) is received as a “potentially infinite” flow of unpredictable events. I will describe the social technology of the stock ticker, a telegraphic device introduced at the New York Stock Exchange in the 1860s, with reference to early twentieth century philosophers of synchronous experience (Bergson), simultaneous sign interpretations (Mead and Peirce), and flows of discrete events (Bachelard). Then, I will show how the ticker’s data flows developed into the 1990s-era technologies of message queues and message brokers, which distinguished themselves through their asynchronous implementation of ticker-like message feeds sent between otherwise incompatible computers and terminals. These latter systems’ characteristic “publish/subscribe” communication pattern was one in which conceptually centralized (if logically distributed) flows of messages would be “published,” and for which “subscribers” would be spontaneously notified when events of interest occurred. This paradigm—common to the so-called “message-oriented middleware” systems of the late 1990s—would re-emerge in different asynchronous distributed system contexts over the following decades, from “push media” to Twitter to the Internet of Things

Observing the clouds : a survey and taxonomy of cloud monitoring

This research was supported by a Royal Society Industry Fellowship and an Amazon Web Services (AWS) grant. Date of Acceptance: 10/12/2014Monitoring is an important aspect of designing and maintaining large-scale systems. Cloud computing presents a unique set of challenges to monitoring including: on-demand infrastructure, unprecedented scalability, rapid elasticity and performance uncertainty. There are a wide range of monitoring tools originating from cluster and high-performance computing, grid computing and enterprise computing, as well as a series of newer bespoke tools, which have been designed exclusively for cloud monitoring. These tools express a number of common elements and designs, which address the demands of cloud monitoring to various degrees. This paper performs an exhaustive survey of contemporary monitoring tools from which we derive a taxonomy, which examines how effectively existing tools and designs meet the challenges of cloud monitoring. We conclude by examining the socio-technical aspects of monitoring, and investigate the engineering challenges and practices behind implementing monitoring strategies for cloud computing.Publisher PDFPeer reviewe

A decentralized framework for cross administrative domain data sharing

Federation of messaging and storage platforms located in remote datacenters is an essential functionality to share data among geographically distributed platforms. When systems are administered by the same owner data replication reduces data access latency bringing data closer to applications and enables fault tolerance to face disaster recovery of an entire location. When storage platforms are administered by different owners data replication across different administrative domains is essential for enterprise application data integration. Contents and services managed by different software platforms need to be integrated to provide richer contents and services. Clients may need to share subsets of data in order to enable collaborative analysis and service integration. Platforms usually include proprietary federation functionalities and specific APIs to let external software and platforms access their internal data. These different techniques may not be applicable to all environments and networks due to security and technological restrictions. Moreover the federation of dispersed nodes under a decentralized administration scheme is still a research issue. This thesis is a contribution along this research direction as it introduces and describes a framework, called \u201cWideGroups\u201d, directed towards the creation and the management of an automatic federation and integration of widely dispersed platform nodes. It is based on groups to exchange messages among distributed applications located in different remote datacenters. Groups are created and managed using client side programmatic configuration without touching servers. WideGroups enables the extension of the software platform services to nodes belonging to different administrative domains in a wide area network environment. It lets different nodes form ad-hoc overlay networks on-the-fly depending on message destinations located in distinct administrative domains. It supports multiple dynamic overlay networks based on message groups, dynamic discovery of nodes and automatic setup of overlay networks among nodes with no server-side configuration. I designed and implemented platform connectors to integrate the framework as the federation module of Message Oriented Middleware and Key Value Store platforms, which are among the most widespread paradigms supporting data sharing in distributed systems

A Scalable Communication Architecture for Advanced Metering Infrastructure

Advanced Metering Infrastructure (AMI), seen as foundation for overall grid modernization, is an integration of many technologies that provides an intelligent connection between consumers and system operators [ami 2008]. One of the biggest challenge that AMI faces is to scalable collect and manage a huge amount of data from a large number of customers. In our paper, we address this challenge by introducing a mixed peer-to-peer (P2P) and client-server communication architecture for AMI in which metering data is aggregated and processed distributedly at multiple levels and in a tree-like manner. Through analysis we show that the architecture is featured with load scalability, resiliency with failure and partly self-organization. The experi- ments performed in large scale French Grid5000 platform [G5k] shows the communication efficiency in the proposed architecture

Software Defined Networks based Smart Grid Communication: A Comprehensive Survey

The current power grid is no longer a feasible solution due to ever-increasing user demand of electricity, old infrastructure, and reliability issues and thus require transformation to a better grid a.k.a., smart grid (SG). The key features that distinguish SG from the conventional electrical power grid are its capability to perform two-way communication, demand side management, and real time pricing. Despite all these advantages that SG will bring, there are certain issues which are specific to SG communication system. For instance, network management of current SG systems is complex, time consuming, and done manually. Moreover, SG communication (SGC) system is built on different vendor specific devices and protocols. Therefore, the current SG systems are not protocol independent, thus leading to interoperability issue. Software defined network (SDN) has been proposed to monitor and manage the communication networks globally. This article serves as a comprehensive survey on SDN-based SGC. In this article, we first discuss taxonomy of advantages of SDNbased SGC.We then discuss SDN-based SGC architectures, along with case studies. Our article provides an in-depth discussion on routing schemes for SDN-based SGC. We also provide detailed survey of security and privacy schemes applied to SDN-based SGC. We furthermore present challenges, open issues, and future research directions related to SDN-based SGC.Comment: Accepte

Interoperable communication framework for bridging RESTful and topic-based communication in IoT

The promise of the Internet of Things (IoT) and the many visions of unprecedented and transforming IoT applications are challenged by the realities of a highly fragmented ecosystem of devices, standards and industries. Systems research in IoT is shifting priorities to explore explicit “thing architectures” that promote and enable the friction-free interactions of things despite such fragmentations. In this paper, we focus on overcoming light-weight communication protocol fragmentation. We introduce the Atlas IoT communication framework which enables interactions among things that speak similar or different communication protocols. The framework tools up Atlas things with protocol translator “attachments” that could be either hosted on board the Atlas thing platform, or in the cloud. The translator enables the seamless communication between heterogeneous things through a set of well-defined interfaces. The proposed framework supports seamless communication among the widely adopted Constrained Application Protocol (CoAP), Representational State Transfer (REST) over Hypertext Transfer protocol HTTP, and the Message Queue Telemetry Transport protocol (MQTT). Our framework is carefully designed to facilitate interoperability among heterogeneously communicating things without taxing the performance of things that are homogenously communicating. The framework itself utilizes the topic concept and uses a meta-topic hierarchy to map out and guide the translations. We present the details of the Atlas IoT communication framework and give a detailed benchmarking study to measure the energy consumption and code footprint characteristics of the different aspects of the framework on real hardware platforms. In addition to basic characterizations, we compare our framework to the Eclipse Ponte framework and show how our framework is advantageous in energy consumption and how it is unique in that it does not tangibly penalize the homogeneous communication case

Middleware services for distributed virtual environments

PhD ThesisDistributed Virtual Environments (DVEs) are virtual environments which allow dispersed users to interact with each other and the virtual world through the underlying network. Scalability is a major challenge in building a successful DVE, which is directly affected by the volume of message exchange. Different techniques have been deployed to reduce the volume of message exchange in order to support large numbers of simultaneous participants in a DVE. Interest management is a popular technique for filtering unnecessary message exchange between users. The rationale behind interest management is to resolve the "interests" of users and decide whether messages should be exchanged between them. There are three basic interest management approaches: region-based, aura-based and hybrid approaches. However, if the time taken for an interest management approach to determine interests is greater than the duration of the interaction, it is not possible to guarantee interactions will occur correctly or at all. This is termed the Missed Interaction Problem, which all existing interest management approaches are susceptible to. This thesis provides a new aura-based interest management approach, termed Predictive Interest management (PIM), to alleviate the missed interaction problem. PIM uses an enlarged aura to detect potential aura-intersections and iii initiate message exchange. It utilises variable message exchange frequencies, proportional to the intersection degree of the objects' expanded auras, to restrict bandwidth usage. This thesis provides an experimental system, the PIM system, which couples predictive interest management with the de-centralised server communication model. It utilises the Common Object Request Broker Architecture (CORBA) middleware standard to provide an interoperable middleware for DVEs. Experimental results are provided to demonstrate that PIM provides a scalable interest management approach which alleviates the missed interaction problem