Topic-based Delivery of Event Messages in Peer-to-Peer Publish/Subscribe Systems

An event-driven publish/subscribe (PS) model of a distributed system is used in various applications.In this paper, we discuss a peer-to-peer (P2P) model of a topic-based PS system (P2PPS model) where each peer process (peer) can both subscribe interesting topics and publish event messages with topics. In our previous studies, we propose the TBC (topic-based-causally delivery) protocol a homogeneous networkswhere maximum delay time between every pair of peers is same. Here, a pair of event messages are checked if the event messages are related in the topic vector and are causally delivered by takingadvantage of physical time and linear time. In this paper, we consider a system where peers are interconnectedin heterogeneous networks. Here, maximum delay time between every pair of peers is not same.We propose a heterogeneous TBC (HTBC) protocol where event messages are TBC-causally delivered to target peers in heterogeneous network. We evaluate the HTBC protocol and show the number of pair of event messages unnecessarily ordered is reduced in the HTBC protocol.Key Words : Publish/subscribe (PS) systems, Peer-to-peer (P2P) model, Topic-based causal dependency, Topic vectors, Heterogeneous TBC (HTBC) protocol, Linear clock, Physical clock, Heterogeneous networks

Object-based Information Flow Control in Peer-to-peer Publish/Subscribe Systems

Distributed systems are getting so scalable like IoT (Internet of Things) and P2P (Peer-to-Peer) systems that millions of devices are connected and support various types of applications. Here, distributed systems are required to be secure in addition to increasing the performance, reliability, and availability and reducing the energy consumption. In distributed systems, information in objects flows to other objects by transactions reading and writing data in the objects. Here, some information of an object may illegally flow to a subject which is not allowed to get the information of the object. Especially, a leakage of sensitive information is to be prevented from occurring. In order to keep information systems secure, illegal information flow among objects has to be prevented. Types of synchronization protocols are so far discussed based on read and write access rights in the RBAC (Role-Based Access Control) model to prevent illegal information flow.In this thesis, we newly propose a P2PPSO (P2P type of topic-based PS (Publish/Subscribe) with Object concept) model and discuss the models and protocols for information flow control. A P2PPSO model is composed of peer processes (peers) which communicate with one another by publishing and subscribing event messages. Each peer can both publish and receive event messages with no centralized coordinator compared with traditional centralized PS models. Each event message published by a source peer carries information to a target peer. The contents carried by an event message are considered to be composed of objects. An object is a unit of data resource. Objects are characterized by topics, and each event message is also characterized by topics named publication topics.In order to make a P2PPSO system secure, we first newly propose a TBAC (Topic-Based Access Control) model. Here, an access right is a pair ⟨t, op⟩ of a topic t and a publish or subscribe operation op. A peer is allowed to publish an event message with publication topics and subscribe interesting topics only if the publication and subscription access rights are granted to the peer, respectively. Suppose an event message e_j published by a peer p_j carries an object on some topics into a target peer p_i. Here, information in the peer p_j illegally flows to the peer p_i if the target peer p_i is not allowed to subscribe the topics. An illegal object is an object whose topics a target peer is not allowed to subscribe. Even if an event message is received by a target peer by checking topics, objects carried by the event message may be illegal at the target peer. Hence, first, we propose a TOBS (Topics-of-Objects-Based Synchronization) protocol to prevent target peers from being delivered illegal objects in the P2PPSO system. Here, even if an event message is received by a target peer, illegal objects in the event message are not delivered to the target peer.In the TOBS protocol, every event message is assumed to be causally delivered to every common target peer in the underlying network. Suppose an event message e_2 is delivered to a target peer p_i before another event message e_1 while the event message e_1 causally precedes the event message e_2 (e_1 →_c e_2). Here, the event message e_2 is premature at the peer p_i. Hence, secondly, we propose a TOBSCO (TOBS with Causally Ordering delivery) protocol where the function to causally deliver every pair of event messages is added to the TOBS protocol. Here, we assume the underlying network supports reliable communication among every pair of peers, i.e. no event message loss, no duplicate message, and the sending order delivery of messages. Every pair of event messages received by using topics are causally delivered to every common target peer by using the vector of sequence numbers.In the TOBS and TOBSCO protocols, objects delivered to target peers are held as replicas of the objects by the target peers. If a peer updates data of an object, the peer distributes event messages, i.e. update event messages, to update every replica of the object obtained by other peers. If a peer updates an object without changing topics, the object is referred to as altered. Here, an update event message for the altered object is meaningless since peers check only topics to exchange event messages. Hence, thirdly, we propose an ETOBSCO (Efficient TOBSCO) protocol where update event messages of objects are published only if topics of the objects are updated to reduce the network overhead.In the evaluation, first, we show how many numbers of event messages and objects are prevented from being delivered to target peers in the TOBS protocol. Next, we show every pair of event messages are causally delivered but it takes longer to deliver event messages in the TOBSCO protocol than the TOBS protocol. Finally, we show the fewer number of event messages are delivered while it takes longer to update replicas of altered objects in the ETOBSCO protocol than the TOBSCO protocol.博士(工学)法政大学 (Hosei University

Virtual Central Control.

A distributed system is a set of cooperating computers (processes) communicating with each other to achieve a common goal. They are broadly classified as centralized and decentralised systems. In a centralized system, a single computer plays the role of central coordinator and controls all the system activities. Whereas in a decentralized system, all the cooperating processes have an equal role to play, therefore solutions to problems involve instructing all the processes and coordinating their actions. Although a centralized system facilitates program development, it has serious drawbacks. If the coordinator fails, the system effectively breaks down. Also, the coordinator can become a performance bottleneck. On the other hand, a decentralized system does not suffer from these shortcomings, but program development is more difficult. In this research, we develop a paradigm for a distributed system that provides the view of a centralized system even though the underlying system is decentralized. Special processes called agents reside on all participating computers. One of the agents acts as a leader and coordinates activities of other agents. Agents communicate through a fault-tolerant agent-to-agent protocol. The paradigm defines a one-shot computation, a construct that enables the expression of programs in a simulated centralized environment, thus making program development easier. Programs, written in terms of one-shot computations that are encapsulated in templates which hide all the lower level details from the programmer. We show that every computation can be expressed as a one-shot computation. The proposed paradigm has a high degree of fault-tolerance. It tolerates up to $n-1$ failures in a system of n processors. A prototype has been implemented using PVM

Load sharing for optimistic parallel simulations on multicore machines

Parallel Discrete Event Simulation (PDES) is based on the partitioning of the simulation model into distinct Logical Processes (LPs), each one modeling a portion of the entire system, which are allowed to execute simulation events concurrently. This allows exploiting parallel computing architectures to speedup model execution, and to make very large models tractable. In this article we cope with the optimistic approach to PDES, where LPs are allowed to concurrently process their events in a speculative fashion, and rollback/ recovery techniques are used to guarantee state consistency in case of causality violations along the speculative execution path. Particularly, we present an innovative load sharing approach targeted at optimizing resource usage for fruitful simulation work when running an optimistic PDES environment on top of multi-processor/multi-core machines. Beyond providing the load sharing model, we also define a load sharing oriented architectural scheme, based on a symmetric multi-threaded organization of the simulation platform. Finally, we present a real implementation of the load sharing architecture within the open source ROme OpTimistic Simulator (ROOT-Sim) package. Experimental data for an assessment of both viability and effectiveness of our proposal are presented as well. Copyright is held by author/owner(s)

Distributed shared memory for virtual environments

Bibliography: leaves 71-77.This work investigated making virtual environments easier to program, by designing a suitable distributed shared memory system. To be usable, the system must keep latency to a minimum, as virtual environments are very sensitive to it. The resulting design is push-based and non-consistent. Another requirement is that the system should be scaleable, over large distances and over large numbers of participants. The latter is hard to achieve with current network protocols, and a proposal was made for a more scaleable multicast addressing system than is used in the Internet protocol. Two sample virtual environments were developed to test the ease-of-use of the system. This showed that the basic concept is sound, but that more support is needed. The next step should be to extend the language and add compiler support, which will enhance ease-of-use and allow numerous optimisations. This can be improved further by providing system-supported containers

Semantically reliable group communication

A utilização de computadores e redes de transmissão de dados em diversas aplicações do quotidiano, torna desejável a adopção de técnicas de tolerância a faltas em sistemas baseados em hardware e software não especializados. A comunicação em grupo é, neste contexto, uma tecnologia particularmente atraente, pois oferece ao programador garantias de fiabilidade que simplificam significativamente a aplicação de técnicas de tolerância a faltas. No entanto, a experiência tem mostrado que a concretização deste modelo em sistemas heterogéneos e de grande escala levanta problemas de desempenho. Embora as limitações de desempenho possam ser evitadas através de um relaxamento das garantias de fiabilidade, os protocolos resultantes são normalmente menos úteis, nomeadamente, na replicação com coerência forte. O desafio reside pois no relaxamento das garantias de fiabilidade sem deixar de oferecer um modelo adequado à programação de aplicações tolerantes a faltas. Esta dissertação estuda modelos e mecanismos que permitem conciliar as vantagens da comunicação em grupo com o elevado desempenho, recorrendo para isso ao enfraquecimento selectivo das garantias oferecidas pelos protocolos. A nossa proposta consiste no uso pelo protocolo de informação sobre a semântica das mensagens, por forma a escolher quais delas têm que ser fiavelmente transmitidas, daí a fiabilidade semântica. Em diversas aplicações, algumas mensagens revogam ou transmitem implicitamente outras mensagens enviadas recentemente, tornando-as obsoletas durante a sua transmissão. Ao omitir apenas as mensagens obsoletas, o desempenho pode ser melhorado sem impacto na correcção da aplicação. São apresentados as especificações e os algoritmos de um conjunto protocolos de comunicação em grupo com fiabilidade semântica, incluindo ordenação e sincronismo virtual. Os protocolos são então avaliados com um modelo analítico, um modelo de simulação e um protótipo. A discussão de uma aplicação concreta ilustra a interface de programação e o desempenho resultanteCurrent usage of computers and data communication networks for a variety of daily tasks, calls for widespread deployment of fault tolerance techniques with inexpensive off-the-shelf hardware and software. Group communication is in this context a particularly appealing technology, as it provides to the application programmer reliability guarantees that highly simplify many fault tolerance techniques. It has however been reported that the performance of group communication toolkits in large and heterogeneous systems is frequently disappointing. Although this can be overcome by relaxing reliability guarantees, the resulting protocol is often much less useful than group communication, in particular, for strong consistent replication. The challenge is thus to relax reliability and still provide a convenient set of guarantees for fault tolerant programming. This thesis addresses models and mechanisms that by selectively relaxing reliability guarantees, offer both the convenience of group communication for fault tolerant programming and high performance. The key to our proposal is to use knowledge about the semantics of messages exchanged to determine which messages need to be reliably delivered, hence semantic reliability. In many applications, some messages implicitly convey or overwrite other messages sent recently before, making them obsolete while still in transit. By omitting only the delivery of obsolete messages, performance can be improved without impact on the correctness of the application. Specifications and algorithms for a complete semantically reliable group communication protocol suite are introduced, encompassing ordered and view synchronous multicast. The protocols are then evaluated with analytical and simulation models and with a prototype implementation. The discussion of a concrete application illustrates the resulting programming interface and performance.Fundação para a Ciência e a Tecnologia - SHIFT (POSI/32869/CHS/2000)

Atomic Broadcast in Heterogeneous Distributed Systems

Communication services have long been recognized as possessing a dominant effect on both performance and robustness of distributed systems. Distributed applications rely on a multitude of protocols for the support of these services. Of crucial importance are multicast protocols. Reliable multicast protocols enhance the efficiency and robustness of distributed systems. Numerous reliable multicast protocols have been proposed, each differing in the set of assumptions adopted, especially for the communication network. These assumptions make each protocol suitable for a specific environment. The presence of different distributed applications that run on different LANs and single distributed applications that span different LANs mandate interaction between protocols on these LANs. This interaction is driven by the necessity of cooperation between individual applications. The state of the art in reliable multicast protocols renders itself inadequate for multicasting in interconnected LANs. The progress in development methodology for efficient and robust LAN software has not been matched by similar advances for WANs. A high-latency, a lower bandwidth, a higher probability of partitions, and a frequent loss of messages are the main restrictive barriers. In our work, we propose a global standard protocol that orchestrates cooperation between the different reliable broadcast protocols that run on different LANs. Our objective is to support a reliable ordered delivery service for inter-LAN messages and achieve the utmost utilization of the underlying local communication services. Our protocol suite accommodates the existence of LANs managed by autonomous authorities. To uphold this autonomy (as a defacto condition), LANs under different authorities must be able to adopt different ordering criteria for group multicasting. The developed suite assumes an environment in which multicasting groups can have members that belong to different LANs; each group can adopt either total or causal order for message delivery to its members. We also recognize the need for interaction between different reliable multicasting protocols. This interaction is a necessity in an autonomous environment in which each local authority selects a protocol that is suitable to its individual needs. Our protocols are capable of interacting with any reliable protocol that achieves a causal order as well as with all timestamp-based total-order protocols. Our protocols can also be used as a medium for interaction between existing reliable multicasting protocols. This feature opens new avenues in interactability between reliable multicasting protocols. Finally, our protocol suite enjoys a communication structure that can be aligned with the actual routing topology, which largely minimizes the necessary protocol messages

ZeroComm: Decentralized, Secure and Trustful Group Communication

In the context of computer networks, decentralization is a network architecture that distributes both workload and control of a system among a set of coequal participants. Applications based on such networks enhance trust involved in communication by eliminating the external author- ities with self-interests, including governments and tech companies. The decentralized model delegates the ownership of data to individual users and thus mitigates undesirable behaviours such as harvesting personal information by external organizations. Consequently, decentral- ization has been adopted as the key feature in the next generation of the Internet model which is known as Web 3.0. DIDComm is a set of abstract protocols which enables secure messaging with decentralization and thus serves for the realization of Web 3.0 networks. It standardizes and transforms existing network applications to enforce secure, trustful and decentralized com- munication. Prior work on DIDComm has only been restricted to pair-wise communication and hence it necessitates a feasible strategy for adapting the Web 3.0 concepts in group-oriented networks. Inspired by the demand for a group communication model in Web 3.0, this study presents Zero- Comm which preserves decentralization, security and trust throughout the fundamental opera- tions of a group such as messaging and membership management. ZeroComm is built atop the publisher-subscriber pattern which serves as a messaging architecture for enabling communi- cation among multiple members based on the subjects of their interests. This is realized in our implementation through ZeroMQ, a low-level network library that facilitates the construction of advanced and distributed messaging patterns. The proposed solution leverages DIDComm protocols to deliver safe communication among group members at the expense of performance and efficiency. ZeroComm offers two different modes of group communication based on the organization of relationships among members with a compromise between performance and security. Our quantitative analysis shows that the proposed model performs efficiently for the messaging operation whereas joining a group is a relatively exhaustive procedure due to the es- tablishment of secure and decentralized relationships among members. ZeroComm primarily serves as a low-level messaging framework but can be extended with advanced features such as message ordering, crash recovery of members and secure routing of messages

Benchmarking MongoDB multi-document transactions in a sharded cluster

Relational databases like Oracle, MySQL, and Microsoft SQL Server offer trans- action processing as an integral part of their design. These databases have been a primary choice among developers for business-critical workloads that need the highest form of consistency. On the other hand, the distributed nature of NoSQL databases makes them suitable for scenarios needing scalability, faster data access, and flexible schema design. Recent developments in the NoSQL database community show that NoSQL databases have started to incorporate transactions in their drivers to let users work on business-critical scenarios without compromising the power of distributed NoSQL features [1]. MongoDB is a leading document store that has supported single document atomicity since its first version. Sharding is the key technique to support the horizontal scalability in MongoDB. The latest version MongoDB 4.2 enables multi-document transactions to run on sharded clusters, seeking both scalability and ACID multi- documents. Transaction processing is a novel feature in MongoDB, and benchmarking the performance of MongoDB multi-document transactions in sharded clusters can encourage developers to use ACID transactions for business-critical workloads. We have adapted pytpcc framework to conduct a series of benchmarking experi- ments aiming at finding the impact of tunable consistency, database size, and design choices on the multi-document transaction in MongoDB sharded clusters. We have used TPC’s OLTP workload under a variety of experimental settings to measure business throughput. To the best of our understanding, this is the first attempt towards benchmarking MongoDB multi-document transactions in a sharded cluster