Session Management in Multicast

As a new network technique to efficiently distribute information from a small number of senders to large numbers of receivers, multicast encounters many problems in scalability, membership management, security, etc. These problems hinder the deployment of multicast technology in commercial applications. To overcome these problems, a more general solution for multicast technology is needed. In this paper, after studying current multicast technologies, we summarized the technical requirements for multicast, including data delivery, scalability, security, group management, reliability, and deployment. In order to understand and meet the requirements, we define a life cycle model that most multicast sessions should follow. According to the requirements and the life cycle model, we propose and design a general solution that can control each phase of a session and satisfy most requirements for multicast technology. This general solution has three parts: hierarchical topology auto-configuration algorithm, Session Management Mechanism, and techniques supporting different multicast protocols. To verify the feasibility of our solution and compare its performance with other multicast techniques, we simulate our solution and compare it with PIM-SM and ESM

A Semantic-Based Middleware for Multimedia Collaborative Applications

The Internet growth and the performance increase of desktop computers have enabled large-scale distributed multimedia applications. They are expected to grow in demand and services and their traffic volume will dominate. Real-time delivery, scalability, heterogeneity are some requirements of these applications that have motivated a revision of the traditional Internet services, the operating systems structures, and the software systems for supporting application development. This work proposes a Java-based lightweight middleware for the development of large-scale multimedia applications. The middleware offers four services for multimedia applications. First, it provides two scalable lightweight protocols for floor control. One follows a centralized model that easily integrates with centralized resources such as a shared too], and the other is a distributed protocol targeted to distributed resources such as audio. Scalability is achieved by periodically multicasting a heartbeat that conveys state information used by clients to request the resource via temporary TCP connections. Second, it supports intra- and inter-stream synchronization algorithms and policies. We introduce the concept of virtual observer, which perceives the session as being in the same room with a sender. We avoid the need for globally synchronized clocks by introducing the concept of user\u27s multimedia presence, which defines a new manner for combining streams coming from multiple sites. It includes a novel algorithm for estimation and removal of clock skew. In addition, it supports event-driven asynchronous message reception, quality of service measures, and traffic rate control. Finally, the middleware provides support for data sharing via a resilient and scalable protocol for transmission of images that can dynamically change in content and size. The effectiveness of the middleware components is shown with the implementation of Odust, a prototypical sharing tool application built on top of the middleware

DIVE on the internet

This dissertation reports research and development of a platform for Collaborative Virtual Environments (CVEs). It has particularly focused on two major challenges: supporting the rapid development of scalable applications and easing their deployment on the Internet. This work employs a research method based on prototyping and refinement and promotes the use of this method for application development. A number of the solutions herein are in line with other CVE systems. One of the strengths of this work consists in a global approach to the issues raised by CVEs and the recognition that such complex problems are best tackled using a multi-disciplinary approach that understands both user and system requirements. CVE application deployment is aided by an overlay network that is able to complement any IP multicast infrastructure in place. Apart from complementing a weakly deployed worldwide multicast, this infrastructure provides for a certain degree of introspection, remote controlling and visualisation. As such, it forms an important aid in assessing the scalability of running applications. This scalability is further facilitated by specialised object distribution algorithms and an open framework for the implementation of novel partitioning techniques. CVE application development is eased by a scripting language, which enables rapid development and favours experimentation. This scripting language interfaces many aspects of the system and enables the prototyping of distribution-related components as well as user interfaces. It is the key construct of a distributed environment to which components, written in different languages, connect and onto which they operate in a network abstracted manner. The solutions proposed are exemplified and strengthened by three collaborative applications. The Dive room system is a virtual environment modelled after the room metaphor and supporting asynchronous and synchronous cooperative work. WebPath is a companion application to a Web browser that seeks to make the current history of page visits more visible and usable. Finally, the London travel demonstrator supports travellers by providing an environment where they can explore the city, utilise group collaboration facilities, rehearse particular journeys and access tourist information data

Media handling for conferencing in MANETs

Mobile Ad hoc NETworks (MANETs) are formed by devices set up temporarily to communicate without using a pre-existing network infrastructure. Devices in these networks are disparate in terms of resource capabilities (e.g. processing power, battery energy). Multihop Cellular Networks (MCNs) incorporate multihop mobile ad-hoc paradigms into 3G conventional single-hop cellular networks. Conferencing, an essential category of applications in MANETs and MCNs, includes popular applications such as audio/video conferencing. It is defined as an interactive multimedia service comprising online exchange of multimedia content among several users. Conferencing requires two sessions: a call signaling session and a media handling session. Call signaling is used to set up, modify, and tear down conference sessions. Media handling deals with aspects such as media transportation, media mixing, and transcoding. In this thesis, we are concerned with media handling for conferencing in MANETs and MCNs. We propose an architecture based on two overlay networks: one for mixing and one for control. The first overlay is composed of nodes acting as mixers. Each node in the network has a media connection with one mixer in the first overlay. A novel distributed mixing architecture that minimizes the number of mixers in end-to-end paths is proposed as an architectural solution for this first overlay. A sub-network of nodes, called controllers, composes the second overlay. Each controller controls a set of mixers, and collectively, they manage and control the two-overlay network. The management and control tasks are assured by a media signaling architecture based on an extended version of Megaco/H.L248. The two-overlay network is self-organizing, and thus automatically assigns users to mixers, controls mixers and controllers, and recovers the network from failures. We propose a novel self-organizing scheme that has three components: self-growing, self-shrinking and self-healing. Self-growing and self-shrinking use novel workload balancing schemes that make decisions to enable and disable mixers and controllers. The workload balancing schemes use resources efficiently by balancing the load among the nodes according to their capabilities. Self-healing detects failed nodes and recovers the network when failures of nodes with responsibilities (mixers and controllers) occur. Detection of failed nodes is based on a novel application-level failure detection architecture. A novel architecture for media handling in MCNs is proposed. We use mediator concepts to connect the media handling entities of a MANET with the media entities of a 3G cellular network. A media mediator assures signaling and media connectivity between the two networks and acts as a translator of the different media handling protocols

Reliable Multicast in Mobile Ad Hoc Wireless Networks

A mobile wireless ad hoc network (MANET) consists of a group of mobile nodes communicating wirelessly with no fixed infrastructure. Each node acts as source or receiver, and all play a role in path discovery and packet routing. MANETs are growing in popularity due to multiple usage models, ease of deployment and recent advances in hardware with which to implement them. MANETs are a natural environment for multicasting, or group communication, where one source transmits data packets through the network to multiple receivers. Proposed applications for MANET group communication ranges from personal network apps, impromptu small scale business meetings and gatherings, to conference, academic or sports complex presentations for large crowds reflect the wide range of conditions such a protocol must handle. Other applications such as covert military operations, search and rescue, disaster recovery and emergency response operations reflect the mission critical nature of many ad hoc applications. Reliable data delivery is important for all categories, but vital for this last one. It is a feature that a MANET group communication protocol must provide. Routing protocols for MANETs are challenged with establishing and maintaining data routes through the network in the face of mobility, bandwidth constraints and power limitations. Multicast communication presents additional challenges to protocols. In this dissertation we study reliability in multicast MANET routing protocols. Several on-demand multicast protocols are discussed and their performance compared. Then a new reliability protocol, R-ODMRP is presented that runs on top of ODMRP, a well documented best effort protocol with high reliability. This protocol is evaluated against ODMRP in a standard network simulator, ns-2. Next, reliable multicast MANET protocols are discussed and compared. We then present a second new protocol, Reyes, also a reliable on-demand multicast communication protocol. Reyes is implemented in the ns-2 simulator and compared against the current standards for reliability, flooding and ODMRP. R-ODMRP is used as a comparison point as well. Performance results are comprehensively described for latency, bandwidth and reliable data delivery. The simulations show Reyes to greatly outperform the other protocols in terms of reliability, while also outperforming R-ODMRP in terms of latency and bandwidth overhead

Evaluating the performance of distributed agreement algorithms:tools, methodology and case studies

Nowadays, networked computers are present in most aspects of everyday life. Moreover, essential parts of society come to depend on distributed systems formed of networked computers, thus making such systems secure and fault tolerant is a top priority. If the particular fault tolerance requirement is high availability, replication of components is a natural choice. Replication is a difficult problem as the state of the replicas must be kept consistent even if some replicas fail, and because in distributed systems, relying on centralized control or a certain timing behavior is often not feasible. Replication in distributed systems is often implemented using group communication. Group communication is concerned with providing high-level multipoint communication primitives and the associated tools. Most often, an emphasis is put on tolerating crash failures of processes. At the heart of most communication primitives lies an agreement problem: the members of a group must agree on things like the set of messages to be delivered to the application, the delivery order of messages, or the set of processes that crashed. A lot of algorithms to solve agreement problems have been proposed and their correctness proven. However, performance aspects of agreement algorithms have been somewhat neglected, for a variety of reasons: the lack of theoretical and practical tools to help performance evaluation, and the lack of well-defined benchmarks for agreement algorithms. Also, most performance studies focus on analyzing failure free runs only. In our view, the limited understanding of performance aspects, in both failure free scenarios and scenarios with failure handling, is an obstacle for adopting agreement protocols in practice, and is part of the explanation why such protocols are not in widespread use in the industry today. The main goal of this thesis is to advance the state of the art in this field. The thesis has major contributions in three domains: new tools, methodology and performance studies. As for new tools, a simulation and prototyping framework offers a practical tool, and some new complexity metrics a theoretical tool for the performance evaluation of agreement algorithms. As for methodology, the thesis proposes a set of well-defined benchmarks for atomic broadcast algorithms (such algorithms are important as they provide the basis for a number of replication techniques). Finally, three studies are presented that investigate important performance issues with agreement algorithms. The prototyping and simulation framework simplifies the tedious task of developing algorithms based on message passing, the communication model that most agreement algorithms are written for. In this framework, the same implementation can be reused for simulations and performance measurements on a real network. This characteristic greatly eases the task of validating simulation results with measurements (or vice versa). As for theoretical tools, we introduce two complexity metrics that predict performance with more accuracy than the traditional time and message complexity metrics. The key point is that our metrics take account for resource contention, both on the network and the hosts; resource contention is widely recognized as having a major impact on the performance of distributed algorithms. Extensive validation studies have been conducted. Currently, no widely accepted benchmarks exist for agreement algorithms or group communication toolkits, which makes comparing performance results from different sources difficult. In an attempt to consolidate the situation, we define a number of benchmarks for atomic broadcast. Our benchmarks include well-defined metrics, workloads and failure scenarios (faultloads). The use of the benchmarks is illustrated in two detailed case studies. Two widespread mechanisms for handling failures are unreliable failure detectors which provide inconsistent information about failures, and a group membership service which provides consistent information about failures, respectively. We analyze the performance tradeoffs of these two techniques, by comparing the performance of two atomic broadcast algorithms designed for an asynchronous system. Based on our results, we advocate a combined use of the two approaches to failure handling. In another case study, we compare two consensus algorithms designed for an asynchronous system. The two algorithms differ in how they coordinate the decision process: the one uses a centralized and the other a decentralized communication schema. Our results show that the performance tradeoffs are highly affected by a number of characteristics of the environment, like the availability of multicast and the amount of contention on the hosts versus the amount of contention on the network. Famous theoretical results state that a lot of important agreement problems are not solvable in the asynchronous system model. In our third case study, we investigate how these results are relevant for implementations of a replicated service, by conducting an experiment in a local area network. We exposed a replicated server to extremely high loads and required that the underlying failure detection service detects crashes very fast; the latter is important as the theoretical results are based on the impossibility of reliable failure detection. We found that our replicated server continued working even with the most extreme settings. We discuss the reasons for the robustness of our replicated server

Secure, Efficient and Privacy-aware Framework for Unstructured Peer-to-Peer Networks

Recently, the advances in Ubiquitous Computing networks and the increased computational power of network devices have led designers to create more flexible distributed network models using decentralised network management systems. Security, resilience and privacy issues within such distributed systems become more complicated while important tasks such as routing, service access and state management become increasingly challenging. Low-level protocols over ubiquitous decentralised systems, which provide autonomy to network nodes, have replaced the traditional client-server arrangements in centralised systems. Small World networks represent a model that addresses many existing challenges within Ubiquitous Computing networks. Therefore, it is imperative to study the properties of Small World networks to help understanding, modelling and improving the performance, usability and resiliency of Ubiquitous Computing networks. Using the network infrastructure and trusted relationships in the Small World networks, this work proposes a framework to enhance security, resilience and trust within scalable Peer-to-Peer (P2P) networks. The proposed framework consists of three major components namely network-aware topology construction, anonymous global communication using community trust, and efficient search and broadcasting based on granularity and pro-active membership management. We utilise the clustering co-efficient and conditional preferential attachment to propose a novel topology construction scheme that organises nodes into groups of trusted users to improve scalability. Network nodes communicate locally without advertising node identity at a global scale, which ensures user anonymity. The global communication is organised and facilitated by Service Centres to maintain security, privacy and integrity of member nodes. Service Centres are allocated using a novel leader election mechanism within unstructured scalable P2P networks. This allows providing fair and equitable access for existing and new nodes without having to make complex changes to the network topology. Moreover, the scale-free and clustering co-efficient characteristics of Small World networks help organising the network layout to maintain its balance in terms of the nodes distribution. Simulation results show that the proposed framework ensures better scalability and membership management in unstructured P2P networks, and improves the performance of the search and broadcasting in terms of the average shortest path and control overhead while maintaining user anonymity and system resiliency

Design and implementation of a QoS-Supportive system for reliable multicast

As the Internet is increasingly being used by business companies to offer and procure services, providers of networked system services are expected to assure customers of specific Quality of Service (QoS) they could offer. This leads to scenarios where users prefer to negotiate required QoS guarantees prior to accepting a service, and service providers assess their ability to provide the customer with the requested QoS on the basis of existing resource availability. A system to be deployed in such scenarios should, in addition to providing the services, (i) monitor resource availability, (ii) be able to assess whether or not requested QoS can be met, and (iii) adapt to QoS perturbations (e.g., node failures) which undermine any assumptions made on continued resource availability. This thesis focuses on building such a QoS-Supportive system for reliably multicasting messages within a group of crash-prone nodes connected by loss-prone networks. System design involves developing a Reliable Multicast protocol and analytically estimating the multicast performance in terms of protocol parameters. It considers two cases regarding message size: small messages that fit into a single packet and large ones that need to be fragmented into multiple packets. Analytical estimations are obtained through stochastic modelling and approximation, and their accuracy is demonstrated using simulations. They allow the affordability of the requested QoS to be numerically assessed for a given set of performance metrics of the underlying network, and also indicate the values to be used for the protocol parameters if the affordable QoS is to be achieved. System implementation takes a modular approach and the major sub-systems built include: the QoS negotiation component, the network monitoring component and the reliable multicast protocol component. Two prototypes have been built. The first one is built as a middleware system in itself to the extent of testing our ideas over a group of geographically distant nodes using PlanetLab. The second prototype is developed as a part of the JGroups Reliable Communication Toolkit and provides, besides an example of scenario directly benefitting of such technology, an example integration of our subsystem into an already-existing system.EThOS - Electronic Theses Online ServiceTAPAS EU-IST-2001-34069 Project : EPSR (Engineering and Physical Sciences Research Council)GBUnited Kingdo

Mechanisms for improving ZooKeeper Atomic Broadcast performance

PhD ThesisCoordination services are essential for building higher-level primitives that are often used in today’s data-center infrastructures, as they greatly facilitate the operation of distributed client applications. Examples of typical functionalities offered by coordination services include the provision of group membership, support for leader election, distributed synchronization, as well as reliable low-volume storage and naming. To provide reliable services to the client applications, coordination services in general are replicated for fault tolerance and should deliver high performance to ensure that they do not become bottlenecks for dependent applications. Apache ZooKeeper, for example, is a well-known coordination service and applies a primary-backup approach in which the leader server processes all state-modifying requests and then forwards the corresponding state updates to a set of follower servers using an atomic broadcast protocol called Zab. Having analyzed state-of-the-art coordination services, we identified two main limitations that prevent existing systems such as Apache ZooKeeper from achieving a higher write performance: First, while this approach prevents the data stored by client applications from being lost as a result of server crashes, it also comes at the cost of a performance penalty. In particular, the fact that it relies on a leader-based protocol, means that its performance becomes bottlenecked when the leader server has to handle an increased message traffic as the number of client requests and replicas increases. Second, Zab requires significant communication between instances (as it entails three communication steps). This can potentially lead to performance overhead and uses up more computer resources, resulting in less guarantees for users who must then build more complex applications to handle these issues. To this end, the work makes four contributions. First, we implement ZooKeeper atomic broadcast, extracting from ZooKeeper in order to make it easier for other developers to build their applications on top of Zab without the complexity of integrating the entire ZooKeeper codebase. Second, we propose three variations of Zab, which are all capable of reaching an agreement in fewer communication steps than Zab. The v variations are built with restriction assumptions that server crashes are independent and a server quorum remains operative at all times. The first variation offers excellent performance but can only be used for 3-server systems; the other two are built without this limitation. Then, we redesigned the latest two Zab variations to operate under the least-restricted Zab fault assumptions. Third, we design and implement a ZooKeeper coin-tossing protocol, called ZabCT which addresses the above concerns by having the other, non-leader server replicas toss a coin and broadcast their acknowledgment of a leader’s proposal only if the toss results in an outcome of Head. We model the ZabCT process and derive analytical expressions for estimating the coin-tossing probability of Head for a given arrival rate of service requests such that the dual objectives of performance gains and traffic reduction can be accomplished. If a coin-tossing protocol, ZabCT is judged not to offer performance benefits over Zab, processes should be able to switch autonomously to Zab. We design protocol switching by letting processes switch between ZabCT and Zab without stopping message delivery. Finally, an extensive performance evaluation is provided for Zab and Zab-variant protocols