Datacenter Traffic Control: Understanding Techniques and Trade-offs

Datacenters provide cost-effective and flexible access to scalable compute and storage resources necessary for today's cloud computing needs. A typical datacenter is made up of thousands of servers connected with a large network and usually managed by one operator. To provide quality access to the variety of applications and services hosted on datacenters and maximize performance, it deems necessary to use datacenter networks effectively and efficiently. Datacenter traffic is often a mix of several classes with different priorities and requirements. This includes user-generated interactive traffic, traffic with deadlines, and long-running traffic. To this end, custom transport protocols and traffic management techniques have been developed to improve datacenter network performance. In this tutorial paper, we review the general architecture of datacenter networks, various topologies proposed for them, their traffic properties, general traffic control challenges in datacenters and general traffic control objectives. The purpose of this paper is to bring out the important characteristics of traffic control in datacenters and not to survey all existing solutions (as it is virtually impossible due to massive body of existing research). We hope to provide readers with a wide range of options and factors while considering a variety of traffic control mechanisms. We discuss various characteristics of datacenter traffic control including management schemes, transmission control, traffic shaping, prioritization, load balancing, multipathing, and traffic scheduling. Next, we point to several open challenges as well as new and interesting networking paradigms. At the end of this paper, we briefly review inter-datacenter networks that connect geographically dispersed datacenters which have been receiving increasing attention recently and pose interesting and novel research problems.Comment: Accepted for Publication in IEEE Communications Surveys and Tutorial

Just Queuing: Policy-Based Scheduling Mechanism for Packet Switching Networks

The pervasiveness of the Internet and its applications lead to the potential increment of the users’ demands for more services with economical prices. The diversity of Internet traffic requires some classification and prioritisation since some traffic deserve much attention with less delay and loss compared to others. Current scheduling mechanisms are exposed to the trade-off between three major properties namely fairness, complexity and protection. Therefore, the question remains about how to improve the fairness and protection with less complex implementation. This research is designed to enhance scheduling mechanism by providing sustainability to the fairness and protection properties with simplicity in implementation; and hence higher service quality particularly for real-time applications. Extra elements are applied to the main fairness equation to improve the fairness property. This research adopts the restricted charge policy which imposes the protection of normal user. In terms of the complexity property, genetic algorithm has an advantage in holding the fitness score of the queue in separate storage space which potentially minimises the complexity of the algorithm. The integrity between conceptual, analytical and experimental approach verifies the efficiency of the proposed mechanism. The proposed mechanism is validated by using the emulation and the validation experiments involve real router flow data. The results of the evaluation showed fair bandwidth distribution similar to the popular Weighted Fair Queuing (WFQ) mechanism. Furthermore, better protection was exhibited in the results compared with the WFQ and two other scheduling mechanisms. The complexity of the proposed mechanism reached O(log(n)) which is considered as potentially low. Furthermore, this mechanism is limited to the wired networks and hence future works could improve the mechanism to be adopted in mobile ad-hoc networks or any other wireless networks. Moreover, more improvements could be applied to the proposed mechanism to enhance its deployment in the virtual circuits switching network such as the asynchronous transfer mode networks

A hierarchical approach to multi-project planning under uncertainty

We survey several viewpoints on the management of the planning complexity of multi-project organisations under uncertainty. A positioning framework is proposed to distinguish between different types of project-driven organisations, which is meant to aid project management in the choice between the various existing planning approaches. We discuss the current state of the art of hierarchical planning approaches both for traditional manufacturing and for project environments. We introduce a generic hierarchical project planning and control framework that serves to position planning methods for multi-project planning under uncertainty. We discuss multiple techniques for dealing with the uncertainty inherent to the different hierarchical stages in a multi-project organisation. In the last part of this paper we discuss two cases from practice and we relate these practical cases to the positioning framework that is put forward in the paper

Final report on the evaluation of RRM/CRRM algorithms

Deliverable public del projecte EVERESTThis deliverable provides a definition and a complete evaluation of the RRM/CRRM algorithms selected in D11 and D15, and evolved and refined on an iterative process. The evaluation will be carried out by means of simulations using the simulators provided at D07, and D14.Preprin

A hierarchical approach to multi-project planning under uncertainty.

We survey several viewpoints on the management of the planning complexity of multi-project organisations under uncertainty. A positioning framework is proposed to distinguish between different types of project-driven organisations, which is meant to aid project management in the choice between the various existing planning approaches. We discuss the current state of the art of hierarchical planning approaches both for traditional manufacturing and for project environments. We introduce a generic hierarchical project planning and control framework that serves to position planning methods for multi-project planning under uncertainty. We discuss multiple techniques for dealing with the uncertainty inherent to the different hierarchical stages in a multi-project organisation. In the last part of this paper we discuss two cases from practice and we relate these practical cases to the positioning framework that is put forward in the paper.Choice; Complexity; Framework; Hierarchical models; Management; Manufacturing; Methods; Multi-project organisations; Planning; Project management; Project planning; Uncertainty;

Virtualization of network I/O on modern operating systems

Network I/O of modern operating systems is incomplete. In this networkage, users and their applications are still unable to control theirown traffic, even on their local host. Network I/O is a sharedresource of a host machine, and traditionally, to address problemswith a shared resource, system research has virtualized the resource.Therefore, it is reasonable to ask if the virtualization can providesolutions to problems in network I/O of modern operating systems, inthe same way as the other components of computer systems, such asmemory and CPU. With the aim of establishing the virtualization ofnetwork I/O as a design principle of operating systems, thisdissertation first presents a virtualization model, hierarchicalvirtualization of network interface. Systematic evaluation illustratesthat the virtualization model possesses desirable properties forvirtualization of network I/O, namely flexible control granularity,resource protection, partitioning of resource consumption, properaccess control and generality as a control model. The implementedprototype exhibits practical performance with expected functionality,and allowed flexible and dynamic network control by users andapplications, unlike existing systems designed solely for systemadministrators. However, because the implementation was hardcoded inkernel source code, the prototype was not perfect in its functionalcoverage and flexibility. Accordingly, this dissertation investigatedhow to decouple OS kernels and packet processing code throughvirtualization, and studied three degrees of code virtualization,namely, limited virtualization, partial virtualization, and completevirtualization. In this process, a novel programming model waspresented, based on embedded Java technology, and the prototypeimplementation exhibited the following characteristics, which aredesirable for network code virtualization. First, users program inJava to carry out safe and simple programming for packetprocessing. Second, anyone, even untrusted applications, can performinjection of packet processing code in the kernel, due to isolation ofcode execution. Third, the prototype implementation empirically provedthat such a virtualization does not jeopardize system performance.These cases illustrate advantages of virtualization, and suggest thatthe hierarchical virtualization of network interfaces can be aneffective solution to problems in network I/O of modern operatingsystems, both in the control model and in implementation