Datacenter Traffic Control: Understanding Techniques and Trade-offs

Datacenters provide cost-effective and flexible access to scalable compute and storage resources necessary for today's cloud computing needs. A typical datacenter is made up of thousands of servers connected with a large network and usually managed by one operator. To provide quality access to the variety of applications and services hosted on datacenters and maximize performance, it deems necessary to use datacenter networks effectively and efficiently. Datacenter traffic is often a mix of several classes with different priorities and requirements. This includes user-generated interactive traffic, traffic with deadlines, and long-running traffic. To this end, custom transport protocols and traffic management techniques have been developed to improve datacenter network performance. In this tutorial paper, we review the general architecture of datacenter networks, various topologies proposed for them, their traffic properties, general traffic control challenges in datacenters and general traffic control objectives. The purpose of this paper is to bring out the important characteristics of traffic control in datacenters and not to survey all existing solutions (as it is virtually impossible due to massive body of existing research). We hope to provide readers with a wide range of options and factors while considering a variety of traffic control mechanisms. We discuss various characteristics of datacenter traffic control including management schemes, transmission control, traffic shaping, prioritization, load balancing, multipathing, and traffic scheduling. Next, we point to several open challenges as well as new and interesting networking paradigms. At the end of this paper, we briefly review inter-datacenter networks that connect geographically dispersed datacenters which have been receiving increasing attention recently and pose interesting and novel research problems.Comment: Accepted for Publication in IEEE Communications Surveys and Tutorial

Analysis of adaptive algorithms for an integrated communication network

Techniques were examined that trade communication bandwidth for decreased transmission delays. When the network is lightly used, these schemes attempt to use additional network resources to decrease communication delays. As the network utilization rises, the schemes degrade gracefully, still providing service but with minimal use of the network. Because the schemes use a combination of circuit and packet switching, they should respond to variations in the types and amounts of network traffic. Also, a combination of circuit and packet switching to support the widely varying traffic demands imposed on an integrated network was investigated. The packet switched component is best suited to bursty traffic where some delays in delivery are acceptable. The circuit switched component is reserved for traffic that must meet real time constraints. Selected packet routing algorithms that might be used in an integrated network were simulated. An integrated traffic places widely varying workload demands on a network. Adaptive algorithms were identified, ones that respond to both the transient and evolutionary changes that arise in integrated networks. A new algorithm was developed, hybrid weighted routing, that adapts to workload changes

Reliability of a Maintainable Manufacturing Network subject to Budget

Applying network analysis, a manufacturing system can be constructed as a manufacturing network by representing each workstation as an arc and each inspection station as a node. In particular, the capacity of each workstation is stochastic (i.e. multistate) due to the possibility of failure, partial failure, and maintenance. In practical cases, such a manufacturing network has to achieve a specified production level to satisfy the customers’ orders. Hence, maintenance is necessary to guarantee a manufacturing network can retain a minimal production level. A maintenance model, namely maintainable manufacturing network (MMN), is proposed to evaluate whether the manufacturing system can provide sufficient capacity subject to maintenance budget or not. The maintenance reliability is further proposed to calculate the probability that the MMN provides a sufficient capacity level to meet the minimal production level under maintenance budget

Distributed Adaptive Routing in Communication Networks

In this report, we present a new adaptive multi-flow routing algorithm to select end- to-end paths in packet-switched networks. This algorithm provides provable optimality guarantees in the following game theoretic sense: The network configuration converges to a configuration arbitrarily close to a pure Nash equilibrium. In this context, a Nash equilibrium is a configuration in which no flow can improve its end-to-end delay by changing its network path. This algorithm has several robustness properties making it suitable for real-life usage: it is robust to measurement errors, outdated information and clocks desynchronization. Furthermore, it is only based on local information and only takes local decisions, making it suitable for a distributed implementation. Our SDN-based proof-of-concept is built as an Openflow controller. We set up an emulation platform based on Mininet to test the behavior of our proof-of-concept implementation in several scenarios. Although real-world conditions do not conform exactly to the theoretical model, all experiments exhibit satisfying behavior, in accordance with the theoretical predictions

Distributed Adaptive Routing in Communication Networks

In this report, we present a new adaptive multi-flow routing algorithm to select end- to-end paths in packet-switched networks. This algorithm provides provable optimality guarantees in the following game theoretic sense: The network configuration converges to a configuration arbitrarily close to a pure Nash equilibrium. In this context, a Nash equilibrium is a configuration in which no flow can improve its end-to-end delay by changing its network path. This algorithm has several robustness properties making it suitable for real-life usage: it is robust to measurement errors, outdated information and clocks desynchronization. Furthermore, it is only based on local information and only takes local decisions, making it suitable for a distributed implementation. Our SDN-based proof-of-concept is built as an Openflow controller. We set up an emulation platform based on Mininet to test the behavior of our proof-of-concept implementation in several scenarios. Although real-world conditions do not conform exactly to the theoretical model, all experiments exhibit satisfying behavior, in accordance with the theoretical predictions

Approximation Knowledge-Based Recurrent Neural Network for Estimating N-Terminal Reliability

The main problem dispersed with in this paper is to find a novel method for the improvement in the reliability analysis of Computer Network. Reliability prediction are estimated during the life cycle of a computer network with the aim of estimating failure. In designing a variable size network, the serviceability, availability and reliability of the any network is a primary consideration. The reliability calculation in varying size network is a problem of NP-hard; it requires more calculation and effort with the amplifying no of nodes and links. Many different approaches have been taken for reliability and probability calculation for triumphant communication between any pair of computers. The paper presents a method for identifying n-terminal network reliability based on RNN technique. The method derived in this paper preceding inputs which increases the speed of computation. The approach works efficiently and overcome the difficulties of the previous approaches defined with neural network model and other reliability estimation techniques. It is proposed that the RNN model be used to replace the most time-consuming component of the system reliability evaluation approach. A variable-length sequence input can be handled by RNN. The main goal of this paper is to predict asperity of reliability which is highly correlated with performance of network in any unfavorable conditions

Stochastic Service Network Design for Intermodal Freight Transportation

In view of the accelerating climate change, greenhouse gas emissions from freight transportation must be significantly reduced over the next decades. Intermodal transportation can make a significant contribution here. During the transportation process, different modes of transportation are combined, enabling a modal shift to environmentally friendly alternatives such as rail and inland waterway transportation. However, at the same time, the organization of several modes is more complex compared to the unimodal case (where, for example, only trucks are employed). In particular, an efficient management of uncertainties, such as fluctuating transportation demand volumes or delays, is required to realize low costs and transportation times, thereby ensuring the attractiveness of intermodal transportation for a further modal shift. Stochastic service network design can explicitly consider such uncertainities in the planning in order to increase the performance of intermodal transportation. Decisions for the network design as well as for the mode choice are defined by mathematical optimization models, which originate from operations research and include relevant uncertainities by stochastic parameters. As central research gap, this dissertation addresses important operational constraints and decision variables of real-life intermodal networks, which have not been considered in these models so far and, in consequence, strongly limit their application in everyday operations. The resulting research contribution are two new variants of stochastic service network design models: The "stochastic service network design with integrated vehicle routing problem" integrates corresponding routing problems for road vehicles into the planning of intermodal networks. This new variant ensures a cost- and delay-minimal mode choice in the case of uncertain transportation times. The "stochastic service network design with short-term schedule modifications" deals with modifications of intermodal transportation schedules in order to adapt them to fluctuating demand as best as possible. For both new model variants, heuristic solution methods are presented which can efficiently solve even large network instances. Extensive case studies with real-world data demonstrate significant savings potentials compared to deterministic models as well as (simplified) stochastic models that already exist in literature

A Quantitative Framework for Assessing Vulnerability and Redundancy of Freight Transportation Networks

Freight transportation networks are an important component of everyday life in modern society. Disruption to these networks can make peoples’ daily lives extremely difficult as well as seriously cripple economic productivity. This dissertation develops a quantitative framework for assessing vulnerability and redundancy of freight transportation networks. The framework consists of three major contributions: (1) a two- stage approach for estimating a statewide truck origin-destination (O-D) trip table, (2) a decision support tool for assessing vulnerability of freight transportation networks, and (3) a quantitative approach for measuring redundancy of freight transportation networks.The dissertation first proposes a two-stage approach to estimate a statewide truck O-D trip table. The proposed approach is supported by two sequential stages: the first stage estimates a commodity-based truck O-D trip table using the commodity flows derived from the Freight Analysis Framework (FAF) database, and the second stage uses the path flow estimator (PFE) concept to refine the truck trip table obtained from the first stage using the truck counts from the statewide truck count program. The model allows great flexibility of incorporating data at different spatial levels for estimating the truck O- D trip table. The results from the second stage provide us a better understanding of truck flows on the statewide truck routes and corridors, and allow us to better manage the anticipated impacts caused by network disruptions.A decision support tool is developed to facilitate the decision making system through the application of its database management capabilities, graphical user interface, GIS-based visualization, and transportation network vulnerability analysis. The vulnerability assessment focuses on evaluating the statewide truck-freight bottlenecks/chokepoints. This dissertation proposes two quantitative measures: O-D connectivity (or detour route) in terms of distance and freight flow pattern change in terms of vehicle miles traveled (VMT). The case study adopts a “what-if” analysis approach by generating the disruption scenarios of the structurally deficient bridges in Utah due to earthquakes. In addition, the potential impacts of disruptions to multiple bridges in both rural and urban areas are evaluated and compared to the single bridge failure scenarios.This dissertation also proposes an approach to measure the redundancy of freight transportation networks based on two main dimensions: route diversity and network spare capacity. The route diversity dimension is used to evaluate the existence of multiple efficient routes available for users or the degree of connections between a specific O-D pair. The network spare capacity dimension is used to quantify the network- wide spare capacity with an explicit consideration of congestion effect. These two dimensions can complement each other by providing a two-dimensional characterization of freight transportation network redundancy. Case studies of the Utah statewide transportation network and coal multimodal network are conducted to demonstrate the features of the vulnerability and redundancy measures and the applicability of the quantitative assessment methodology

Reliability Analysis of Social Networks

The primary focus of this dissertation is on the quantification of actor interaction and the dissemination of information through Social networks. Social networks have long been used to model the interactions between people in various Social and professional contexts. These networks allow for the explicit modeling of the complex interrelations between relevant individuals within an organization and the role they play in the decision making process. This dissertation considers Social networks represented as network flow models in which actors have the ability to provide some level of influence over other actors within the network. The models developed incorporate performance metrics and reliability analysis established in the multi-state reliability literature to gain insights into organizational behavior. After a brief introduction, Chapter 2 provides a survey of the relevant literature on several topics of interest within this dissertation. In Chapter 3, actor criticality findings using traditional Social network analysis are compared to those obtained via multi-state reliability importance measures. Chapter 4 extends the model developed in Chapter 3 to consider that an actor\u27s Social interaction and level of influence within the organization are not only multi-valued and stochastic in nature but also a function of the interactions with its neighbors. A Monte Carlo simulation model is presented to evaluate the reliability of the network, and network reliability is evaluated under various influence communication rules. In Chapter 5, a hierarchical network structure is investigated where actors are arranged in layers and communication exists between layers. A probability mass function is developed to compute the expected level of influence at the target nodes as a function of the existing communication paths within the network. An illustrative example is used to demonstrate the effects on expected influence at the target as connections are either added or removed and when the uncertainty associated with an actor\u27s influence level is removed. Finally, in Chapter 6, a methodology is developed for eliciting the probabilities associated with the influence levels used in the network analysis of Chapters 3 - 5