Datacenter Traffic Control: Understanding Techniques and Trade-offs

Datacenters provide cost-effective and flexible access to scalable compute and storage resources necessary for today's cloud computing needs. A typical datacenter is made up of thousands of servers connected with a large network and usually managed by one operator. To provide quality access to the variety of applications and services hosted on datacenters and maximize performance, it deems necessary to use datacenter networks effectively and efficiently. Datacenter traffic is often a mix of several classes with different priorities and requirements. This includes user-generated interactive traffic, traffic with deadlines, and long-running traffic. To this end, custom transport protocols and traffic management techniques have been developed to improve datacenter network performance. In this tutorial paper, we review the general architecture of datacenter networks, various topologies proposed for them, their traffic properties, general traffic control challenges in datacenters and general traffic control objectives. The purpose of this paper is to bring out the important characteristics of traffic control in datacenters and not to survey all existing solutions (as it is virtually impossible due to massive body of existing research). We hope to provide readers with a wide range of options and factors while considering a variety of traffic control mechanisms. We discuss various characteristics of datacenter traffic control including management schemes, transmission control, traffic shaping, prioritization, load balancing, multipathing, and traffic scheduling. Next, we point to several open challenges as well as new and interesting networking paradigms. At the end of this paper, we briefly review inter-datacenter networks that connect geographically dispersed datacenters which have been receiving increasing attention recently and pose interesting and novel research problems.Comment: Accepted for Publication in IEEE Communications Surveys and Tutorial

Measurement Based Reconfigurations in Optical Ring Metro Networks

Single-hop wavelength division multiplexing (WDM) optical ring networks operating in packet mode are one of themost promising architectures for the design of innovative metropolitan network (metro) architectures. They permit a cost-effective design, with a good combination of optical and electronic technologies, while supporting features like restoration and reconfiguration that are essential in any metro scenario. In this article, we address the tunability requirements that lead to an effective resource usage and permit reconfiguration in optical WDM metros.We introduce reconfiguration algorithms that, on the basis of traffic measurements, adapt the network configuration to traffic demands to optimize performance. Using a specific network architecture as a reference case, the paper aims at the broader goal of showing which are the advantages fostered by innovative network designs exploiting the features of optical technologies

Handling packet dropouts and random delays for unstable delayed processes in NCS by optimal tuning of PIλDμ controllers with evolutionary algorithms

This is the author accepted manuscript. The final version is available from Elsevier via the DOI in this record.The issues of stochastically varying network delays and packet dropouts in Networked Control System (NCS) applications have been simultaneously addressed by time domain optimal tuning of fractional order (FO) PID controllers. Different variants of evolutionary algorithms are used for the tuning process and their performances are compared. Also the effectiveness of the fractional order PI(λ)D(μ) controllers over their integer order counterparts is looked into. Two standard test bench plants with time delay and unstable poles which are encountered in process control applications are tuned with the proposed method to establish the validity of the tuning methodology. The proposed tuning methodology is independent of the specific choice of plant and is also applicable for less complicated systems. Thus it is useful in a wide variety of scenarios. The paper also shows the superiority of FOPID controllers over their conventional PID counterparts for NCS applications.This work has been supported by the Board of Research in Nuclear Sciences (BRNS) of the Department of Atomic Energy (DAE), India, sanction no. 2009/36/62-BRNS, dated November 2009

Discretized Distributed Optimization over Dynamic Digraphs

We consider a discrete-time model of continuous-time distributed optimization over dynamic directed-graphs (digraphs) with applications to distributed learning. Our optimization algorithm works over general strongly connected dynamic networks under switching topologies, e.g., in mobile multi-agent systems and volatile networks due to link failures. Compared to many existing lines of work, there is no need for bi-stochastic weight designs on the links. The existing literature mostly needs the link weights to be stochastic using specific weight-design algorithms needed both at the initialization and at all times when the topology of the network changes. This paper eliminates the need for such algorithms and paves the way for distributed optimization over time-varying digraphs. We derive the bound on the gradient-tracking step-size and discrete time-step for convergence and prove dynamic stability using arguments from consensus algorithms, matrix perturbation theory, and Lyapunov theory. This work, particularly, is an improvement over existing stochastic-weight undirected networks in case of link removal or packet drops. This is because the existing literature may need to rerun time-consuming and computationally complex algorithms for stochastic design, while the proposed strategy works as long as the underlying network is weight-symmetric and balanced. The proposed optimization framework finds applications to distributed classification and learning

Reliable load-balancing routing for resource-constrained wireless sensor networks

Wireless sensor networks (WSNs) are energy and resource constrained. Energy limitations make it advantageous to balance radio transmissions across multiple sensor nodes. Thus, load balanced routing is highly desirable and has motivated a significant volume of research. Multihop sensor network architecture can also provide greater coverage, but requires a highly reliable and adaptive routing scheme to accommodate frequent topology changes. Current reliability-oriented protocols degrade energy efficiency and increase network latency. This thesis develops and evaluates a novel solution to provide energy-efficient routing while enhancing packet delivery reliability. This solution, a reliable load-balancing routing (RLBR), makes four contributions in the area of reliability, resiliency and load balancing in support of the primary objective of network lifetime maximisation. The results are captured using real world testbeds as well as simulations. The first contribution uses sensor node emulation, at the instruction cycle level, to characterise the additional processing and computation overhead required by the routing scheme. The second contribution is based on real world testbeds which comprises two different TinyOS-enabled senor platforms under different scenarios. The third contribution extends and evaluates RLBR using large-scale simulations. It is shown that RLBR consumes less energy while reducing topology repair latency and supports various aggregation weights by redistributing packet relaying loads. It also shows a balanced energy usage and a significant lifetime gain. Finally, the forth contribution is a novel variable transmission power control scheme which is created based on the experience gained from prior practical and simulated studies. This power control scheme operates at the data link layer to dynamically reduce unnecessarily high transmission power while maintaining acceptable link reliability

Security techniques for sensor systems and the Internet of Things

Sensor systems are becoming pervasive in many domains, and are recently being generalized by the Internet of Things (IoT). This wide deployment, however, presents significant security issues. We develop security techniques for sensor systems and IoT, addressing all security management phases. Prior to deployment, the nodes need to be hardened. We develop nesCheck, a novel approach that combines static analysis and dynamic checking to efficiently enforce memory safety on TinyOS applications. As security guarantees come at a cost, determining which resources to protect becomes important. Our solution, OptAll, leverages game-theoretic techniques to determine the optimal allocation of security resources in IoT networks, taking into account fixed and variable costs, criticality of different portions of the network, and risk metrics related to a specified security goal. Monitoring IoT devices and sensors during operation is necessary to detect incidents. We design Kalis, a knowledge-driven intrusion detection technique for IoT that does not target a single protocol or application, and adapts the detection strategy to the network features. As the scale of IoT makes the devices good targets for botnets, we design Heimdall, a whitelist-based anomaly detection technique for detecting and protecting against IoT-based denial of service attacks. Once our monitoring tools detect an attack, determining its actual cause is crucial to an effective reaction. We design a fine-grained analysis tool for sensor networks that leverages resident packet parameters to determine whether a packet loss attack is node- or link-related and, in the second case, locate the attack source. Moreover, we design a statistical model for determining optimal system thresholds by exploiting packet parameters variances. With our techniques\u27 diagnosis information, we develop Kinesis, a security incident response system for sensor networks designed to recover from attacks without significant interruption, dynamically selecting response actions while being lightweight in communication and energy overhead