Deep Reinforcement Learning for Smart Queue Management

With the goal of meeting the stringent throughput and delay requirements of classified network flows, we propose a Deep Q-learning Network (DQN) for optimal weight selection in an active queue management system based on Weighted Fair Queuing (WFQ). Our system schedules flows belonging to different priority classes (Gold, Silver, and Bronze) into separate queues, and learns how and when to dequeue from each queue. The neural network implements deep reinforcement learning tools such as target networks and replay buffers to help learn the best weights depending on the network state. We show, via simulations, that our algorithm converges to an efficient model capable of adapting to the flow demands, producing thus lower delays with respect to traditional WFQ

Softair: Software-defined networking and network function virtualization solutions for 5g cellular systems

One of the main building blocks and major challenges for 5G cellular systems is the design of flexible network architectures, which can be realized by the paradigm of software-defined networking (SDN) and network function virtualization (NFV). Existing commercial cellular systems rely on closed and inflexible hardware-based architectures both at the radio frontend and in the core network. These problems significantly delay the adoption and deployment of new standards, impose great challenges in implementing new techniques to maximize the network capacity and coverage, and prevent provisioning of truly-differentiated services for highly variable traffic patterns. The objective of this thesis is to introduce an innovative software-defined architecture for 5G cellular systems, called SoftAir. First, a detailed overview is provided for priori wireless SDN architecture solutions. Second, the SoftAir architecture is introduced with key design elements. Third, four essential management tools for SoftAir are developed. Last, novel software-defined traffic engineering, enabled by SoftAir, are proposed. Through the synergy of SDN and NFV, SoftAir enables the next-generation cellular networks with the needed flexibility for evolving and adapting to the ever-changing network context, and lays out the foundation for 5G wireless software-defined cellular systems.Ph.D.Ph.D

A dynamic DRR scheduling algorithm for flow level QOS assurances for elastic traffic

Cataloged from PDF version of article.Best effort service, used to transport the Internet traffic today, does not provide any QoS assurances. Intserv, DiffServ and recently proposed Proportional Diff- Serv architectures have been introduced to provide QoS. In these architectures, some applications with more stringent QoS requirement such as real time traffic are prioritized, while elastic flows share the remaining bandwidth. As opposed to the well studied differential treatment of delay and/or loss sensitive traffic to satisfy QoS constraints, our aim is satisfy QoS requirements of elastic traffic at the flow level. We intend to maintain different average rate levels for different classes of elastic traffic. For differential treatment of elastic flows, a dynamic variant of Deficit Round Robin Scheduler (DRR) is used as oppose to a FIFO queue. In this scheduling algorithm, all classes are served in a round robin fashion in proportion to their weights at each round. The main difference of our scheduler from the original DRR scheduler is that, we update the weights, which are called quantums of the scheduler at each round in response to the feedback from the network, which is in terms of the rate of phantom connection sharing capacity fairly with the other flows in the same queue. According to the rate measured in the last time interval, the controller updates the weights in proportion with the bandwidth requirements of each class to satisfy their QoS requirements, while the remaining bandwidth will be used by the best effort traffic. In order to find an optimal policy for the controller a simulation-based learning algorithm is performed using a processor sharing model of TCP, then the resultant policies are applied to a more realistic scenario to solve Dynamic DRR scheduling problem through ns-2 simulations.Kurugöl, SılaM.S

Stochastic Dynamic Programming and Stochastic Fluid-Flow Models in the Design and Analysis of Web-Server Farms

A Web-server farm is a specialized facility designed specifically for housing Web servers catering to one or more Internet facing Web sites. In this dissertation, stochastic dynamic programming technique is used to obtain the optimal admission control policy with different classes of customers, and stochastic uid- ow models are used to compute the performance measures in the network. The two types of network traffic considered in this research are streaming (guaranteed bandwidth per connection) and elastic (shares available bandwidth equally among connections). We first obtain the optimal admission control policy using stochastic dynamic programming, in which, based on the number of requests of each type being served, a decision is made whether to allow or deny service to an incoming request. In this subproblem, we consider a xed bandwidth capacity server, which allocates the requested bandwidth to the streaming requests and divides all of the remaining bandwidth equally among all of the elastic requests. The performance metric of interest in this case will be the blocking probability of streaming traffic, which will be computed in order to be able to provide Quality of Service (QoS) guarantees. Next, we obtain bounds on the expected waiting time in the system for elastic requests that enter the system. This will be done at the server level in such a way that the total available bandwidth for the requests is constant. Trace data will be converted to an ON-OFF source and fluid- flow models will be used for this analysis. The results are compared with both the mean waiting time obtained by simulating real data, and the expected waiting time obtained using traditional queueing models. Finally, we consider the network of servers and routers within the Web farm where data from servers flows and merges before getting transmitted to the requesting users via the Internet. We compute the waiting time of the elastic requests at intermediate and edge nodes by obtaining the distribution of the out ow of the upstream node. This out ow distribution is obtained by using a methodology based on minimizing the deviations from the constituent in flows. This analysis also helps us to compute waiting times at different bandwidth capacities, and hence obtain a suitable bandwidth to promise or satisfy the QoS guarantees. This research helps in obtaining performance measures for different traffic classes at a Web-server farm so as to be able to promise or provide QoS guarantees; while at the same time helping in utilizing the resources of the server farms efficiently, thereby reducing the operational costs and increasing energy savings

Dynamic routing optimization using traffic prediction

In this dissertation, a new efficient routing maintenance algorithm, called Predicting of Future Load-based Routing (PFLR), is introduced for optimizing the routing performance in IP-based networks. The main idea of PFLR algorithm is combing the predicted link load with the current link load with an effective method to optimize the link weights and so reduce the network congestions. Another research objective is introducing a new efficient Traffic Engineering (TE) algorithm, called Prediction-based Decentralized Routing (PDR) algorithm, which is fully decentralized and self-organized approach

Traffic Optimization in Data Center and Software-Defined Programmable Networks

L'abstract è presente nell'allegato / the abstract is in the attachmen

Towards providing reliable job completion time predictions using PCS

In this paper we build a case for providing job completion time predictions to cloud users, similar to the delivery date of a package or arrival time of a booked ride. Our analysis reveals that providing predictability can come at the expense of performance and fairness. Existing cloud scheduling systems optimize for extreme points in the trade-off space, making them either extremely unpredictable or impractical. To address this challenge, we present PCS, a new scheduling framework that aims to provide predictability while balancing other traditional objectives. The key idea behind PCS is to use Weighted-Fair-Queueing (WFQ) and find a suitable configuration of different WFQ parameters (e.g., class weights) that meets specific goals for predictability. It uses a simulation-aided search strategy, to efficiently discover WFQ configurations that lie on the Pareto front of the trade-off space between these objectives. We implement and evaluate PCS in the context of DNN job scheduling on GPUs. Our evaluation, on a small scale GPU testbed and larger-scale simulations, shows that PCS can provide accurate completion time estimates while marginally compromising on performance and fairness

Smart Decision-Making via Edge Intelligence for Smart Cities

Smart cities are an ambitious vision for future urban environments. The ultimate aim of smart cities is to use modern technology to optimize city resources and operations while improving overall quality-of-life of its citizens. Realizing this ambitious vision will require embracing advancements in information communication technology, data analysis, and other technologies. Because smart cities naturally produce vast amounts of data, recent artificial intelligence (AI) techniques are of interest due to their ability to transform raw data into insightful knowledge to inform decisions (e.g., using live road traffic data to control traffic lights based on current traffic conditions). However, training and providing these AI applications is non-trivial and will require sufficient computing resources. Traditionally, cloud computing infrastructure have been used to process computationally intensive tasks; however, due to the time-sensitivity of many of these smart city applications, novel computing hardware/technologies are required. The recent advent of edge computing provides a promising computing infrastructure to support the needs of the smart cities of tomorrow. Edge computing pushes compute resources close to end users to provide reduced latency and improved scalability — making it a viable candidate to support smart cities. However, it comes with hardware limitations that are necessary to consider. This thesis explores the use of the edge computing paradigm for smart city applications and how to make efficient, smart decisions related to their available resources. This is done while considering the quality-of-service provided to end users. This work can be seen as four parts. First, this work touches on how to optimally place and serve AI-based applications on edge computing infrastructure to maximize quality-of-service to end users. This is cast as an optimization problem and solved with efficient algorithms that approximate the optimal solution. Second, this work investigates the applicability of compression techniques to reduce offloading costs for AI-based applications in edge computing systems. Finally, this thesis then demonstrate how edge computing can support AI-based solutions for smart city applications, namely, smart energy and smart traffic. These applications are approached using the recent paradigm of federated learning. The contributions of this thesis include the design of novel algorithms and system design strategies for placement and scheduling of AI-based services on edge computing systems, formal formulation for trade-offs between delivered AI model performance and latency, compression for offloading decisions for communication reductions, and evaluation of federated learning-based approaches for smart city applications

Stochastic Dynamic Programming and Stochastic Fluid-Flow Models in the Design and Analysis of Web-Server Farms

A Web-server farm is a specialized facility designed specifically for housing Web servers catering to one or more Internet facing Web sites. In this dissertation, stochastic dynamic programming technique is used to obtain the optimal admission control policy with different classes of customers, and stochastic uid- ow models are used to compute the performance measures in the network. The two types of network traffic considered in this research are streaming (guaranteed bandwidth per connection) and elastic (shares available bandwidth equally among connections). We first obtain the optimal admission control policy using stochastic dynamic programming, in which, based on the number of requests of each type being served, a decision is made whether to allow or deny service to an incoming request. In this subproblem, we consider a xed bandwidth capacity server, which allocates the requested bandwidth to the streaming requests and divides all of the remaining bandwidth equally among all of the elastic requests. The performance metric of interest in this case will be the blocking probability of streaming traffic, which will be computed in order to be able to provide Quality of Service (QoS) guarantees. Next, we obtain bounds on the expected waiting time in the system for elastic requests that enter the system. This will be done at the server level in such a way that the total available bandwidth for the requests is constant. Trace data will be converted to an ON-OFF source and fluid- flow models will be used for this analysis. The results are compared with both the mean waiting time obtained by simulating real data, and the expected waiting time obtained using traditional queueing models. Finally, we consider the network of servers and routers within the Web farm where data from servers flows and merges before getting transmitted to the requesting users via the Internet. We compute the waiting time of the elastic requests at intermediate and edge nodes by obtaining the distribution of the out ow of the upstream node. This out ow distribution is obtained by using a methodology based on minimizing the deviations from the constituent in flows. This analysis also helps us to compute waiting times at different bandwidth capacities, and hence obtain a suitable bandwidth to promise or satisfy the QoS guarantees. This research helps in obtaining performance measures for different traffic classes at a Web-server farm so as to be able to promise or provide QoS guarantees; while at the same time helping in utilizing the resources of the server farms efficiently, thereby reducing the operational costs and increasing energy savings

Flexible and intelligent network programming for cloud networks

As modern online services are evolving promptly and involving larger amount of data and computation than ever, the demand for cloud networks keeps growing rapidly, which also brings new challenges to network programming. Network programming is a complicated and crucial task for building high-performance cloud networks. Current network programming mainly presents two shortcomings: (1) it is inflexible as adding new data-plane features usually takes several years; (2) it is unintelligent as it heavily depends on human-designed heuristic algorithms to solve production-scale problems. To overcome these shortcomings, this dissertation realizes flexible and intelligent network programming by leveraging the recent development of new technologies both in hardware and software. Specifically, it presents four systems with new performance features that cannot be achieved by conventional network programming: (i) Harmonia: A new replicated storage architecture that provides near-linear scalability without sacrificing consistency. By exploiting the programming flexibility of new-generation programmable switches, Harmonia checks read-write conflicts in network for guaranteeing consistency, and enables any replica to serve reads for objects with no pending writes for near-linear scalability. (ii) RackSched: A microsecond-scale scheduler for rack-scale computers. It proposes a two-layer scheduling framework that integrates the inter-server scheduler in the top-of-rack (ToR) switch with intra-server schedulers on each server. The in-network inter-server scheduler is programmed to realize power-of-k-choices, ensure request affinity, and track server loads accurately and efficiently. (iii) NetVRM: A network management system that supports dynamic register memory sharing in the network. It orchestrates the register memory allocation between multiple concurrent network applications to optimize the multiplexing benefits. This goal is achieved with three major features: a virtual register memory abstraction, a dynamic memory allocation algorithm, and a domain-specific programming language extension. (iv) NeuroPlan: Automated and efficient network planning with deep reinforcement learning (RL). It leverages a two-stage hybrid approach that first uses deep RL to prune a large and complex search space and then uses an Integer Linear Programming (ILP) solver to find the final solution. Such an automated approach avoids human efforts to design heuristic algorithms manually and reduces network plan cost efficiently. We have done theoretical analysis, built testbeds, and evaluated these systems with prototype experiments and simulations under realistic setups from production networks