A Bayesian multi-armed bandit algorithm for dynamic end-to-end routing in SDN-based networks with piecewise-stationary rewards

To handle the exponential growth of data-intensive network edge services and automatically solve new challenges in routing management, machine learning is steadily being incorporated into software-defined networking solutions. In this line, the article presents the design of a piecewise-stationary Bayesian multi-armed bandit approach for the online optimum end-to-end dynamic routing of data flows in the context of programmable networking systems. This learning-based approach has been analyzed with simulated and emulated data, showing the proposal’s ability to sequentially and proactively self-discover the end-to-end routing path with minimal delay among a considerable number of alternatives, even when facing abrupt changes in transmission delay distributions due to both variable congestion levels on path network devices and dynamic delays to transmission links.info:eu-repo/semantics/publishedVersio

Network utility maximization for delay-sensitive applications in unknown communication settings

In the last decades the Internet traffic has greatly evolved. The advent of new Internet services and applications has, in fact, led to a significant growth of the amount of data transmitted, as well as to a transformation of the data type. As a matter of fact, nowadays, the largest amount of traffic share consists of multimedia data, which do not represent classical Internet data. Due to the increasing amount of traffic, the network resources might be scarce, and in such cases it becomes extremely important to optimize network transmission in order to provide a satisfying service to the users. Although methods for maximizing the network utility in scenarios with limited resources have been studied extensively, the evolution of the Internet services poses continuously new challenges that require novel solution methods to meet the transmission requirements. In this thesis we propose novel solutions methods to network utility maximization problems that arise in the context of nowadays network communications. In particular we analyze problems related to delay-sensitive Internet applications and rate allocation in unknown network settings. In the first problem we study how to effectively allocate the transmission rates in a multiparty videoconference system. The main contribution of this chapter is an approximate fast rate rate allocation method that is able to adapt quickly to changes in the videoconference conditions. This fast adaptation cannot be achieved with classical network utility maximization solving methods, as they are usually based on iterative approaches. In this case we leverage the particular structure of the problem to design a novel distributed solving method which proves to be very effective when compared to baseline solutions. The next problem that we address is the design of a congestion control algorithm for delay-sensitive applications. One of the main problems of existing delay-based congestion control algorithms is that they tend to achieve an extremely low throughput when competing against loss-based algorithms. In order to overcome this difficulty we propose a novel adaptive controller based on a bandit problem approach. The adaptive controller tries to infer how the network responds, in terms of rate-delay pair at equilibrium, when changing the delay sensitivity of an underlying delay-based congestion control. Once the network response is inferred, the controller selects the sensitivity that leads to the best trade-off between the transmitting rate and the experienced delay. In the final problem, we analyze the design of an overlay rate allocation systems to be used when: the amount of available network resources is not known, and the user congestion feedback cannot be used as valid signal to reach the optimal rate allocation. Such a scenario appears when an Internet application wants to maximize a certain utility metric, but, at the same time, it must operate using a specific congestion control algorithm that is completely unaware of the application utility. To solve this problem we design a distributed system that coordinates the users in order to perform active learning on the amount of network resource. Adopting such a method reveals to be the key to an effective maximization of the long term application utility for the entire system

Adaptive Matching for Expert Systems with Uncertain Task Types

A matching in a two-sided market often incurs an externality: a matched resource may become unavailable to the other side of the market, at least for a while. This is especially an issue in online platforms involving human experts as the expert resources are often scarce. The efficient utilization of experts in these platforms is made challenging by the fact that the information available about the parties involved is usually limited. To address this challenge, we develop a model of a task-expert matching system where a task is matched to an expert using not only the prior information about the task but also the feedback obtained from the past matches. In our model the tasks arrive online while the experts are fixed and constrained by a finite service capacity. For this model, we characterize the maximum task resolution throughput a platform can achieve. We show that the natural greedy approaches where each expert is assigned a task most suitable to her skill is suboptimal, as it does not internalize the above externality. We develop a throughput optimal backpressure algorithm which does so by accounting for the `congestion' among different task types. Finally, we validate our model and confirm our theoretical findings with data-driven simulations via logs of Math.StackExchange, a StackOverflow forum dedicated to mathematics.Comment: A part of it presented at Allerton Conference 2017, 18 page

Online Learning of Energy Consumption for Navigation of Electric Vehicles

Energy efficient navigation constitutes an important challenge in electric vehicles, due to their limited battery capacity. We employ a Bayesian approach to model the energy consumption at road segments for efficient navigation. In order to learn the model parameters, we develop an online learning framework and investigate several exploration strategies such as Thompson Sampling and Upper Confidence Bound. We then extend our online learning framework to the multi-agent setting, where multiple vehicles adaptively navigate and learn the parameters of the energy model. We analyze Thompson Sampling and establish rigorous regret bounds on its performance in the single-agent and multi-agent settings, through an analysis of the algorithm under batched feedback. Finally, we demonstrate the performance of our methods via experiments on several real-world city road networks