深層学習に基づくパケット伝送ストラテジーを用いた知的ネットワーク制御に関する研究

Tohoku University加藤寧課

Split Federated Learning for 6G Enabled-Networks: Requirements, Challenges and Future Directions

Sixth-generation (6G) networks anticipate intelligently supporting a wide range of smart services and innovative applications. Such a context urges a heavy usage of Machine Learning (ML) techniques, particularly Deep Learning (DL), to foster innovation and ease the deployment of intelligent network functions/operations, which are able to fulfill the various requirements of the envisioned 6G services. Specifically, collaborative ML/DL consists of deploying a set of distributed agents that collaboratively train learning models without sharing their data, thus improving data privacy and reducing the time/communication overhead. This work provides a comprehensive study on how collaborative learning can be effectively deployed over 6G wireless networks. In particular, our study focuses on Split Federated Learning (SFL), a technique recently emerged promising better performance compared with existing collaborative learning approaches. We first provide an overview of three emerging collaborative learning paradigms, including federated learning, split learning, and split federated learning, as well as of 6G networks along with their main vision and timeline of key developments. We then highlight the need for split federated learning towards the upcoming 6G networks in every aspect, including 6G technologies (e.g., intelligent physical layer, intelligent edge computing, zero-touch network management, intelligent resource management) and 6G use cases (e.g., smart grid 2.0, Industry 5.0, connected and autonomous systems). Furthermore, we review existing datasets along with frameworks that can help in implementing SFL for 6G networks. We finally identify key technical challenges, open issues, and future research directions related to SFL-enabled 6G networks

Networks, Communication, and Computing Vol. 2

Networks, communications, and computing have become ubiquitous and inseparable parts of everyday life. This book is based on a Special Issue of the Algorithms journal, and it is devoted to the exploration of the many-faceted relationship of networks, communications, and computing. The included papers explore the current state-of-the-art research in these areas, with a particular interest in the interactions among the fields

Resource Management and Backhaul Routing in Millimeter-Wave IAB Networks Using Deep Reinforcement Learning

Thesis (PhD (Electronic Engineering))--University of Pretoria, 2023..The increased densification of wireless networks has led to the development of integrated access and backhaul (IAB) networks. In this thesis, deep reinforcement learning was applied to solve resource management and backhaul routing problems in millimeter-wave IAB networks. In the research work, a resource management solution that aims to avoid congestion for access users in an IAB network was proposed and implemented. The proposed solution applies deep reinforcement learning to learn an optimized policy that aims to achieve effective resource allocation whilst minimizing congestion and satisfying the user requirements. In addition, a deep reinforcement learning-based backhaul adaptation strategy that leverages a recursive discrete choice model was implemented in simulation. Simulation results where the proposed algorithms were compared with two baseline methods showed that the proposed scheme provides better throughput and delay performance.Sentech Chair in Broadband Wireless Multimedia Communications.Electrical, Electronic and Computer EngineeringPhD (Electronic Engineering)Unrestricte

Du placement des services à la surveillance des services dans les réseaux 5G et post-5G

5G and beyond 5G (B5G) networks are expected to accommodate a plethora of network services with diverse requirements using a single physical infrastructure. Hence, the ``one-size fits all'' paradigm that characterized the 4th generation of wireless networks is no longer suitable. By leveraging the last advent of Network Function Virtualization (NFV) and Software-Defined Networking (SDN), Network Slicing (NS) is considered as one of the key enablers of this paradigm shift. NS will enable the coexistence of heterogeneous services by partitioning the physical infrastructure into a set of virtual networks ''(the slices)'', each running a particular service. Besides, NS offers more flexibility and agility in business operations.Despite the advantages it brings, NS raises some technical challenges. The placement of network slices is one of them, it is known in the literature as the Virtual Network Embedding Problem (VNEP), and it is an NP-Hard problem. Therefore, the first part of this thesis focuses on unveiling the potential of Deep Reinforcement Learning (DRL) and Graph Neural Networks (GNNs) to solve the network slice placement problem and overcome the limitations of existing methods. Two approaches are considered: The first one aims to learn automatically how to solve the VNEP. Instead of putting any constraint on the topology of the physical infrastructure or extracting features manually, we formulate the task as a reinforcement problem, and we use a graph convolutional-based neural architecture to learn how to find an optimal solution. Next, instead of training a DRL agent from scratch to find the optimal solution, a process that may result in unsafe training, we train it to reduce the optimality gap of existing heuristics. The motivation behind this contribution is to ensure safety during the training of the DRL agent.The placement of the slices is not the only challenge raised by NS. Once the slices are placed, monitoring the status of network slices becomes a priority for both network slices' tenants and providers in order to ensure that Service Level Agreements (SLAs) are not violated. In the second part of this thesis, we propose to leverage machine learning techniques and network tomography to monitor the network slices. Network Tomography (NT) is defined as a set of methods that aim to infer unmeasured network metrics using an end-to-end measurement between monitors.We focus on two main challenges. First, on the inference of slices metrics based on some end-to-end measurements between monitors, as well as on the efficient monitor placement. For the inference, we model the task as a multi-output regression problem, which we solve using neural networks. We propose to train on synthetic data to augment the diversity of the training data and avoid the overfitting issue. Moreover, to deal with the changes that may occur either on the slices we monitor or the topology on top of which they are placed, we use transfer learning techniques.Regarding the monitor's placement problem, we consider a special case where only cycles' probes are allowed. The probing cycle schemes have a significant advantage compared to regular paths since the source probe is actually the destination, which reduces the synchronization problems. We formulate the problem as a variant of the Minimum Set Cover problem. Owing to its complexity, we introduce a standalone solution based on GNNs and genetic algorithms to find a trade-off between the quality of monitors placement and the cost to achieve it.Les réseaux 5G et au-delà sont destinés à servir un large éventail de services réseau aux besoins très disparates tout en utilisant la même infrastructure physique. En scindant l'infrastructure physique en un ensemble de réseaux virtuels, chacun exploitant un service spécifique, le Network Slicing (NS) permettra la coexistence de ces services. En dépit de ses avantages, le NS est complexe d'un point de vue technique puisqu'il s'agit d'un problème NP-hard. La première section de la thèse explore le potentiel de l'apprentissage par renforcement profond (DRL) basé sur des graphes de réseaux neuronaux pour résoudre le problème du placement des tranches de réseau et remédier aux limites des techniques existantes. Deux approches sont proposées : la première consiste à apprendre à résoudre automatiquement le problème du placement. Plutôt que de se limiter à la topologie de l'infrastructure physique ou à extraire manuellement des caractéristiques, le problème est formulé sous la forme d'un processus de décision markovien qui est résolu à l'aide d’un réseau de neurones convolutif à base de graphes pour apprendre à découvrir une solution optimale. Ensuite, plutôt que de former un agent DRL de zéro pour identifier la meilleure solution, ce qui pourrait entraîner un défaut de fiabilité, un agent est présenté pour réduire l'écart d'optimalité des heuristiques existantes. Une fois les tranches placées, la surveillance de l'état des tranches de réseau devient une priorité pour s'assurer que les SLAs sont respectés. Ainsi, dans la deuxième partie de la thèse, il est proposé d'utiliser des techniques d'apprentissage automatique et la tomographie réseau (NT) pour surveiller les tranches de réseau. Il y a deux problèmes majeurs à prendre en compte. Premièrement, les métriques de slices sont déduites sur la base de diverses mesures de bout en bout entre les moniteurs, ainsi que du placement efficace des moniteurs. Des réseaux neuronaux sont utilisés pour traiter l'inférence des métriques. Une approche d'apprentissage par transfert est également utilisée pour faire face aux changements qui peuvent se produire sur les slices surveillés ou sur la topologie physique sur laquelle elles sont placées. Des sondes cycliques sont envisagées pour le problème du placement des moniteurs. Le problème est formulé comme une variante du problème de couverture par ensembles. En raison de sa complexité, il est proposé d'introduire une solution autonome basée sur des réseaux neuronaux à base de graphes (GNN) et des algorithmes génétiques pour trouver un compromis entre la qualité du placement des moniteurs et le coût pour y parvenir

Reinforcement Learning

Brains rule the world, and brain-like computation is increasingly used in computers and electronic devices. Brain-like computation is about processing and interpreting data or directly putting forward and performing actions. Learning is a very important aspect. This book is on reinforcement learning which involves performing actions to achieve a goal. The first 11 chapters of this book describe and extend the scope of reinforcement learning. The remaining 11 chapters show that there is already wide usage in numerous fields. Reinforcement learning can tackle control tasks that are too complex for traditional, hand-designed, non-learning controllers. As learning computers can deal with technical complexities, the tasks of human operators remain to specify goals on increasingly higher levels. This book shows that reinforcement learning is a very dynamic area in terms of theory and applications and it shall stimulate and encourage new research in this field

Machine Learning-Powered Management Architectures for Edge Services in 5G Networks

L'abstract è presente nell'allegato / the abstract is in the attachmen

Contribution to multi-domain network slicing : resource orchestration framework and algorithms

5G/6G services and applications, in the context of the eMBB, mMTC and uRLLC network slicing framework, whose network infrastructure requirements may span beyond the coverage area of a single Infrastructure Provider (InP), are envisaged to be supported by leasing resources from multiple InPs. A challenging aspect for a Service Provider (SP) is how to obtain an optimal set of InPs on which to provision the requests and the particular substrate nodes and links within each InP on which to map the different VNFs and virtual links of the service requests, respectively, for a seamless, reliable and cost-effective orchestration of service requests. Existing works in this area either perform service mapping in uncoordinated manner, do not incorporate service reliability or do so from the perspective of stateless VNFs. Also they assume full information disclosure, or are based on exact approaches, which considerations are not well suited for future network scenarios characterized by delay sensitive mission critical applications and resource constrained networks. This thesis contributes to the above challenge by breaking the multi-domain service orchestration problem into two interlinked sub-problems that are solved in a coordinated manner: (1) Request splitting/partitioning (sub-problem 1), involving obtaining a subset of InPs and the corresponding inter-domain links on which to provision the different VNFs and virtual links of the service request; (2) Intra-domain VNF orchestration (sub-problem 2), involving obtaining the intra-domain nodes and links to provision the VNFs and virtual links of the sub-SFC associated with each InP. In this way, the thesis sets out four key targets that are necessary to align with the mission critical and delay sensitive use-cases envisaged in 5G and future networks in terms of service deployment cost and QoS: (1) coordinated mapping of service requests, with a view of realizing better utilization of the substrate resources; (2) survivability and fault-tolerant orchestration of service requests, to tame both QoS violations and the penalties from such violations; (3) limited disclosure of InP internal information, in order adhere to the privacy requirements InPs, and (4) achieving all the above targets in polynomial time. In order to realize the above targets, the thesis sought for solution techniques that are: (1) able to incorporate information learned in the previous solutions search space and historical mapping decisions, hence, resulting in acceptable performance even in scenarios of limited information exposure and fuzzy environments; (2) robust and less problem specific, hence, can be tailored to different optimization objectives, network topologies and service request constraints, thus enabling to deal with requests with either chained topologies or with bifurcated paths; (3) capable of dealing with an optimization problem that is jointly affected by multiple attributes, since in practice, the service deployment cost is jointly affected by multiple conflicting costs; (4) able to realize near-optimal solutions in practical run-times, thus rendering well suited approaches for delay sensitive and resource constrained scenarios. Three different algorithms namely, an RL, Genetic Algorithm (GA) and a fully distributed multi-stage graph-based algorithms are proposed for sub-problem 1. In addition, five different algorithms based on GA, Harmony search, RL, and multi-stage graph approach are proposed for sub-problem 2. Finally, in order to guide the implementation and adherence of the thesis proposals to the four main targets of the thesis, an architectural framework is proposed, aligned with the ETSI NFV-MANO architectural framework. Overall, the simulations results proved that the thesis proposals are optimized in terms of request acceptance ratios, mapping cost and execution time, hence, rendering such proposals well suited for 5G and future scenarios.Els serveis que es poden presentar en el marc de la tecnologia de “slicing” de xarxa de 5G/6G, com ara eMBB, mMTC o uRLLC, es possible que no els pugui oferir un sol proveïdor d’infraestructura (InP) degut a les limitacions que pot tenir la seva xarxa, i per tant que faci necessària la cooperació de múltiples InPs. En aquest cas, el primer repte que afronta el Proveïdor de Servei (SP) que rep la sol·licitud de desplegament es determinar el conjunt òptim de InPs que hi han d’intervenir i en concret els nodes i enllaços de cada un d’ells que s’han d’utilitzar per al mapatge de les diferents VNFs i enllaços virtuals de la sol·licitud. Els treballs que existeixen en aquesta àrea duen a terme el mapatge del servei be sigui de manera no coordinada, o no incorporen la fiabilitat, o ho fan des de la perspectiva de VNFs sense estat. També, pressuposen la divulgació total de la informació, o estan basats en metodologies exactes que fa que no siguin idonis per a escenaris de xarxes del futur, caracteritzats per aplicacions de missió critica, sensibles al retard i sobre xarxes amb recursos limitats. Aquesta tesi contribueix a afrontar aquests reptes dividint el problema d’orquestració de serveis multi domini en dos subproblemes relacionats, que es resolen de manera coordinada. (1) Divisió / partició de la sol·licitud de servei (sub-problema 1), que implica l'obtenció d'un subconjunt d'InPs i els enllaços interdomini corresponents sobre els quals proporcionar les diferents VNF i enllaços virtuals de la sol·licitud de servei; (2) Orquestració VNF intradomini (sub-problema 2), que implica l'obtenció dels nodes i enllaços intradomini per aprovisionar les VNF i enllaços virtuals dels sub-SFC associats a cada InP. D'aquesta manera, la tesi estableix quatre objectius clau que són necessaris per alinear-se amb els casos d'ús de missió crítica i sensibles al retard previstos en 5G i xarxes futures en termes de cost de desplegament del servei i QoS: (1) mapatge coordinat de les sol·licituds de servei, amb l'objectiu de realitzar una millor utilització dels recursos del substrat; (2) orquestració de les sol·licituds de servei contemplant la supervivència del servei en situacions de fallides, minimitzant les violacions de la QoS i les sancions derivades d'aquestes violacions; (3) divulgació limitada de la informació interna de l’InP, per tal d'adherir-se als requisits de privadesa dels InPs, i (4) aconseguir tots els objectius anteriors en temps polinòmic. Per tal de realitzar els objectius anteriors, la tesi busca solucions que siguin: (1) capaces d'incorporar informació apresa en les solucions anteriors de l'espai de cerca i decisions de mapatge històric, donant lloc a un rendiment acceptable fins i tot en escenaris d'exposició limitada a la informació i entorns difusos; (2) robustes i menys dependents dels problemes específics, i per tant, que es poden adaptar a diferents objectius d'optimització, topologies de xarxa i restriccions de sol·licitud de servei, permetent així fer front a sol·licituds amb cadenes de funcions de topologies molt diverses; (3) capaces de fer front a un problema d'optimització de múltiples atributs, ja que a la pràctica, el cost de desplegament del servei depèn de múltiples costos; (4) capaces de trobar solucions gairebé òptimes en temps suficientment breus, resultant així adequades a escenaris sensibles al retard i amb limitació de recursos. La tesi proposa tres algorismes diferents per al sub-problema 1: un algorisme de RL, un algorisme genètic (GA) i un algorisme multi etapa basat en grafs i completament distribuït. A més, es proposen cinc algorismes diferents basats en l'enfocament de grafs, un algorisme GA, un algorisme de cerca d’harmonia, un algorisme de RL i un algorisme multi-etapa per al sub-problema 2. Finalment, per tal de guiar la implementació i l'adhesió de les propostes als quatre objectius principals de la tesi, es proposa...Postprint (published version

Online learning on the programmable dataplane

This thesis makes the case for managing computer networks with datadriven methods automated statistical inference and control based on measurement data and runtime observations—and argues for their tight integration with programmable dataplane hardware to make management decisions faster and from more precise data. Optimisation, defence, and measurement of networked infrastructure are each challenging tasks in their own right, which are currently dominated by the use of hand-crafted heuristic methods. These become harder to reason about and deploy as networks scale in rates and number of forwarding elements, but their design requires expert knowledge and care around unexpected protocol interactions. This makes tailored, per-deployment or -workload solutions infeasible to develop. Recent advances in machine learning offer capable function approximation and closed-loop control which suit many of these tasks. New, programmable dataplane hardware enables more agility in the network— runtime reprogrammability, precise traffic measurement, and low latency on-path processing. The synthesis of these two developments allows complex decisions to be made on previously unusable state, and made quicker by offloading inference to the network. To justify this argument, I advance the state of the art in data-driven defence of networks, novel dataplane-friendly online reinforcement learning algorithms, and in-network data reduction to allow classification of switchscale data. Each requires co-design aware of the network, and of the failure modes of systems and carried traffic. To make online learning possible in the dataplane, I use fixed-point arithmetic and modify classical (non-neural) approaches to take advantage of the SmartNIC compute model and make use of rich device local state. I show that data-driven solutions still require great care to correctly design, but with the right domain expertise they can improve on pathological cases in DDoS defence, such as protecting legitimate UDP traffic. In-network aggregation to histograms is shown to enable accurate classification from fine temporal effects, and allows hosts to scale such classification to far larger flow counts and traffic volume. Moving reinforcement learning to the dataplane is shown to offer substantial benefits to stateaction latency and online learning throughput versus host machines; allowing policies to react faster to fine-grained network events. The dataplane environment is key in making reactive online learning feasible—to port further algorithms and learnt functions, I collate and analyse the strengths of current and future hardware designs, as well as individual algorithms