Adapting reinforcement learning for multimedia transmission on SDN

[EN] Multimedia transmissions require a high quantity of resources to ensure their quality. In the last years, some technologies that provide a better resource management have appeared. Software defined networks (SDNs) are presented as a solution to improve this management. Furthermore, combining SDN with artificial intelligence (AI) techniques, networks are able to provide a higher performance using the same resources. In this paper, a redefinition of reinforcement learning is proposed. This model is focused on multimedia transmission in a SDN environment. Moreover, the architecture needed and the algorithm of the reinforcement learning are described. Using the Openflow protocol, several sample actions are defined in the system. Results show that using the system users perceive an increase in the image quality three times better. Moreover, the loss rate is reduced more than half the value of losses recorded when the algorithm is not applied. Regarding bandwidth, the maximum throughput increases from 987.16 kbps to 24.73 Mbps while the average bandwidth improves from 412.42 kbps to 7.83 Mbps.Ayudas para contratos predoctorales de Formación del Profesorado Universitario FPU (Convocatoria 2015), Grant/Award Number: FPU15/06837; Programa Estatal de Investigación Científica y Técnica de Excelencia (Convocatoria 2017), Grant/Award Number: TIN2017-84802-C2-1-P; Programa Estatal De Investigación, Desarrollo e Innovación Orientada a los retos de la sociedad (Convocatoria 2016), Grant/Award Number: TEC2016-76795-C6-4-R; ERANETMED, Grant/Award Number: ERANETMED3-227 SMARTWATIRRego Mañez, A.; Sendra, S.; García-García, L.; Lloret, J. (2019). Adapting reinforcement learning for multimedia transmission on SDN. Transactions on Emerging Telecommunications Technologies. 30(9):1-15. https://doi.org/10.1002/ett.3643S11530

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Cognition-Based Networks: A New Perspective on Network Optimization Using Learning and Distributed Intelligence

IEEE Access Volume 3, 2015, Article number 7217798, Pages 1512-1530 Open Access Cognition-based networks: A new perspective on network optimization using learning and distributed intelligence (Article) Zorzi, M.a , Zanella, A.a, Testolin, A.b, De Filippo De Grazia, M.b, Zorzi, M.bc a Department of Information Engineering, University of Padua, Padua, Italy b Department of General Psychology, University of Padua, Padua, Italy c IRCCS San Camillo Foundation, Venice-Lido, Italy View additional affiliations View references (107) Abstract In response to the new challenges in the design and operation of communication networks, and taking inspiration from how living beings deal with complexity and scalability, in this paper we introduce an innovative system concept called COgnition-BAsed NETworkS (COBANETS). The proposed approach develops around the systematic application of advanced machine learning techniques and, in particular, unsupervised deep learning and probabilistic generative models for system-wide learning, modeling, optimization, and data representation. Moreover, in COBANETS, we propose to combine this learning architecture with the emerging network virtualization paradigms, which make it possible to actuate automatic optimization and reconfiguration strategies at the system level, thus fully unleashing the potential of the learning approach. Compared with the past and current research efforts in this area, the technical approach outlined in this paper is deeply interdisciplinary and more comprehensive, calling for the synergic combination of expertise of computer scientists, communications and networking engineers, and cognitive scientists, with the ultimate aim of breaking new ground through a profound rethinking of how the modern understanding of cognition can be used in the management and optimization of telecommunication network

Review of SDN-based load-balancing methods, issues, challenges, and roadmap

The development of the Internet and smart end systems, such as smartphones and portable laptops, along with the emergence of cloud computing, social networks, and the Internet of Things, has brought about new network requirements. To meet these requirements, a new architecture called software-defined network (SDN) has been introduced. However, traffic distribution in SDN has raised challenges, especially in terms of uneven load distribution impacting network performance. To address this issue, several SDN load balancing (LB) techniques have been developed to improve efficiency. This article provides an overview of SDN and its effect on load balancing, highlighting key elements and discussing various load-balancing schemes based on existing solutions and research challenges. Additionally, the article outlines performance metrics used to evaluate these algorithms and suggests possible future research directions

AROMA: An adapt-or-reroute strategy for multimedia applications over SDN-based wireless environments

To support new and advanced multimedia-rich applications and services while providing satisfactory user experience, the underlying network infrastructure needs to evolve and adapt. One of the key enabling technologies of the next generation (5G) networks is the integration of Software Defined Networking (SDN) within a heterogeneous wireless environment to enable interoperability and QoS provisioning. Leveraging on the features of the SDN paradigm, it is possible to introduce new solutions to handle the increasing mobile video transmission challenges with strict QoS requirements, such as: low delay, jitter, packet loss, and high bandwidth demands. However, degradation and instability perceived from video traffic makes it difficult to satisfy various end-users. In this context, this paper proposes AROMA, an Adapt-or-reROute strategy for Multimedia Applications over SDN-based wireless environments. AROMA enables QoS provisioning over multimedia-oriented SDN-based WLAN environments. The proposed solution is evaluated using a real experimental test-bed setup

Intelligent multimedia flow transmission through heterogeneous networks using cognitive software defined networks

[ES] La presente tesis aborda el problema del encaminamiento en las redes definidas por software (SDN). Específicamente, aborda el problema del diseño de un protocolo de encaminamiento basado en inteligencia artificial (AI) para garantizar la calidad de servicio (QoS) en transmisiones multimedia. En la primera parte del trabajo, el concepto de SDN es introducido. Su arquitectura, protocolos y ventajas son comentados. A continuación, el estado del arte es presentado, donde diversos trabajos acerca de QoS, encaminamiento, SDN y AI son detallados. En el siguiente capítulo, el controlador SDN, el cual juega un papel central en la arquitectura propuesta, es presentado. Se detalla el diseño del controlador y se compara su rendimiento con otro controlador comúnmente utilizado. Más tarde, se describe las propuestas de encaminamiento. Primero, se aborda la modificación de un protocolo de encaminamiento tradicional. Esta modificación tiene como objetivo adaptar el protocolo de encaminamiento tradicional a las redes SDN, centrado en las transmisiones multimedia. A continuación, la propuesta final es descrita. Sus mensajes, arquitectura y algoritmos son mostrados. Referente a la AI, el capítulo 5 detalla el módulo de la arquitectura que la implementa, junto con los métodos inteligentes usados en la propuesta de encaminamiento. Además, el algoritmo inteligente de decisión de rutas es descrito y la propuesta es comparada con el protocolo de encaminamiento tradicional y con su adaptación a las redes SDN, mostrando un incremento de la calidad final de la transmisión. Finalmente, se muestra y se describe algunas aplicaciones basadas en la propuesta. Las aplicaciones son presentadas para demostrar que la solución presentada en la tesis está diseñada para trabajar en redes heterogéneas.[CA] La present tesi tracta el problema de l'encaminament en les xarxes definides per programari (SDN). Específicament, tracta el problema del disseny d'un protocol d'encaminament basat en intel·ligència artificial (AI) per a garantir la qualitat de servici (QoS) en les transmissions multimèdia. En la primera part del treball, s'introdueix les xarxes SDN. Es comenten la seva arquitectura, els protocols i els avantatges. A continuació, l'estat de l'art és presentat, on es detellen els diversos treballs al voltant de QoS, encaminament, SDN i AI. Al següent capítol, el controlador SDN, el qual juga un paper central a l'arquitectura proposta, és presentat. Es detalla el disseny del controlador i es compara el seu rendiment amb altre controlador utilitzat comunament. Més endavant, es descriuen les propostes d'encaminament. Primer, s'aborda la modificació d'un protocol d'encaminament tradicional. Aquesta modificació té com a objectiu adaptar el protocol d'encaminament tradicional a les xarxes SDN, centrat a les transmissions multimèdia. A continuació, la proposta final és descrita. Els seus missatges, arquitectura i algoritmes són mostrats. Pel que fa a l'AI, el capítol 5 detalla el mòdul de l'arquitectura que la implementa, junt amb els mètodes intel·ligents usats en la proposta d'encaminament. A més a més, l'algoritme intel·ligent de decisió de rutes és descrit i la proposta és comparada amb el protocol d'encaminament tradicional i amb la seva adaptació a les xarxes SDN, mostrant un increment de la qualitat final de la transmissió. Finalment, es mostra i es descriuen algunes aplicacions basades en la proposta. Les aplicacions són presentades per a demostrar que la solució presentada en la tesi és dissenyada per a treballar en xarxes heterogènies.[EN] This thesis addresses the problem of routing in Software Defined Networks (SDN). Specifically, the problem of designing a routing protocol based on Artificial Intelligence (AI) for ensuring Quality of Service (QoS) in multimedia transmissions. In the first part of the work, SDN is introduced. Its architecture, protocols and advantages are discussed. Then, the state of the art is presented, where several works regarding QoS, routing, SDN and AI are detailed. In the next chapter, the SDN controller, which plays the central role in the proposed architecture, is presented. The design of the controller is detailed and its performance compared to another common controller. Later, the routing proposals are described. First, a modification of a traditional routing protocol is discussed. This modification intends to adapt a traditional routing protocol to SDN, focused on multimedia transmissions. Then, the final proposal is described. Its messages, architecture and algorithms are depicted. As regards AI, chapter 5 details the module of the architecture that implements it, along with all the intelligent methods used in the routing proposal. Furthermore, the intelligent route decision algorithm is described and the final proposal is compared to the traditional routing protocol and its adaptation to SDN, showing an increment of the end quality of the transmission. Finally, some applications based on the routing proposal are described. The applications are presented to demonstrate that the proposed solution can work with heterogeneous networks.Rego Máñez, A. (2020). Intelligent multimedia flow transmission through heterogeneous networks using cognitive software defined networks [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/160483TESI