Resource allocation for dataflow applications in FANETs using anypath routing

Management of network resources in advanced IoT applications is a challenging topic due to their distributed nature from the Edge to the Cloud, and the heavy demand of real-time data from many sources to take action in the deployment. FANETs (Flying Ad-hoc Networks) are a clear example of heterogeneous multi-modal use cases, which require strict quality in the network communications, as well as the coordination of the computing capabilities, in order to operate correctly the final service. In this paper, we present a Virtual Network Embedding (VNE) framework designed for the allocation of dataflow applications, composed of nano-services that produce or consume data, in a wireless infrastructure, such as an airborne network. To address the problem, an anypath-based heuristic algorithm that considers the quality demand of the communication between nano-services is proposed, coined as Quality-Revenue Paired Anypath Dataflow VNE (QRPAD-VNE). We also provide a simulation environment for the evaluation of its performance according to the virtual network (VN) request load in the system. Finally, we show the suitability of a multi-parameter framework in conjunction with anypath routing in order to have better performance results that guarantee minimum quality in the wireless communications.Xunta de Galicia | Ref. ED431C 2022/04 T254Ministerio de Universidades | Ref. FPU19/01284Agencia Estatal de Investigación | Ref. PCI2020-112174Agencia Estatal de Investigación | Ref. PID2020-113795RB-C33Agencia Estatal de Investigación | Ref. PID2020-116329GB-C21Universidade de Vigo/CISU

Network Infrastructures for Highly Distributed Cloud-Computing

Software-Defined-Network (SDN) is emerging as a solid opportunity for the Network Service Providers (NSP) to reduce costs while at the same time providing better and/or new services. The possibility to flexibly manage and configure highly-available and scalable network services through data model abstractions and easy-to-consume APIs is attractive and the adoption of such technologies is gaining momentum. At the same time, NSPs are planning to innovate their infrastructures through a process of network softwarisation and programmability. The SDN paradigm aims at improving the design, configuration, maintenance and service provisioning agility of the network through a centralised software control. This can be easily achievable in local area networks, typical of data-centers, where the benefits of having programmable access to the entire network is not restricted by latency between the network devices and the SDN controller which is reasonably located in the same LAN of the data path nodes. In Wide Area Networks (WAN), instead, a centralised control plane limits the speed of responsiveness in reaction to time-constrained network events due to unavoidable latencies caused by physical distances. Moreover, an end-to-end control shall involve the participation of multiple, domain-specific, controllers: access devices, data-center fabrics and backbone networks have very different characteristics and their control-plane could hardly coexist in a single centralised entity, unless of very complex solutions which inevitably lead to software bugs, inconsistent states and performance issues. In recent years, the idea to exploit SDN for WAN infrastructures to connect multiple sites together has spread in both the scientific community and the industry. The former has produced interesting results in terms of framework proposals, complexity and performance analysis for network resource allocation schemes and open-source proof of concept prototypes targeting SDN architectures spanning multiple technological and administrative domains. On the other hand, much of the work still remains confined to the academy mainly because based on pure Openflow prototype implementation, networks emulated on a single general-purpose machine or on simulations proving algorithms effectiveness. The industry has made SDN a reality via closed-source systems, running on single administrative domain networks with little if no diversification of access and backbone devices. In this dissertation we present our contributions to the design and the implementation of SDN architectures for the control plane of WAN infrastructures. In particular, we studied and prototyped two SDN platforms to build a programmable, intent-based, control-plane suitable for the today highly distributed cloud infrastructures. Our main contributions are: (i) an holistic and architectural description of a distributed SDN control-plane for end-end QoS provisioning; we compare the legacy IntServ RSVP protocol with a novel approach for prioritising application-sensitive flows via centralised vantage points. It is based on a peer-to-peer architecture and could so be suitable for the inter-authoritative domains scenario. (ii) An open-source platform based on a two-layer hierarchy of network controllers designed to provision end-to-end connectivity in real networks composed by heterogeneous devices and links within a single authoritative domain. This platform has been integrated in CORD, an open-source project whose goal is to bring data-center economics and cloud agility to the NSP central office infrastructures, combining NFV (Network Function Virtualization), SDN and the elasticity of commodity clouds. Our platform enables the provisioning of connectivity services between multiple CORD sites, up to the customer premises. Thus our system and software contributions in SDN has been combined with a NFV infrastructure for network service automation and orchestration

Analyzing and Enhancing Routing Protocols for Friend-to-Friend Overlays

The threat of surveillance by governmental and industrial parties is more eminent than ever. As communication moves into the digital domain, the advances in automatic assessment and interpretation of enormous amounts of data enable tracking of millions of people, recording and monitoring their private life with an unprecedented accurateness. The knowledge of such an all-encompassing loss of privacy affects the behavior of individuals, inducing various degrees of (self-)censorship and anxiety. Furthermore, the monopoly of a few large-scale organizations on digital communication enables global censorship and manipulation of public opinion. Thus, the current situation undermines the freedom of speech to a detrimental degree and threatens the foundations of modern society. Anonymous and censorship-resistant communication systems are hence of utmost importance to circumvent constant surveillance. However, existing systems are highly vulnerable to infiltration and sabotage. In particular, Sybil attacks, i.e., powerful parties inserting a large number of fake identities into the system, enable malicious parties to observe and possibly manipulate a large fraction of the communication within the system. Friend-to-friend (F2F) overlays, which restrict direct communication to parties sharing a real-world trust relationship, are a promising countermeasure to Sybil attacks, since the requirement of establishing real-world trust increases the cost of infiltration drastically. Yet, existing F2F overlays suffer from a low performance, are vulnerable to denial-of-service attacks, or fail to provide anonymity. Our first contribution in this thesis is concerned with an in-depth analysis of the concepts underlying the design of state-of-the-art F2F overlays. In the course of this analysis, we first extend the existing evaluation methods considerably, hence providing tools for both our and future research in the area of F2F overlays and distributed systems in general. Based on the novel methodology, we prove that existing approaches are inherently unable to offer acceptable delays without either requiring exhaustive maintenance costs or enabling denial-of-service attacks and de-anonymization. Consequentially, our second contribution lies in the design and evaluation of a novel concept for F2F overlays based on insights of the prior in-depth analysis. Our previous analysis has revealed that greedy embeddings allow highly efficient communication in arbitrary connectivity-restricted overlays by addressing participants through coordinates and adapting these coordinates to the overlay structure. However, greedy embeddings in their original form reveal the identity of the communicating parties and fail to provide the necessary resilience in the presence of dynamic and possibly malicious users. Therefore, we present a privacy-preserving communication protocol for greedy embeddings based on anonymous return addresses rather than identifying node coordinates. Furthermore, we enhance the communication’s robustness and attack-resistance by using multiple parallel embeddings and alternative algorithms for message delivery. We show that our approach achieves a low communication complexity. By replacing the coordinates with anonymous addresses, we furthermore provably achieve anonymity in the form of plausible deniability against an internal local adversary. Complementary, our simulation study on real-world data indicates that our approach is highly efficient and effectively mitigates the impact of failures as well as powerful denial-of-service attacks. Our fundamental results open new possibilities for anonymous and censorship-resistant applications.Die Bedrohung der Überwachung durch staatliche oder kommerzielle Stellen ist ein drängendes Problem der modernen Gesellschaft. Heutzutage findet Kommunikation vermehrt über digitale Kanäle statt. Die so verfügbaren Daten über das Kommunikationsverhalten eines Großteils der Bevölkerung in Kombination mit den Möglichkeiten im Bereich der automatisierten Verarbeitung solcher Daten erlauben das großflächige Tracking von Millionen an Personen, deren Privatleben mit noch nie da gewesener Genauigkeit aufgezeichnet und beobachtet werden kann. Das Wissen über diese allumfassende Überwachung verändert das individuelle Verhalten und führt so zu (Selbst-)zensur sowie Ängsten. Des weiteren ermöglicht die Monopolstellung einiger weniger Internetkonzernen globale Zensur und Manipulation der öffentlichen Meinung. Deshalb stellt die momentane Situation eine drastische Einschränkung der Meinungsfreiheit dar und bedroht die Grundfesten der modernen Gesellschaft. Systeme zur anonymen und zensurresistenten Kommunikation sind daher von ungemeiner Wichtigkeit. Jedoch sind die momentanen System anfällig gegen Sabotage. Insbesondere ermöglichen es Sybil-Angriffe, bei denen ein Angreifer eine große Anzahl an gefälschten Teilnehmern in ein System einschleust und so einen großen Teil der Kommunikation kontrolliert, Kommunikation innerhalb eines solchen Systems zu beobachten und zu manipulieren. F2F Overlays dagegen erlauben nur direkte Kommunikation zwischen Teilnehmern, die eine Vertrauensbeziehung in der realen Welt teilen. Dadurch erschweren F2F Overlays das Eindringen von Angreifern in das System entscheidend und verringern so den Einfluss von Sybil-Angriffen. Allerdings leiden die existierenden F2F Overlays an geringer Leistungsfähigkeit, Anfälligkeit gegen Denial-of-Service Angriffe oder fehlender Anonymität. Der erste Beitrag dieser Arbeit liegt daher in der fokussierten Analyse der Konzepte, die in den momentanen F2F Overlays zum Einsatz kommen. Im Zuge dieser Arbeit erweitern wir zunächst die existierenden Evaluationsmethoden entscheidend und erarbeiten so Methoden, die Grundlagen für unsere sowie zukünftige Forschung in diesem Bereich bilden. Basierend auf diesen neuen Evaluationsmethoden zeigen wir, dass die existierenden Ansätze grundlegend nicht fähig sind, akzeptable Antwortzeiten bereitzustellen ohne im Zuge dessen enorme Instandhaltungskosten oder Anfälligkeiten gegen Angriffe in Kauf zu nehmen. Folglich besteht unser zweiter Beitrag in der Entwicklung und Evaluierung eines neuen Konzeptes für F2F Overlays, basierenden auf den Erkenntnissen der vorangehenden Analyse. Insbesondere ergab sich in der vorangehenden Evaluation, dass Greedy Embeddings hoch-effiziente Kommunikation erlauben indem sie Teilnehmer durch Koordinaten adressieren und diese an die Struktur des Overlays anpassen. Jedoch sind Greedy Embeddings in ihrer ursprünglichen Form nicht auf anonyme Kommunikation mit einer dynamischen Teilnehmermengen und potentiellen Angreifern ausgelegt. Daher präsentieren wir ein Privätssphäre-schützenden Kommunikationsprotokoll für F2F Overlays, in dem die identifizierenden Koordinaten durch anonyme Adressen ersetzt werden. Des weiteren erhöhen wir die Resistenz der Kommunikation durch den Einsatz mehrerer Embeddings und alternativer Algorithmen zum Finden von Routen. Wir beweisen, dass unser Ansatz eine geringe Kommunikationskomplexität im Bezug auf die eigentliche Kommunikation sowie die Instandhaltung des Embeddings aufweist. Ferner zeigt unsere Simulationstudie, dass der Ansatz effiziente Kommunikation mit kurzen Antwortszeiten und geringer Instandhaltungskosten erreicht sowie den Einfluss von Ausfälle und Angriffe erfolgreich abschwächt. Unsere grundlegenden Ergebnisse eröffnen neue Möglichkeiten in der Entwicklung anonymer und zensurresistenter Anwendungen

Network-provider-independent overlays for resilience and quality of service.

PhDOverlay networks are viewed as one of the solutions addressing the inefficiency and slow evolution of the Internet and have been the subject of significant research. Most existing overlays providing resilience and/or Quality of Service (QoS) need cooperation among different network providers, but an inter-trust issue arises and cannot be easily solved. In this thesis, we mainly focus on network-provider-independent overlays and investigate their performance in providing two different types of service. Specifically, this thesis addresses the following problems: Provider-independent overlay architecture: A provider-independent overlay framework named Resilient Overlay for Mission-Critical Applications (ROMCA) is proposed. We elaborate its structure including component composition and functions and also provide several operational examples. Overlay topology construction for providing resilience service: We investigate the topology design problem of provider-independent overlays aiming to provide resilience service. To be more specific, based on the ROMCA framework, we formulate this problem mathematically and prove its NP-hardness. Three heuristics are proposed and extensive simulations are carried out to verify their effectiveness. Application mapping with resilience and QoS guarantees: Assuming application mapping is the targeted service for ROMCA, we formulate this problem as an Integer Linear Program (ILP). Moreover, a simple but effective heuristic is proposed to address this issue in a time-efficient manner. Simulations with both synthetic and real networks prove the superiority of both solutions over existing ones. Substrate topology information availability and the impact of its accuracy on overlay performance: Based on our survey that summarizes the methodologies available for inferring the selective substrate topology formed among a group of nodes through active probing, we find that such information is usually inaccurate and additional mechanisms are needed to secure a better inferred topology. Therefore, we examine the impact of inferred substrate topology accuracy on overlay performance given only inferred substrate topology information

Secure and dependable virtual network embedding

Tese de mestrado, Engenharia Informática (Arquitectura, Sistemas e Redes de Computadores) Universidade de Lisboa, Faculdade de Ciências, 2016A virtualização de redes tornou-se uma técnica poderosa que permite que várias redes virtuais, criadas por diferentes utilizadores, operem numa infraestrutura partilhada. Com o avanço de tecnologias como Redes Definidas por Software1, a virtualização de redes ganhou um novo ímpeto e tornou-se uma funcionalidade central em ambientes de computação em nuvem. Um dos grandes desafios que a virtualização de redes apresenta é como utilizar de forma eficiente os recursos oferecidos pelas redes físicas dos fornecedores de infraestruturas, nomeadamente os nós - entidades de uma rede com capacidade computacional – e ligações – entidades de uma rede que transportam dados entre pares de nós. De forma a resolver este problema, vários trabalhos da área de virtualização de redes têm sido desenvolvidos [1]. Em particular, têm sido propostos algoritmos que encontram formas eficazes para decidir onde mapear os nós e as ligações virtuais na rede física. Estes algoritmos podem assumir uma de três aproximações diferentes: soluções exatas, que resolvem pequenas instâncias do problema e encontram soluções ótimas para a localização dos recursos virtuais na rede física; soluções baseadas em heurísticas, que se focam em obter um bom resultado, próximo do ótimo, em pouco tempo; e meta-heurísticas, que usam técnicas específicas independentes do problema para achar um resultado próximo do ótimo. Tipicamente o objetivo destes algoritmos é achar estes mapeamentos tendo em conta determinadas métricas, como qualidade de serviço, custos económicos ou confiabilidade. Neste contexto, uma das métricas menos exploradas é a garantia da segurança das redes virtuais, um tema que é cada vez mais importante, especialmente em ambientes de computação em nuvem. As plataformas de virtualização propostas recentemente dão aos utilizadores a liberdade para especificarem de forma arbitrária as topologias virtuais para as suas redes e esquemas de endereçamento. Estas plataformas têm sido desenvolvidas considerando apenas um provedor de nuvem, forçando os clientes a confiarem que este provedor mantém os seus dados e cargas de trabalho seguros e disponíveis. Infelizmente, existem evidências de que problemas nestes ambientes ocorrem, tanto de natureza maliciosa (ataques causados através de algum elemento corrompido na rede) como benigna (falhas em elementos individuais da rede, ou falhas causadas, por exemplo, por catástrofes, afetando vários elementos da rede em simultâneo) [2]. Deste modo, nesta tese defendemos que a segurança e a confiabilidade são dois fatores críticos e, por isso, devem ser considerados durante o processo de mapeamento das redes virtuais. Nesse sentido, neste trabalho definimos um problema denominado Mapeamento de Redes Virtuais Seguro e Confiável, e construímos um algoritmo que resolve este problema num ambiente constituído por várias nuvens (i.e., múltiplos provedores de recursos físicos). Ao considerar-se um ambiente como este, evita-se que o cliente fique restringido a apenas um provedor, aumentando a possibilidade de a sua rede e o seu serviço resistirem a falhas em elementos da rede física ou interrupções numa nuvem, através da replicação dos serviços por diversas nuvens. A segurança das redes virtuais também é melhorada na medida em que os serviços mais sensíveis podem ser colocados em nuvens que oferecem maiores garantias de segurança. O problema em si tem como principal objetivo mapear redes virtuais sobre a rede física, distribuída potencialmente por diferentes nuvens, utilizando a menor quantidade de recursos, e satisfazendo, ao mesmo tempo, os seguintes requisitos: (i) cada nó e ligação virtual é mapeado na rede física satisfazendo os requisitos de capacidade de computação e de largura de banda, respetivamente, e também os requisitos de segurança e confiabilidade associados; (ii) cada nó virtual ´e mapeado num nó físico cuja localização satisfaz os requisitos do primeiro (isto é, se por exemplo um nó virtual procura uma nuvem que forneça um nível de máxima segurança, o nó físico que será alocado tem de pertencer a uma nuvem com essa característica); (iii) a rede virtual está protegida contra erros na rede física ou disrupção numa nuvem, de modo a cumprir os requisitos de confiabilidade. O algoritmo que apresentamos nesta tese cobre todos os requisitos deste problema, juntando, pela primeira vez, as propriedades segurança e confiabilidade. Adicionalmente, esta solução considera um ambiente de múltiplos domínios (neste caso, múltiplas nuvens), de maneira a eliminar eventuais limitações que surgem quando se usa um único provedor de nuvem. A solução criada é uma solução exata, desenvolvida através de uma técnica de otimização de programação inteira mista, e tem como objetivo minimizar os custos de mapeamento de redes virtuais, cobrindo sempre os seus requisitos de segurança e confiabilidade. Nesta solução são definidas diversas restrições que têm de ser cumpridas para que uma rede virtual possa ser mapeada sobre uma rede física. O nosso algoritmo oferece vários níveis de segurança e confiabilidade que podem ser escolhidos na definição das redes virtuais, nomeadamente associados aos nós e às ligações que as compõem. O cliente pode escolher arbitrariamente que níveis deseja para cada recurso virtual, para além de poder especificar também a topologia da sua rede e os requisitos de capacidade de computação e largura de banda para os nós e ligações, respetivamente. Sumariamente, nesta tese consideramos que são suportados vários níveis de segurança para os nós e ligações virtuais, que vão desde segurança por omissão, isto é, garantias mínimas de segurança, até à inclusão de mecanismos criptográficos que garantem maior segurança. Em relação à confiabilidade, os clientes podem optar por adicionar redundância aos seus recursos virtuais de modo a tolerar falhas. Quando é requisitada redundância, os clientes podem escolher, para cada nó virtual, se desejam a respetiva reserva adicional na mesma nuvem onde se encontra o nó primário, tolerando apenas falhas locais, ou localizada noutra nuvem, com o intuito de aumentar a probabilidade de a sua rede virtual sobreviver a uma disrupção¸ ao de uma nuvem. Na nossa solução, as nuvens são também distinguidas entre si consoante o nível de confiança que fornecem ao cliente. Podem ser consideradas nuvens públicas (pertencentes a provedores), privadas (pertencentes aos próprios clientes), entre outras. A definição de diferentes tipos de nuvem dá a possibilidade ao cliente de escolher as nuvens consoante a sensibilidade da sua informação. Nesta tese é ainda apresentada uma interface de programação de aplicações, que fornece como funcionalidade o mapeamento de redes virtuais segura e confiável, e que pode ser utilizada por plataformas de virtualização que tenham em conta ambientes de múltiplos domínios [3]. Quanto aos resultados, quando segurança e confiabilidade são requisitadas pelas redes virtuais, os mesmos mostram que existe um custo adicional (já esperado) para fornecer estas propriedades. No entanto, um ligeiro ajuste no preço dos recursos permite aos fornecedores de infraestruturas que fornecem segurança e confiabilidade obter um lucro semelhante (ou superior) ao dos fornecedores que não fornecem este tipo de propriedades. Os resultados mostram ainda que o nosso algoritmo se comporta de maneira similar ao algoritmo mais utilizado para mapeamento de redes virtuais, D-ViNE [4, 5], quando os requisitos de segurança e confiabilidade não são considerados. Apesar de serem uma boa base para novos trabalhos na área, as soluções exatas Não escalam (este tipo de soluções apenas consegue resolver problemas num tempo razoável se estes forem de pequena escala). Deste modo, como trabalho futuro, o primeiro caminho a tomar será o desenvolvimento de uma heurística que garanta as propriedades de segurança e confiabilidade.Network virtualization is emerging as a powerful technique to allow multiple virtual networks (VN), eventually specified by different tenants, to run on a shared infrastructure. With the recent advances on Software Defined Networks (SDN), network virtualization – traditionally limited to Virtual Local Area Networks (VLAN) – has gained new traction. A major challenge in network virtualization is how to make efficient use of the shared resources. Virtual network embedding (VNE) addresses this problem by finding an effective mapping of the virtual nodes and links onto the substrate network (SN). VNE has been studied in the network virtualization literature, with several different algorithms having been proposed to solve the problem. Typically, these algorithms address various requirements, such as quality of service (QoS), economic costs or dependability. A mostly unexplored perspective on this problem is providing security assurances, a gap increasingly more relevant to organizations, as they move their critical services to the cloud. Recently proposed virtualization platforms give tenants the freedom to specify their network topologies and addressing schemes. These platforms have been targeting only a datacenter of a single cloud provider, forcing complete trust on the provider to run the workloads correctly and limiting dependability. Unfortunately, there is increasing evidence that problems do occur at a cloud scale, of both malicious and benign natures. Thus, in this thesis we argue that security and dependability is becoming a critical factor that should be considered by VNE algorithms. Motivated by this, we define the secure and dependable VNE problem, and design an algorithm that addresses this problem in multiple cloud environments. By not relying on a single cloud we avoid internet-scale single points of failures, ensuring the recovery from cloud outages by replicating workloads across providers. Our solution can also enhance security by leaving sensitive workloads in more secure clouds: for instance, in private clouds under control of the user or in facilities that employ the required security features. The results from our experiments show that there is a cost in providing security and availability that may reduce the provider profit. However, a relatively small increase in the price of the richer features of our solution (e.g., security resources) enables the provider to offer secure and dependable network services at a profit. Our experiments also show that our algorithm behaves similarly to the most commonly used VNE algorithm when security and dependability are not requested by VNs

Greedy routing and virtual coordinates for future networks

At the core of the Internet, routers are continuously struggling with ever-growing routing and forwarding tables. Although hardware advances do accommodate such a growth, we anticipate new requirements e.g. in data-oriented networking where each content piece has to be referenced instead of hosts, such that current approaches relying on global information will not be viable anymore, no matter the hardware progress. In this thesis, we investigate greedy routing methods that can achieve similar routing performance as today but use much less resources and which rely on local information only. To this end, we add specially crafted name spaces to the network in which virtual coordinates represent the addressable entities. Our scheme enables participating routers to make forwarding decisions using only neighbourhood information, as the overarching pseudo-geometric name space structure already organizes and incorporates "vicinity" at a global level. A first challenge to the application of greedy routing on virtual coordinates to future networks is that of "routing dead-ends" that are local minima due to the difficulty of consistent coordinates attribution. In this context, we propose a routing recovery scheme based on a multi-resolution embedding of the network in low-dimensional Euclidean spaces. The recovery is performed by routing greedily on a blurrier view of the network. The different network detail-levels are obtained though the embedding of clustering-levels of the graph. When compared with higher-dimensional embeddings of a given network, our method shows a significant diminution of routing failures for similar header and control-state sizes. A second challenge to the application of virtual coordinates and greedy routing to future networks is the support of "customer-provider" as well as "peering" relationships between participants, resulting in a differentiated services environment. Although an application of greedy routing within such a setting would combine two very common fields of today's networking literature, such a scenario has, surprisingly, not been studied so far. In this context we propose two approaches to address this scenario. In a first approach we implement a path-vector protocol similar to that of BGP on top of a greedy embedding of the network. This allows each node to build a spatial map associated with each of its neighbours indicating the accessible regions. Routing is then performed through the use of a decision-tree classifier taking the destination coordinates as input. When applied on a real-world dataset (the CAIDA 2004 AS graph) we demonstrate an up to 40% compression ratio of the routing control information at the network's core as well as a computationally efficient decision process comparable to methods such as binary trees and tries. In a second approach, we take inspiration from consensus-finding in social sciences and transform the three-dimensional distance data structure (where the third dimension encodes the service differentiation) into a two-dimensional matrix on which classical embedding tools can be used. This transformation is achieved by agreeing on a set of constraints on the inter-node distances guaranteeing an administratively-correct greedy routing. The computed distances are also enhanced to encode multipath support. We demonstrate a good greedy routing performance as well as an above 90% satisfaction of multipath constraints when relying on the non-embedded obtained distances on synthetic datasets. As various embeddings of the consensus distances do not fully exploit their multipath potential, the use of compression techniques such as transform coding to approximate the obtained distance allows for better routing performances

A Machine Learning Enhanced Scheme for Intelligent Network Management

The versatile networking services bring about huge influence on daily living styles while the amount and diversity of services cause high complexity of network systems. The network scale and complexity grow with the increasing infrastructure apparatuses, networking function, networking slices, and underlying architecture evolution. The conventional way is manual administration to maintain the large and complex platform, which makes effective and insightful management troublesome. A feasible and promising scheme is to extract insightful information from largely produced network data. The goal of this thesis is to use learning-based algorithms inspired by machine learning communities to discover valuable knowledge from substantial network data, which directly promotes intelligent management and maintenance. In the thesis, the management and maintenance focus on two schemes: network anomalies detection and root causes localization; critical traffic resource control and optimization. Firstly, the abundant network data wrap up informative messages but its heterogeneity and perplexity make diagnosis challenging. For unstructured logs, abstract and formatted log templates are extracted to regulate log records. An in-depth analysis framework based on heterogeneous data is proposed in order to detect the occurrence of faults and anomalies. It employs representation learning methods to map unstructured data into numerical features, and fuses the extracted feature for network anomaly and fault detection. The representation learning makes use of word2vec-based embedding technologies for semantic expression. Next, the fault and anomaly detection solely unveils the occurrence of events while failing to figure out the root causes for useful administration so that the fault localization opens a gate to narrow down the source of systematic anomalies. The extracted features are formed as the anomaly degree coupled with an importance ranking method to highlight the locations of anomalies in network systems. Two types of ranking modes are instantiated by PageRank and operation errors for jointly highlighting latent issue of locations. Besides the fault and anomaly detection, network traffic engineering deals with network communication and computation resource to optimize data traffic transferring efficiency. Especially when network traffic are constrained with communication conditions, a pro-active path planning scheme is helpful for efficient traffic controlling actions. Then a learning-based traffic planning algorithm is proposed based on sequence-to-sequence model to discover hidden reasonable paths from abundant traffic history data over the Software Defined Network architecture. Finally, traffic engineering merely based on empirical data is likely to result in stale and sub-optimal solutions, even ending up with worse situations. A resilient mechanism is required to adapt network flows based on context into a dynamic environment. Thus, a reinforcement learning-based scheme is put forward for dynamic data forwarding considering network resource status, which explicitly presents a promising performance improvement. In the end, the proposed anomaly processing framework strengthens the analysis and diagnosis for network system administrators through synthesized fault detection and root cause localization. The learning-based traffic engineering stimulates networking flow management via experienced data and further shows a promising direction of flexible traffic adjustment for ever-changing environments