934 research outputs found
An Overview on Application of Machine Learning Techniques in Optical Networks
Today's telecommunication networks have become sources of enormous amounts of
widely heterogeneous data. This information can be retrieved from network
traffic traces, network alarms, signal quality indicators, users' behavioral
data, etc. Advanced mathematical tools are required to extract meaningful
information from these data and take decisions pertaining to the proper
functioning of the networks from the network-generated data. Among these
mathematical tools, Machine Learning (ML) is regarded as one of the most
promising methodological approaches to perform network-data analysis and enable
automated network self-configuration and fault management. The adoption of ML
techniques in the field of optical communication networks is motivated by the
unprecedented growth of network complexity faced by optical networks in the
last few years. Such complexity increase is due to the introduction of a huge
number of adjustable and interdependent system parameters (e.g., routing
configurations, modulation format, symbol rate, coding schemes, etc.) that are
enabled by the usage of coherent transmission/reception technologies, advanced
digital signal processing and compensation of nonlinear effects in optical
fiber propagation. In this paper we provide an overview of the application of
ML to optical communications and networking. We classify and survey relevant
literature dealing with the topic, and we also provide an introductory tutorial
on ML for researchers and practitioners interested in this field. Although a
good number of research papers have recently appeared, the application of ML to
optical networks is still in its infancy: to stimulate further work in this
area, we conclude the paper proposing new possible research directions
Faster and More Accurate Measurement through Additive-Error Counters
Counters are a fundamental building block for networking applications such as
load balancing, traffic engineering, and intrusion detection, which require
estimating flow sizes and identifying heavy hitter flows. Existing works
suggest replacing counters with shorter multiplicative error \emph{estimators}
that improve the accuracy by fitting more of them within a given space.
However, such estimators impose a computational overhead that degrades the
measurement throughput. Instead, we propose \emph{additive} error estimators,
which are simpler, faster, and more accurate when used for network measurement.
Our solution is rigorously analyzed and empirically evaluated against several
other measurement algorithms on real Internet traces. For a given error target,
we improve the speed of the uncompressed solutions by -, and
the space by up to . Compared with existing state-of-the-art
estimators, our solution is - faster while being
considerably more accurate.Comment: To appear in IEEE INFOCOM 202
Per-host DDoS mitigation by direct-control reinforcement learning
DDoS attacks plague the availability of online services today, yet like many cybersecurity problems are evolving and non-stationary. Normal and attack patterns shift as new protocols and applications are introduced, further compounded by burstiness and seasonal variation. Accordingly, it is difficult to apply machine learning-based techniques and defences in practice. Reinforcement learning (RL) may overcome this detection problem for DDoS attacks by managing and monitoring consequences; an agent’s role is to learn to optimise performance criteria (which are always available) in an online manner. We advance the state-of-the-art in RL-based DDoS mitigation by introducing two agent classes designed to act on a per-flow basis, in a protocol-agnostic manner for any network topology. This is supported by an in-depth investigation of feature suitability and empirical evaluation. Our results show the existence of flow features with high predictive power for different traffic classes, when used as a basis for feedback-loop-like control. We show that the new RL agent models can offer a significant increase in goodput of legitimate TCP traffic for many choices of host density
Security Enhanced Applications for Information Systems
Every day, more users access services and electronically transmit information which is usually disseminated over insecure networks and processed by websites and databases, which lack proper security protection mechanisms and tools. This may have an impact on both the users’ trust as well as the reputation of the system’s stakeholders. Designing and implementing security enhanced systems is of vital importance. Therefore, this book aims to present a number of innovative security enhanced applications. It is titled “Security Enhanced Applications for Information Systems” and includes 11 chapters. This book is a quality guide for teaching purposes as well as for young researchers since it presents leading innovative contributions on security enhanced applications on various Information Systems. It involves cases based on the standalone, network and Cloud environments
Machine learning-driven credit risk: a systemic review
Credit risk assessment is at the core of modern economies. Traditionally, it is measured by statistical methods and manual auditing. Recent advances in financial artificial intelligence stemmed from a new wave of machine learning (ML)-driven credit risk models that gained tremendous attention from both industry and academia. In this paper, we systematically review a series of major research contributions (76 papers) over the past eight years using statistical, machine learning and deep learning techniques to address the problems of credit risk. Specifically, we propose a novel classification methodology for ML-driven credit risk algorithms and their performance ranking using public datasets. We further discuss the challenges including data imbalance, dataset inconsistency, model transparency, and inadequate utilization of deep learning models. The results of our review show that: 1) most deep learning models outperform classic machine learning and statistical algorithms in credit risk estimation, and 2) ensemble methods provide higher accuracy compared with single models. Finally, we present summary tables in terms of datasets and proposed models
Accurate and Resource-Efficient Monitoring for Future Networks
Monitoring functionality is a key component of any network management system. It is essential for profiling network resource usage, detecting attacks, and capturing the performance of a multitude of services using the network. Traditional monitoring solutions operate on long timescales producing periodic reports, which are mostly used for manual and infrequent network management tasks. However, these practices have been recently questioned by the advent of Software Defined Networking (SDN). By empowering management applications with the right tools to perform automatic, frequent, and fine-grained network reconfigurations, SDN has made these applications more dependent than before on the accuracy and timeliness of monitoring reports. As a result, monitoring systems are required to collect considerable amounts of heterogeneous measurement data, process them in real-time, and expose the resulting knowledge in short timescales to network decision-making processes. Satisfying these requirements is extremely challenging given today’s larger network scales, massive and dynamic traffic volumes, and the stringent constraints on time availability and hardware resources. This PhD thesis tackles this important challenge by investigating how an accurate and resource-efficient monitoring function can be realised in the context of future, software-defined networks. Novel monitoring methodologies, designs, and frameworks are provided in this thesis, which scale with increasing network sizes and automatically adjust to changes in the operating conditions. These achieve the goal of efficient measurement collection and reporting, lightweight measurement- data processing, and timely monitoring knowledge delivery
DiffGuard: Semantic Mismatch-Guided Out-of-Distribution Detection using Pre-trained Diffusion Models
Given a classifier, the inherent property of semantic Out-of-Distribution
(OOD) samples is that their contents differ from all legal classes in terms of
semantics, namely semantic mismatch. There is a recent work that directly
applies it to OOD detection, which employs a conditional Generative Adversarial
Network (cGAN) to enlarge semantic mismatch in the image space. While achieving
remarkable OOD detection performance on small datasets, it is not applicable to
ImageNet-scale datasets due to the difficulty in training cGANs with both input
images and labels as conditions. As diffusion models are much easier to train
and amenable to various conditions compared to cGANs, in this work, we propose
to directly use pre-trained diffusion models for semantic mismatch-guided OOD
detection, named DiffGuard. Specifically, given an OOD input image and the
predicted label from the classifier, we try to enlarge the semantic difference
between the reconstructed OOD image under these conditions and the original
input image. We also present several test-time techniques to further strengthen
such differences. Experimental results show that DiffGuard is effective on both
Cifar-10 and hard cases of the large-scale ImageNet, and it can be easily
combined with existing OOD detection techniques to achieve state-of-the-art OOD
detection results.Comment: Accepted by ICCV2023, with supplementary material
- …