Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Mining Network Events using Traceroute Empathy

In the never-ending quest for tools that enable an ISP to smooth troubleshooting and improve awareness of network behavior, very much effort has been devoted in the collection of data by active and passive measurement at the data plane and at the control plane level. Exploitation of collected data has been mostly focused on anomaly detection and on root-cause analysis. Our objective is somewhat in the middle. We consider traceroutes collected by a network of probes and aim at introducing a practically applicable methodology to quickly spot measurements that are related to high-impact events happened in the network. Such filtering process eases further in- depth human-based analysis, for example with visual tools which are effective only when handling a limited amount of data. We introduce the empathy relation between traceroutes as the cornerstone of our formal characterization of the traceroutes related to a network event. Based on this model, we describe an algorithm that finds traceroutes related to high-impact events in an arbitrary set of measurements. Evidence of the effectiveness of our approach is given by experimental results produced on real-world data.Comment: 8 pages, 7 figures, extended version of Discovering High-Impact Routing Events using Traceroutes, in Proc. 20th International Symposium on Computers and Communications (ISCC 2015

Global Modeling and Prediction of Computer Network Traffic

We develop a probabilistic framework for global modeling of the traffic over a computer network. This model integrates existing single-link (-flow) traffic models with the routing over the network to capture the global traffic behavior. It arises from a limit approximation of the traffic fluctuations as the time--scale and the number of users sharing the network grow. The resulting probability model is comprised of a Gaussian and/or a stable, infinite variance components. They can be succinctly described and handled by certain 'space-time' random fields. The model is validated against simulated and real data. It is then applied to predict traffic fluctuations over unobserved links from a limited set of observed links. Further, applications to anomaly detection and network management are briefly discussed

Security, trust and cooperation in wireless sensor networks

Wireless sensor networks are a promising technology for many real-world applications such as critical infrastructure monitoring, scientific data gathering, smart buildings, etc.. However, given the typically unattended and potentially unsecured operation environment, there has been an increased number of security threats to sensor networks. In addition, sensor networks have very constrained resources, such as limited energy, memory, computational power, and communication bandwidth. These unique challenges call for new security mechanisms and algorithms. In this dissertation, we propose novel algorithms and models to address some important and challenging security problems in wireless sensor networks. The first part of the dissertation focuses on data trust in sensor networks. Since sensor networks are mainly deployed to monitor events and report data, the quality of received data must be ensured in order to make meaningful inferences from sensor data. We first study a false data injection attack in the distributed state estimation problem and propose a distributed Bayesian detection algorithm, which could maintain correct estimation results when less than one half of the sensors are compromised. To deal with the situation where more than one half of the sensors may be compromised, we introduce a special class of sensor nodes called \textit{trusted cores}. We then design a secure distributed trust aggregation algorithm that can utilize the trusted cores to improve network robustness. We show that as long as there exist some paths that can connect each regular node to one of these trusted cores, the network can not be subverted by attackers. The second part of the dissertation focuses on sensor network monitoring and anomaly detection. A sensor network may suffer from system failures due to loss of links and nodes, or malicious intrusions. Therefore, it is critical to continuously monitor the overall state of the network and locate performance anomalies. The network monitoring and probe selection problem is formulated as a budgeted coverage problem and a Markov decision process. Efficient probing strategies are designed to achieve a flexible tradeoff between inference accuracy and probing overhead. Based on the probing results on traffic measurements, anomaly detection can be conducted. To capture the highly dynamic network traffic, we develop a detection scheme based on multi-scale analysis of the traffic using wavelet transforms and hidden Markov models. The performance of the probing strategy and of the detection scheme are extensively evaluated in malicious scenarios using the NS-2 network simulator. Lastly, to better understand the role of trust in sensor networks, a game theoretic model is formulated to mathematically analyze the relation between trust and cooperation. Given the trust relations, the interactions among nodes are modeled as a network game on a trust-weighted graph. We then propose an efficient heuristic method that explores network heterogeneity to improve Nash equilibrium efficiency

Transforming Bell's Inequalities into State Classifiers with Machine Learning

Quantum information science has profoundly changed the ways we understand, store, and process information. A major challenge in this field is to look for an efficient means for classifying quantum state. For instance, one may want to determine if a given quantum state is entangled or not. However, the process of a complete characterization of quantum states, known as quantum state tomography, is a resource-consuming operation in general. An attractive proposal would be the use of Bell's inequalities as an entanglement witness, where only partial information of the quantum state is needed. The problem is that entanglement is necessary but not sufficient for violating Bell's inequalities, making it an unreliable state classifier. Here we aim at solving this problem by the methods of machine learning. More precisely, given a family of quantum states, we randomly picked a subset of it to construct a quantum-state classifier, accepting only partial information of each quantum state. Our results indicated that these transformed Bell-type inequalities can perform significantly better than the original Bell's inequalities in classifying entangled states. We further extended our analysis to three-qubit and four-qubit systems, performing classification of quantum states into multiple species. These results demonstrate how the tools in machine learning can be applied to solving problems in quantum information science

A Model for Seasonal Dynamic Networks

Sociotechnological and geospatial processes exhibit time varying structure that make insight discovery challenging. This paper presents statistical model of systems with seasonal dynamics, modeled as a dynamic network, to address this challenge. It assumes the probability of edge formations depend on a type assigned to incident nodes and the current time. Time dependencies are modeled by unique seasonal processes. The model is studied on several synthetic and real datasets. Superior fidelity of this model on seasonal datasets compared to existing network models, while being able to remain equally accurate for networks with randomly changing structure, is shown. The model is found to be twice as accurate at predicting future edge counts over competing models on New York City taxi trips, U.S. airline flights, and email communication within the Enron company. An anomaly detection use case for the model is shown for NYC traffic dynamics and email communications between Enron employees