Dynamic Multi-Arm Bandit Game Based Multi-Agents Spectrum Sharing Strategy Design

For a wireless avionics communication system, a Multi-arm bandit game is mathematically formulated, which includes channel states, strategies, and rewards. The simple case includes only two agents sharing the spectrum which is fully studied in terms of maximizing the cumulative reward over a finite time horizon. An Upper Confidence Bound (UCB) algorithm is used to achieve the optimal solutions for the stochastic Multi-Arm Bandit (MAB) problem. Also, the MAB problem can also be solved from the Markov game framework perspective. Meanwhile, Thompson Sampling (TS) is also used as benchmark to evaluate the proposed approach performance. Numerical results are also provided regarding minimizing the expectation of the regret and choosing the best parameter for the upper confidence bound

Reinforcement learning based anti-jamming schedule in cyber-physical systems

In this paper, the security issue of cyber-physical systems is investigated, where the observation data is transmitted from a sensor to an estimator through wireless channels disturbed by an attacker. The failure of this data transmission occurs, when the sensor accesses the channel that happens to be attacked by the jammer. Since the system performance measured by the estimation error depends on whether the data transmission is a success, the problem of selecting the channel to alleviate the attack effect is studied. Moreover, the state of each channel is time-variant due to various factors, such as path loss and shadowing. Motivated by energy conservation, the problem of selecting the channel with the best state is also considered. With the help of cognitive radio technique, the sensor has the ability of selecting a sequence of channels dynamically. Based on this, the problem of selecting the channel is resolved by means of reinforcement learning to jointly avoid the attack and enjoy the channel with the best state. A corresponding algorithm is presented to obtain the sequence of channels for the sensor, and its effectiveness is proved analytically. Numerical simulations further verify the derived results

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Deep Predictive Coding Neural Network for RF Anomaly Detection in Wireless Networks

Intrusion detection has become one of the most critical tasks in a wireless network to prevent service outages that can take long to fix. The sheer variety of anomalous events necessitates adopting cognitive anomaly detection methods instead of the traditional signature-based detection techniques. This paper proposes an anomaly detection methodology for wireless systems that is based on monitoring and analyzing radio frequency (RF) spectrum activities. Our detection technique leverages an existing solution for the video prediction problem, and uses it on image sequences generated from monitoring the wireless spectrum. The deep predictive coding network is trained with images corresponding to the normal behavior of the system, and whenever there is an anomaly, its detection is triggered by the deviation between the actual and predicted behavior. For our analysis, we use the images generated from the time-frequency spectrograms and spectral correlation functions of the received RF signal. We test our technique on a dataset which contains anomalies such as jamming, chirping of transmitters, spectrum hijacking, and node failure, and evaluate its performance using standard classifier metrics: detection ratio, and false alarm rate. Simulation results demonstrate that the proposed methodology effectively detects many unforeseen anomalous events in real time. We discuss the applications, which encompass industrial IoT, autonomous vehicle control and mission-critical communications services.Comment: 7 pages, 7 figures, Communications Workshop ICC'1

Software Defined Networks based Smart Grid Communication: A Comprehensive Survey

The current power grid is no longer a feasible solution due to ever-increasing user demand of electricity, old infrastructure, and reliability issues and thus require transformation to a better grid a.k.a., smart grid (SG). The key features that distinguish SG from the conventional electrical power grid are its capability to perform two-way communication, demand side management, and real time pricing. Despite all these advantages that SG will bring, there are certain issues which are specific to SG communication system. For instance, network management of current SG systems is complex, time consuming, and done manually. Moreover, SG communication (SGC) system is built on different vendor specific devices and protocols. Therefore, the current SG systems are not protocol independent, thus leading to interoperability issue. Software defined network (SDN) has been proposed to monitor and manage the communication networks globally. This article serves as a comprehensive survey on SDN-based SGC. In this article, we first discuss taxonomy of advantages of SDNbased SGC.We then discuss SDN-based SGC architectures, along with case studies. Our article provides an in-depth discussion on routing schemes for SDN-based SGC. We also provide detailed survey of security and privacy schemes applied to SDN-based SGC. We furthermore present challenges, open issues, and future research directions related to SDN-based SGC.Comment: Accepte