Anti-Intelligent UAV Jamming Strategy via Deep Q-Networks

The downlink communications are vulnerable to intelligent unmanned aerial vehicle (UAV) jamming attack which can learn the optimal attack strategy in complex communication environments. In this paper, we propose an anti-intelligent UAV jamming strategy, in which the mobile users can learn the optimal defense strategy to prevent jamming. Specifically, the UAV jammer acts as a leader and the users act as followers. The problem is formulated as a stackelberg dynamic game, which includes the leader sub-game and the followers sub-game. As the UAV jammer is only aware of the incomplete channel state information (CSI) of the users, we model the leader sub-game as a partially observable Markov decision process (POMDP). The optimal jamming trajectory is obtained via deep recurrent Q-networks (DRQN) in the three-dimension space. For the followers sub-game, we use the Markov decision process (MDP) to model it. Then the optimal communication trajectory can be learned via deep Q-networks (DQN) in the two-dimension space. We prove the existence of the stackelberg equilibrium. The simulations show that the proposed strategy outperforms the benchmark strategies

Defeating Proactive Jammers Using Deep Reinforcement Learning for Resource-Constrained IoT Networks

Traditional anti-jamming techniques like spread spectrum, adaptive power/rate control, and cognitive radio, have demonstrated effectiveness in mitigating jamming attacks. However, their robustness against the growing complexity of internet-of-thing (IoT) networks and diverse jamming attacks is still limited. To address these challenges, machine learning (ML)-based techniques have emerged as promising solutions. By offering adaptive and intelligent anti-jamming capabilities, ML-based approaches can effectively adapt to dynamic attack scenarios and overcome the limitations of traditional methods. In this paper, we propose a deep reinforcement learning (DRL)-based approach that utilizes state input from realistic wireless network interface cards. We train five different variants of deep Q-network (DQN) agents to mitigate the effects of jamming with the aim of identifying the most sample-efficient, lightweight, robust, and least complex agent that is tailored for power-constrained devices. The simulation results demonstrate the effectiveness of the proposed DRL-based anti-jamming approach against proactive jammers, regardless of their jamming strategy which eliminates the need for a pattern recognition or jamming strategy detection step. Our findings present a promising solution for securing IoT networks against jamming attacks and highlights substantial opportunities for continued investigation and advancement within this field

Game Theory for Multi-Access Edge Computing:Survey, Use Cases, and Future Trends

Game theory (GT) has been used with significant success to formulate, and either design or optimize, the operation of many representative communications and networking scenarios. The games in these scenarios involve, as usual, diverse players with conflicting goals. This paper primarily surveys the literature that has applied theoretical games to wireless networks, emphasizing use cases of upcoming multiaccess edge computing (MEC). MEC is relatively new and offers cloud services at the network periphery, aiming to reduce service latency backhaul load, and enhance relevant operational aspects such as quality of experience or security. Our presentation of GT is focused on the major challenges imposed by MEC services over the wireless resources. The survey is divided into classical and evolutionary games. Then, our discussion proceeds to more specific aspects which have a considerable impact on the game's usefulness, namely, rational versus evolving strategies, cooperation among players, available game information, the way the game is played (single turn, repeated), the game's model evaluation, and how the model results can be applied for both optimizing resource-constrained resources and balancing diverse tradeoffs in real edge networking scenarios. Finally, we reflect on lessons learned, highlighting future trends and research directions for applying theoretical model games in upcoming MEC services, considering both network design issues and usage scenarios

Multi-agent Reinforcement Learning as Applied to Autonomous Systems

Multi-agent reinforcement learning (MARL) is a relatively unexplored area. Existing MARL methods and algorithms often scale poorly to the number of agents or suffer from the issue of non-stationary learning. This thesis aims to develop distributed training methods and algorithms for RL of the multi-agent autonomous systems that ensure scalability and stabilized learning.Specifically, we consider three common paradigms of multi-agent interaction: cooperative/ team, strategic/competitive, and leader-follower settings. We first study game-theoretic RL for the last two settings. Game-theoretic models are inherently distributed, and each agent takes other agents’ responses into account in the resulting game-theoretical equilibrium. Therefore, learning based on the game-theoretic equilibrium can effectively address the issue of non-stationary learning. We investigate Nash Q-learning in the strategic/competitive setting and Stackelberg Q-learning in the leader-follower setting, and apply them successfully to several applications that involve simple yet basic multi-agent interactions. We then propose a distributed deep Q-leaning for the cooperative/team setting, where each agent updates the estimate of her Q-value based on her own reward and her neighbors’ Q-values. We analyze the convergence of the proposed algorithm, characterize its performance gap to the centralized Q-learning, and evaluate it with a cooperative multi-agent navigation task

The University Defence Research Collaboration In Signal Processing

This chapter describes the development of algorithms for automatic detection of anomalies from multi-dimensional, undersampled and incomplete datasets. The challenge in this work is to identify and classify behaviours as normal or abnormal, safe or threatening, from an irregular and often heterogeneous sensor network. Many defence and civilian applications can be modelled as complex networks of interconnected nodes with unknown or uncertain spatio-temporal relations. The behavior of such heterogeneous networks can exhibit dynamic properties, reflecting evolution in both network structure (new nodes appearing and existing nodes disappearing), as well as inter-node relations. The UDRC work has addressed not only the detection of anomalies, but also the identification of their nature and their statistical characteristics. Normal patterns and changes in behavior have been incorporated to provide an acceptable balance between true positive rate, false positive rate, performance and computational cost. Data quality measures have been used to ensure the models of normality are not corrupted by unreliable and ambiguous data. The context for the activity of each node in complex networks offers an even more efficient anomaly detection mechanism. This has allowed the development of efficient approaches which not only detect anomalies but which also go on to classify their behaviour

Physical layer security for IoT applications

The increasing demands for Internet of things (IoT) applications and the tremendous increase in the volume of IoT generated data bring novel challenges for the fifth generation (5G) network. Verticals such as e-Health, vehicle to everything (V2X) and unmanned aerial vehicles (UAVs) require solutions that can guarantee low latency, energy efficiency,massive connectivity, and high reliability. In particular, finding strong security mechanisms that satisfy the above is of central importance for bringing the IoT to life. In this regards, employing physical layer security (PLS) methods could be greatly beneficial for IoT networks. While current security solutions rely on computational complexity, PLS is based on information theoretic proofs. By removing the need for computational power, PLS is ideally suited for resource constrained devices. In detail, PLS can ensure security using the inherit randomness already present in the physical channel. Promising schemes from the physical layer include physical unclonable functions (PUFs), which are seen as the hardware fingerprint of a device, and secret key generation (SKG) from wireless fading coefficients, which provide the wireless fingerprint of the communication channel between devices. The present thesis develops several PLS-based techniques that pave the way for a new breed of latency-aware, lightweight, security protocols. In particular, the work proposes: i) a fast multi-factor authentication solution with verified security properties based on PUFs, proximity detection and SKG; ii) an authenticated encryption SKG approach that interweaves data transmission and key generation; and, iii) a set of countermeasures to man-in-the-middle and jamming attacks. Overall, PLS solutions show promising performance, especially in the context of IoT applications, therefore, the advances in this thesis should be considered for beyond-5G networks

The University Defence Research Collaboration In Signal Processing: 2013-2018

Signal processing is an enabling technology crucial to all areas of defence and security. It is called for whenever humans and autonomous systems are required to interpret data (i.e. the signal) output from sensors. This leads to the production of the intelligence on which military outcomes depend. Signal processing should be timely, accurate and suited to the decisions to be made. When performed well it is critical, battle-winning and probably the most important weapon which you’ve never heard of. With the plethora of sensors and data sources that are emerging in the future network-enabled battlespace, sensing is becoming ubiquitous. This makes signal processing more complicated but also brings great opportunities. The second phase of the University Defence Research Collaboration in Signal Processing was set up to meet these complex problems head-on while taking advantage of the opportunities. Its unique structure combines two multi-disciplinary academic consortia, in which many researchers can approach different aspects of a problem, with baked-in industrial collaboration enabling early commercial exploitation. This phase of the UDRC will have been running for 5 years by the time it completes in March 2018, with remarkable results. This book aims to present those accomplishments and advances in a style accessible to stakeholders, collaborators and exploiters