Competing Mobile Network Game: Embracing antijamming and jamming strategies with reinforcement learning

We introduce Competing Mobile Network Game (CMNG), a stochastic game played by cognitive radio networks that compete for dominating an open spectrum access. Differentiated from existing approaches, we incorporate both communicator and jamming nodes to form a network for friendly coalition, integrate antijamming and jamming subgames into a stochastic framework, and apply Q-learning techniques to solve for an optimal channel access strategy. We empirically evaluate our Q-learning based strategies and find that Minimax-Q learning is more suitable for an aggressive environment than Nash-Q while Friend-or-foe Q-learning can provide the best solution under distributed mobile ad hoc networking scenarios in which the centralized control can hardly be available.Engineering and Applied Science

Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

A Survey on the Communication Protocols and Security in Cognitive Radio Networks

A cognitive radio (CR) is a radio that can change its transmission parameters based on the perceived availability of the spectrum bands in its operating environment. CRs support dynamic spectrum access and can facilitate a secondary unlicensed user to efficiently utilize the available underutilized spectrum allocated to the primary licensed users. A cognitive radio network (CRN) is composed of both the secondary users with CR-enabled radios and the primary users whose radios need not be CR-enabled. Most of the active research conducted in the area of CRNs has been so far focused on spectrum sensing, allocation and sharing. There is no comprehensive review paper available on the strategies for medium access control (MAC), routing and transport layer protocols, and the appropriate representative solutions for CRNs. In this paper, we provide an exhaustive analysis of the various techniques/mechanisms that have been proposed in the literature for communication protocols (at the MAC, routing and transport layers), in the context of a CRN, as well as discuss in detail several security attacks that could be launched on CRNs and the countermeasure solutions that have been proposed to avoid or mitigate them. This paper would serve as a good comprehensive review and analysis of the strategies for MAC, routing and transport protocols and security issues for CRNs as well as would lay a strong foundation for someone to further delve onto any particular aspect in greater depth

Opportunistic Spectrum Utilization by Cognitive Radio Networks: Challenges and Solutions

Cognitive Radio Network (CRN) is an emerging paradigm that makes use of Dynamic Spectrum Access (DSA) to communicate opportunistically, in the un-licensed Industrial, Scientific and Medical bands or frequency bands otherwise licensed to incumbent users such as TV broadcast. Interest in the development of CRNs is because of severe under-utilization of spectrum bands by the incumbent Primary Users (PUs) that have the license to use them coupled with an ever-increasing demand for unlicensed spectrum for a variety of new mobile and wireless applications. The essence of Cognitive Radio (CR) operation is the cooperative and opportunistic utilization of licensed spectrum bands by the Secondary Users (SUs) that collectively form the CRN without causing any interference to PUs\u27 communications. CRN operation is characterized by factors such as network-wide quiet periods for cooperative spectrum sensing, opportunistic/dynamic spectrum access and non-deterministic operation of PUs. These factors can have a devastating impact on the overall throughput and can significantly increase the control overheads. Therefore, to support the same level of QoS as traditional wireless access technologies, very closer interaction is required between layers of the protocol stack. Opportunistic spectrum utilization without causing interference to the PUs is only possible if the SUs periodically sense the spectrum for the presence of PUs\u27 signal. To minimize the effects of hardware capabilities, terrain features and PUs\u27 transmission ranges, DSA is undertaken in a collaborative manner where SUs periodically carry out spectrum sensing in their respective geographical locations. Collaborative spectrum sensing has numerous security loopholes and can be favorable to malicious nodes in the network that may exploit vulnerabilities associated with DSA such as launching a spectrum sensing data falsification (SSDF) attack. Some CRN standards such as the IEEE 802.22 wireless regional area network employ a two-stage quiet period mechanism based on a mandatory Fast Sensing and an optional Fine Sensing stage for DSA. This arrangement is meant to strike a balance between the conflicting goals of proper protection of incumbent PUs\u27 signals and optimum QoS for SUs so that only as much time is spent for spectrum sensing as needed. Malicious nodes in the CRN however, can take advantage of the two-stage spectrum sensing mechanism to launch smart denial of service (DoS) jamming attacks on CRNs during the fast sensing stage. Coexistence protocols enable collocated CRNs to contend for and share the available spectrum. However, most coexistence protocols do not take into consideration the fact that channels of the available spectrum can be heterogeneous in the sense that they can vary in their characteristics and quality such as SNR or bandwidth. Without any mechanism to enforce fairness in accessing varying quality channels, ensuring coexistence with minimal contention and efficient spectrum utilization for CRNs is likely to become a very difficult task. The cooperative and opportunistic nature of communication has many challenges associated with CRNs\u27 operation. In view of the challenges described above, this dissertation presents solutions including cross-layer approaches, reputation system, optimization and game theoretic approaches to handle (1) degradation in TCP\u27s throughput resulting from packet losses and disruptions in spectrum availability due non-deterministic use of spectrum by the PUs (2) presence of malicious SUs in the CRN that may launch various attacks on CRNs\u27 including SSDF and jamming and (3) sharing of heterogeneous spectrum resources among collocated CRNs without a centralized mechanism to enforce cooperation among otherwise non-cooperative CRN