Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Game-Theoretic Frameworks and Strategies for Defense Against Network Jamming and Collocation Attacks

Modern networks are becoming increasingly more complex, heterogeneous, and densely connected. While more diverse services are enabled to an ever-increasing number of users through ubiquitous networking and pervasive computing, several important challenges have emerged. For example, densely connected networks are prone to higher levels of interference, which makes them more vulnerable to jamming attacks. Also, the utilization of software-based protocols to perform routing, load balancing and power management functions in Software-Defined Networks gives rise to more vulnerabilities that could be exploited by malicious users and adversaries. Moreover, the increased reliance on cloud computing services due to a growing demand for communication and computation resources poses formidable security challenges due to the shared nature and virtualization of cloud computing. In this thesis, we study two types of attacks: jamming attacks on wireless networks and side-channel attacks on cloud computing servers. The former attacks disrupt the natural network operation by exploiting the static topology and dynamic channel assignment in wireless networks, while the latter attacks seek to gain access to unauthorized data by co-residing with target virtual machines (VMs) on the same physical node in a cloud server. In both attacks, the adversary faces a static attack surface and achieves her illegitimate goal by exploiting a stationary aspect of the network functionality. Hence, this dissertation proposes and develops counter approaches to both attacks using moving target defense strategies. We study the strategic interactions between the adversary and the network administrator within a game-theoretic framework. First, in the context of jamming attacks, we present and analyze a game-theoretic formulation between the adversary and the network defender. In this problem, the attack surface is the network connectivity (the static topology) as the adversary jams a subset of nodes to increase the level of interference in the network. On the other side, the defender makes judicious adjustments of the transmission footprint of the various nodes, thereby continuously adapting the underlying network topology to reduce the impact of the attack. The defender\u27s strategy is based on playing Nash equilibrium strategies securing a worst-case network utility. Moreover, scalable decomposition-based approaches are developed yielding a scalable defense strategy whose performance closely approaches that of the non-decomposed game for large-scale and dense networks. We study a class of games considering discrete as well as continuous power levels. In the second problem, we consider multi-tenant clouds, where a number of VMs are typically collocated on the same physical machine to optimize performance and power consumption and maximize profit. This increases the risk of a malicious virtual machine performing side-channel attacks and leaking sensitive information from neighboring VMs. The attack surface, in this case, is the static residency of VMs on a set of physical nodes, hence we develop a timed migration defense approach. Specifically, we analyze a timing game in which the cloud provider decides when to migrate a VM to a different physical machine to mitigate the risk of being compromised by a collocated malicious VM. The adversary decides the rate at which she launches new VMs to collocate with the victim VMs. Our formulation captures a data leakage model in which the cost incurred by the cloud provider depends on the duration of collocation with malicious VMs. It also captures costs incurred by the adversary in launching new VMs and by the defender in migrating VMs. We establish sufficient conditions for the existence of Nash equilibria for general cost functions, as well as for specific instantiations, and characterize the best response for both players. Furthermore, we extend our model to characterize its impact on the attacker\u27s payoff when the cloud utilizes intrusion detection systems that detect side-channel attacks. Our theoretical findings are corroborated with extensive numerical results in various settings as well as a proof-of-concept implementation in a realistic cloud setting

Opportunistic Spectrum Utilization by Cognitive Radio Networks: Challenges and Solutions

Cognitive Radio Network (CRN) is an emerging paradigm that makes use of Dynamic Spectrum Access (DSA) to communicate opportunistically, in the un-licensed Industrial, Scientific and Medical bands or frequency bands otherwise licensed to incumbent users such as TV broadcast. Interest in the development of CRNs is because of severe under-utilization of spectrum bands by the incumbent Primary Users (PUs) that have the license to use them coupled with an ever-increasing demand for unlicensed spectrum for a variety of new mobile and wireless applications. The essence of Cognitive Radio (CR) operation is the cooperative and opportunistic utilization of licensed spectrum bands by the Secondary Users (SUs) that collectively form the CRN without causing any interference to PUs\u27 communications. CRN operation is characterized by factors such as network-wide quiet periods for cooperative spectrum sensing, opportunistic/dynamic spectrum access and non-deterministic operation of PUs. These factors can have a devastating impact on the overall throughput and can significantly increase the control overheads. Therefore, to support the same level of QoS as traditional wireless access technologies, very closer interaction is required between layers of the protocol stack. Opportunistic spectrum utilization without causing interference to the PUs is only possible if the SUs periodically sense the spectrum for the presence of PUs\u27 signal. To minimize the effects of hardware capabilities, terrain features and PUs\u27 transmission ranges, DSA is undertaken in a collaborative manner where SUs periodically carry out spectrum sensing in their respective geographical locations. Collaborative spectrum sensing has numerous security loopholes and can be favorable to malicious nodes in the network that may exploit vulnerabilities associated with DSA such as launching a spectrum sensing data falsification (SSDF) attack. Some CRN standards such as the IEEE 802.22 wireless regional area network employ a two-stage quiet period mechanism based on a mandatory Fast Sensing and an optional Fine Sensing stage for DSA. This arrangement is meant to strike a balance between the conflicting goals of proper protection of incumbent PUs\u27 signals and optimum QoS for SUs so that only as much time is spent for spectrum sensing as needed. Malicious nodes in the CRN however, can take advantage of the two-stage spectrum sensing mechanism to launch smart denial of service (DoS) jamming attacks on CRNs during the fast sensing stage. Coexistence protocols enable collocated CRNs to contend for and share the available spectrum. However, most coexistence protocols do not take into consideration the fact that channels of the available spectrum can be heterogeneous in the sense that they can vary in their characteristics and quality such as SNR or bandwidth. Without any mechanism to enforce fairness in accessing varying quality channels, ensuring coexistence with minimal contention and efficient spectrum utilization for CRNs is likely to become a very difficult task. The cooperative and opportunistic nature of communication has many challenges associated with CRNs\u27 operation. In view of the challenges described above, this dissertation presents solutions including cross-layer approaches, reputation system, optimization and game theoretic approaches to handle (1) degradation in TCP\u27s throughput resulting from packet losses and disruptions in spectrum availability due non-deterministic use of spectrum by the PUs (2) presence of malicious SUs in the CRN that may launch various attacks on CRNs\u27 including SSDF and jamming and (3) sharing of heterogeneous spectrum resources among collocated CRNs without a centralized mechanism to enforce cooperation among otherwise non-cooperative CRN