Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

A Mixed-Integer Programming Approach for Jammer Placement Problems for Flow-Jamming Attacks on Wireless Communication Networks

In this dissertation, we study an important problem of security in wireless networks. We study different attacks and defense strategies in general and more specifically jamming attacks. We begin the dissertation by providing a tutorial introducing the operations research community to the various types of attacks and defense strategies in wireless networks. In this tutorial, we give examples of mathematical programming models to model jamming attacks and defense against jamming attacks in wireless networks. Later we provide a comprehensive taxonomic classification of the various types of jamming attacks and defense against jamming attacks. The classification scheme will provide a one stop location for future researchers on various jamming attack and defense strategies studied in literature. This classification scheme also highlights the areas of research in jamming attack and defense against jamming attacks which have received less attention and could be a good area of focus for future research. In the next chapter, we provide a bi-level mathematical programming model to study jamming attack and defense strategy. We solve this using a game-theoretic approach and also study the impact of power level, location of jamming device, and the number of transmission channels available to transmit data on the attack and defense against jamming attacks. We show that by increasing the number of jamming devices the throughput of the network drops by at least 7%. Finally we study a special type of jamming attack, flow-jamming attack. We provide a mathematical programming model to solve the location of jamming devices to increase the impact of flow-jamming attacks on wireless networks. We provide a Benders decomposition algorithm along with some acceleration techniques to solve large problem instances in reasonable amount of time. We draw some insights about the impact of power, location and size of the network on the impact of flow-jamming attacks in wireless networks

A Mixed-Integer Programming Approach for Jammer Placement Problems for Flow-Jamming Attacks on Wireless Communication Networks

In this dissertation, we study an important problem of security in wireless networks. We study different attacks and defense strategies in general and more specifically jamming attacks. We begin the dissertation by providing a tutorial introducing the operations research community to the various types of attacks and defense strategies in wireless networks. In this tutorial, we give examples of mathematical programming models to model jamming attacks and defense against jamming attacks in wireless networks. Later we provide a comprehensive taxonomic classification of the various types of jamming attacks and defense against jamming attacks. The classification scheme will provide a one stop location for future researchers on various jamming attack and defense strategies studied in literature. This classification scheme also highlights the areas of research in jamming attack and defense against jamming attacks which have received less attention and could be a good area of focus for future research. In the next chapter, we provide a bi-level mathematical programming model to study jamming attack and defense strategy. We solve this using a game-theoretic approach and also study the impact of power level, location of jamming device, and the number of transmission channels available to transmit data on the attack and defense against jamming attacks. We show that by increasing the number of jamming devices the throughput of the network drops by at least 7%. Finally we study a special type of jamming attack, flow-jamming attack. We provide a mathematical programming model to solve the location of jamming devices to increase the impact of flow-jamming attacks on wireless networks. We provide a Benders decomposition algorithm along with some acceleration techniques to solve large problem instances in reasonable amount of time. We draw some insights about the impact of power, location and size of the network on the impact of flow-jamming attacks in wireless networks

Modeling Security and Cooperation in Wireless Networks Using Game Theory

This research involves the design, development, and theoretical demonstration of models resulting in integrated misbehavior resolution protocols for ad hoc networked devices. Game theory was used to analyze strategic interaction among independent devices with conflicting interests. Packet forwarding at the routing layer of autonomous ad hoc networks was investigated. Unlike existing reputation based or payment schemes, this model is based on repeated interactions. To enforce cooperation, a community enforcement mechanism was used, whereby selfish nodes that drop packets were punished not only by the victim, but also by all nodes in the network. Then, a stochastic packet forwarding game strategy was introduced. Our solution relaxed the uniform traffic demand that was pervasive in other works. To address the concerns of imperfect private monitoring in resource aware ad hoc networks, a belief-free equilibrium scheme was developed that reduces the impact of noise in cooperation. This scheme also eliminated the need to infer the private history of other nodes. Moreover, it simplified the computation of an optimal strategy. The belief-free approach reduced the node overhead and was easily tractable. Hence it made the system operation feasible. Motivated by the versatile nature of evolutionary game theory, the assumption of a rational node is relaxed, leading to the development of a framework for mitigating routing selfishness and misbehavior in Multi hop networks. This is accomplished by setting nodes to play a fixed strategy rather than independently choosing a rational strategy. A range of simulations was carried out that showed improved cooperation between selfish nodes when compared to older results. Cooperation among ad hoc nodes can also protect a network from malicious attacks. In the absence of a central trusted entity, many security mechanisms and privacy protections require cooperation among ad hoc nodes to protect a network from malicious attacks. Therefore, using game theory and evolutionary game theory, a mathematical framework has been developed that explores trust mechanisms to achieve security in the network. This framework is one of the first steps towards the synthesis of an integrated solution that demonstrates that security solely depends on the initial trust level that nodes have for each other

Markov decision processes with applications in wireless sensor networks: A survey

Ministry of Education, Singapore under its Academic Research Funding Tier

Contributions to Wireless multi-hop networks : Quality of Services and Security concerns

Ce document résume mes travaux de recherche conduits au cours de ces 6 dernières années. Le principal sujet de recherche de mes contributions est la conception et l’évaluation des solutions pour les réseaux sans fil multi-sauts en particulier les réseaux mobiles adhoc (MANETs), les réseaux véhiculaires ad hoc (VANETs), et les réseaux de capteurs sans fil (WSNs). La question clé de mes travaux de recherche est la suivante : « comment assurer un transport des données e cace en termes de qualité de services (QoS), de ressources énergétiques, et de sécurité dans les réseaux sans fil multi-sauts? » Pour répondre à cette question, j’ai travaillé en particulier sur les couches MAC et réseau et utilisé une approche inter-couches.Les réseaux sans fil multi-sauts présentent plusieurs problèmes liés à la gestion des ressources et au transport des données capable de supporter un grand nombre de nœuds, et d’assurer un haut niveau de qualité de service et de sécurité.Dans les réseaux MANETs, l’absence d’infrastructure ne permet pas d’utiliser l’approche centralisée pour gérer le partage des ressources, comme l’accès au canal.Contrairement au WLAN (réseau sans fil avec infrastructure), dans les réseaux Ad hoc les nœuds voisins deviennent concurrents et il est di cile d’assurer l’équité et l’optimisation du débit. La norme IEEE802.11 ne prend pas en compte l’équité entre les nœuds dans le contexte des MANETs. Bien que cette norme propose di érents niveaux de transmission, elle ne précise pas comment allouer ces débits de manière e cace. En outre, les MANETs sont basés sur le concept de la coopération entre les nœuds pour former et gérer un réseau. Le manque de coopération entre les nœuds signifie l’absence de tout le réseau. C’est pourquoi, il est primordial de trouver des solutions pour les nœuds non-coopératifs ou égoïstes. Enfin, la communication sans fil multi-sauts peut participer à l’augmentation de la couverture radio. Les nœuds de bordure doivent coopérer pour transmettre les paquets des nœuds voisins qui se trouvent en dehors de la zone de couverture de la station de base.Dans les réseaux VANETs, la dissémination des données pour les applications de sureté est un vrai défi. Pour assurer une distribution rapide et globale des informations, la méthode de transmission utilisée est la di usion. Cette méthode présente plusieurs inconvénients : perte massive des données due aux collisions, absence de confirmation de réception des paquets, non maîtrise du délai de transmission, et redondance de l’information. De plus, les applications de sureté transmettent des informations critiques, dont la fiabilité et l’authenticité doivent être assurées.Dans les réseaux WSNs, la limitation des ressources (bande passante, mémoire, énergie, et capacité de calcul), ainsi que le lien sans fil et la mobilité rendent la conception d’un protocole de communication e cace di cile. Certaines applications nécessitent un taux important de ressources (débit, énergie, etc) ainsi que des services de sécurité, comme la confidentialité et l’intégrité des données et l’authentification mutuelle. Ces paramètres sont opposés et leur conciliation est un véritable défi. De plus, pour transmettre de l’information, certaines applications ont besoin de connaître la position des nœuds dans le réseau. Les techniques de localisation sou rent d’un manque de précision en particulier dans un environnement fermé (indoor), et ne permettent pas de localiser les nœuds dans un intervalle de temps limité. Enfin, la localisation des nœuds est nécessaire pour assurer le suivi d’objet communicant ou non. Le suivi d’objet est un processus gourmand en énergie, et requiert de la précision.Pour répondre à ces défis, nous avons proposé et évalué des solutions, présentées de la manière suivante : l’ensemble des contributions dédiées aux réseaux MANETs est présenté dans le deuxième chapitre. Le troisième chapitre décrit les solutions apportées dans le cadre des réseaux VANETs. Enfin, les contributions liées aux réseaux WSNs sont présentées dans le quatrième chapitre

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig