Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

A Cognitive Framework to Secure Smart Cities

The advancement in technology has transformed Cyber Physical Systems and their interface with IoT into a more sophisticated and challenging paradigm. As a result, vulnerabilities and potential attacks manifest themselves considerably more than before, forcing researchers to rethink the conventional strategies that are currently in place to secure such physical systems. This manuscript studies the complex interweaving of sensor networks and physical systems and suggests a foundational innovation in the field. In sharp contrast with the existing IDS and IPS solutions, in this paper, a preventive and proactive method is employed to stay ahead of attacks by constantly monitoring network data patterns and identifying threats that are imminent. Here, by capitalizing on the significant progress in processing power (e.g. petascale computing) and storage capacity of computer systems, we propose a deep learning approach to predict and identify various security breaches that are about to occur. The learning process takes place by collecting a large number of files of different types and running tests on them to classify them as benign or malicious. The prediction model obtained as such can then be used to identify attacks. Our project articulates a new framework for interactions between physical systems and sensor networks, where malicious packets are repeatedly learned over time while the system continually operates with respect to imperfect security mechanisms

An Energy Aware and Secure MAC Protocol for Tackling Denial of Sleep Attacks in Wireless Sensor Networks

Wireless sensor networks which form part of the core for the Internet of Things consist of resource constrained sensors that are usually powered by batteries. Therefore, careful energy awareness is essential when working with these devices. Indeed,the introduction of security techniques such as authentication and encryption, to ensure confidentiality and integrity of data, can place higher energy load on the sensors. However, the absence of security protection c ould give room for energy drain attacks such as denial of sleep attacks which have a higher negative impact on the life span ( of the sensors than the presence of security features. This thesis, therefore, focuses on tackling denial of sleep attacks from two perspectives A security perspective and an energy efficiency perspective. The security perspective involves evaluating and ranking a number of security based techniques to curbing denial of sleep attacks. The energy efficiency perspective, on the other hand, involves exploring duty cycling and simulating three Media Access Control ( protocols Sensor MAC, Timeout MAC andTunableMAC under different network sizes and measuring different parameters such as the Received Signal Strength RSSI) and Link Quality Indicator ( Transmit power, throughput and energy efficiency Duty cycling happens to be one of the major techniques for conserving energy in wireless sensor networks and this research aims to answer questions with regards to the effect of duty cycles on the energy efficiency as well as the throughput of three duty cycle protocols Sensor MAC ( Timeout MAC ( and TunableMAC in addition to creating a novel MAC protocol that is also more resilient to denial of sleep a ttacks than existing protocols. The main contributions to knowledge from this thesis are the developed framework used for evaluation of existing denial of sleep attack solutions and the algorithms which fuel the other contribution to knowledge a newly developed protocol tested on the Castalia Simulator on the OMNET++ platform. The new protocol has been compared with existing protocols and has been found to have significant improvement in energy efficiency and also better resilience to denial of sleep at tacks Part of this research has been published Two conference publications in IEEE Explore and one workshop paper