Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Mobile Ad-Hoc Networks

Being infrastructure-less and without central administration control, wireless ad-hoc networking is playing a more and more important role in extending the coverage of traditional wireless infrastructure (cellular networks, wireless LAN, etc). This book includes state-of-the-art techniques and solutions for wireless ad-hoc networks. It focuses on the following topics in ad-hoc networks: quality-of-service and video communication, routing protocol and cross-layer design. A few interesting problems about security and delay-tolerant networks are also discussed. This book is targeted to provide network engineers and researchers with design guidelines for large scale wireless ad hoc networks

On distributed ledger technology for the internet of things: design and applications

Distributed ledger technology (DLT) can used to store information in such a way that no individual or organisation can compromise its veracity, contrary to a traditional centralised ledger. This nascent technology has received a great deal of attention from both researchers and practitioners in recent years due to the vast array of open questions related to its design and the assortment novel applications it unlocks. In this thesis, we are especially interested in the design of DLTs suitable for application in the domain of the internet of things (IoT), where factors such as efficiency, performance and scalability are of paramount importance. This work confronts the challenges of designing IoT-oriented distributed ledgers through analysis of ledger properties, development of design tools and the design of a number of core protocol components. We begin by introducing a class of DLTs whose data structures consist of directed acyclic graphs (DAGs) and which possess properties that make them particularly well suited to IoT applications. With a focus on the DAG structure, we then present analysis through mathematical modelling and simulations which provides new insights to the properties of this class of ledgers and allows us to propose novel security enhancements. Next, we shift our focus away from the DAG structure itself to another open problem for DAG-based distributed ledgers, that of access control. Specifically, we present a networking approach which removes the need for an expensive and inefficient mechanism known as Proof of Work, solving an open problem for IoT-oriented distributed ledgers. We then draw upon our analysis of the DAG structure to integrate and test our new access control with other core components of the DLT. Finally, we present a mechanism for orchestrating the interaction between users of a DLT and its operators, seeking to improves the usability of DLTs for IoT applications. In the appendix, we present two projects also carried out during this PhD which showcase applications of this technology in the IoT domain.Open Acces

OPTIMIZATION OF RAILWAY TRANSPORTATION HAZMATS AND REGULAR COMMODITIES

Transportation of dangerous goods has been receiving more attention in the realm of academic and scientific research during the last few decades as countries have been increasingly becoming industrialized throughout the world, thereby making Hazmats an integral part of our life style. However, the number of scholarly articles in this field is not as many as those of other areas in SCM. Considering the low-probability-and-high-consequence (LPHC) essence of transportation of Hazmats, on the one hand, and immense volume of shipments accounting for more than hundred tons in North America and Europe, on the other, we can safely state that the number of scholarly articles and dissertations have not been proportional to the significance of the subject of interest. On this ground, we conducted our research to contribute towards further developing the domain of Hazmats transportation, and sustainable supply chain management (SSCM), in general terms. Transportation of Hazmats, from logistical standpoint, may include all modes of transport via air, marine, road and rail, as well as intermodal transportation systems. Although road shipment is predominant in most of the literature, railway transportation of Hazmats has proven to be a potentially significant means of transporting dangerous goods with respect to both economies of scale and risk of transportation; these factors, have not just given rise to more thoroughly investigation of intermodal transportation of Hazmats using road and rail networks, but has encouraged the competition between rail and road companies which may indeed have some inherent advantages compared to the other medium due to their infrastructural and technological backgrounds. Truck shipment has ostensibly proven to be providing more flexibility; trains, per contra, provide more reliability in terms of transport risk for conveying Hazmats in bulks. In this thesis, in consonance with the aforementioned motivation, we provide an introduction into the hazardous commodities shipment through rail network in the first chapter of the thesis. Providing relevant statistics on the volume of Hazmat goods, number of accidents, rate of incidents, and rate of fatalities and injuries due to the incidents involving Hazmats, will shed light onto the significance of the topic under study. As well, we review the most pertinent articles while putting more emphasis on the state-of-the-art papers, in chapter two. Following the discussion in chapter 3 and looking at the problem from carrier company’s perspective, a mixed integer quadratically constraint problem (MIQCP) is developed which seeks for the minimization of transportation cost under a set of constraints including those associating with Hazmats. Due to the complexity of the problem, the risk function has been piecewise linearized using a set of auxiliary variables, thereby resulting in an MIP problem. Further, considering the interests of both carrier companies and regulatory agencies, which are minimization of cost and risk, respectively, a multiobjective MINLP model is developed, which has been reduced to an MILP through piecewise linearization of the risk term in the objective function. For both single-objective and multiobjective formulations, model variants with bifurcated and nonbifurcated flows have been presented. Then, in chapter 4, we carry out experiments considering two main cases where the first case presents smaller instances of the problem and the second case focuses on a larger instance of the problem. Eventually, in chapter five, we conclude the dissertation with a summary of the overall discussion as well as presenting some comments on avenues of future work

Revolutionizing Future Connectivity: A Contemporary Survey on AI-empowered Satellite-based Non-Terrestrial Networks in 6G

Non-Terrestrial Networks (NTN) are expected to be a critical component of 6th Generation (6G) networks, providing ubiquitous, continuous, and scalable services. Satellites emerge as the primary enabler for NTN, leveraging their extensive coverage, stable orbits, scalability, and adherence to international regulations. However, satellite-based NTN presents unique challenges, including long propagation delay, high Doppler shift, frequent handovers, spectrum sharing complexities, and intricate beam and resource allocation, among others. The integration of NTNs into existing terrestrial networks in 6G introduces a range of novel challenges, including task offloading, network routing, network slicing, and many more. To tackle all these obstacles, this paper proposes Artificial Intelligence (AI) as a promising solution, harnessing its ability to capture intricate correlations among diverse network parameters. We begin by providing a comprehensive background on NTN and AI, highlighting the potential of AI techniques in addressing various NTN challenges. Next, we present an overview of existing works, emphasizing AI as an enabling tool for satellite-based NTN, and explore potential research directions. Furthermore, we discuss ongoing research efforts that aim to enable AI in satellite-based NTN through software-defined implementations, while also discussing the associated challenges. Finally, we conclude by providing insights and recommendations for enabling AI-driven satellite-based NTN in future 6G networks.Comment: 40 pages, 19 Figure, 10 Tables, Surve

ENSURING SPECIFICATION COMPLIANCE, ROBUSTNESS, AND SECURITY OF WIRELESS NETWORK PROTOCOLS

Several newly emerged wireless technologies (e.g., Internet-of-Things, Bluetooth, NFC)—extensively backed by the tech industry—are being widely adopted and have resulted in a proliferation of diverse smart appliances and gadgets (e.g., smart thermostat, wearables, smartphones), which has ensuingly shaped our modern digital life. These technologies include several communication protocols that usually have stringent requirements stated in their specifications. Failing to comply with such requirements can result in incorrect behaviors, interoperability issues, or even security vulnerabilities. Moreover, lack of robustness of the protocol implementation to malicious attacks—exploiting subtle vulnerabilities in the implementation—mounted by the compromised nodes in an adversarial environment can limit the practical utility of the implementation by impairing the performance of the protocol and can even have detrimental effects on the availability of the network. Even having a compliant and robust implementation alone may not suffice in many cases because these technologies often expose new attack surfaces as well as new propagation vectors, which can be exploited by unprecedented malware and can quickly lead to an epidemic

Emerging Communications for Wireless Sensor Networks

Wireless sensor networks are deployed in a rapidly increasing number of arenas, with uses ranging from healthcare monitoring to industrial and environmental safety, as well as new ubiquitous computing devices that are becoming ever more pervasive in our interconnected society. This book presents a range of exciting developments in software communication technologies including some novel applications, such as in high altitude systems, ground heat exchangers and body sensor networks. Authors from leading institutions on four continents present their latest findings in the spirit of exchanging information and stimulating discussion in the WSN community worldwide

Security and Energy Efficiency in Resource-Constrained Wireless Multi-hop Networks

In recent decades, there has been a huge improvement and interest from the research community in wireless multi-hop networks. Such networks have widespread applications in civil, commercial and military applications. Paradigms of this type of networks that are critical for many aspects of human lives are mobile ad-hoc networks, sensor networks, which are used for monitoring buildings and large agricultural areas, and vehicular networks with applications in traffic monitoring and regulation. Internet of Things (IoT) is also envisioned as a multi-hop network consisting of small interconnected devices, called ``things", such as smart meters, smart traffic lights, thermostats etc. Wireless multi-hop networks suffer from resource constraints, because all the devices have limited battery, computational power and memory. Battery level of these devices should be preserved in order to ensure reliability and communication across the network. In addition, these devices are not a priori designed to defend against sophisticated adversaries, which may be deployed across the network in order to disrupt network operation. In addition, the distributed nature of this type of networks introduces another limitation to protocol performance in the presence of adversaries. Hence, the inherit nature of this type of networks poses severe limitations on designing and optimizing protocols and network operations. In this dissertation, we focus on proposing novel techniques for designing more resilient protocols to attackers and more energy efficient protocols. In the first part of the dissertation, we investigate the scenario of multiple adversaries deployed across the network, which reduce significantly the network performance. We adopt a component-based and a cross-layer view of network protocols to make protocols secure and resilient to attacks and to utilize our techniques across existing network protocols. We use the notion of trust between network entities to propose lightweight defense mechanisms, which also satisfy performance requirements. Using cryptographic primitives in our network scenario can introduce significant computational overhead. In addition, behavioral aspects of entities are not captured by cryptographic primitives. Hence, trust metrics provide an efficient security metric in these scenarios, which can be utilized to introduce lightweight defense mechanisms applicable to deployed network protocols. In the second part of the dissertation, we focus on energy efficiency considerations in this type of networks. Our motivation for this work is to extend network lifetime, but at the same time maintain critical performance requirements. We propose a distributed sleep management framework for heterogeneous machine-to-machine networks and two novel energy efficient metrics. This framework and the routing metrics are integrated into existing routing protocols for machine-to-machine networks. We demonstrate the efficiency of our approach in terms of increasing network lifetime and maintaining packet delivery ratio. Furthermore, we propose a novel multi-metric energy efficient routing protocol for dynamic networks (i.e. mobile ad-hoc networks) and illustrate its performance in terms of network lifetime. Finally, we investigate the energy-aware sensor coverage problem and we propose a novel game theoretic approach to capture the tradeoff between sensor coverage efficiency and energy consumption

Multi-Robot Path Planning for Persistent Monitoring in Stochastic and Adversarial Environments

In this thesis, we study multi-robot path planning problems for persistent monitoring tasks. The goal of such persistent monitoring tasks is to deploy a team of cooperating mobile robots in an environment to continually observe locations of interest in the environment. Robots patrol the environment in order to detect events arriving at the locations of the environment. The events stay at those locations for a certain amount of time before leaving and can only be detected if one of the robots visits the location of an event while the event is there. In order to detect all possible events arriving at a vertex, the maximum time spent by the robots between visits to that vertex should be less than the duration of the events arriving at that vertex. We consider the problem of finding the minimum number of robots to satisfy these revisit time constraints, also called latency constraints. The decision version of this problem is PSPACE-complete. We provide an O(log p) approximation algorithm for this problem where p is the ratio of the maximum and minimum latency constraints. We also present heuristic algorithms to solve the problem and show through simulations that a proposed orienteering-based heuristic algorithm gives better solutions than the approximation algorithm. We additionally provide an algorithm for the problem of minimizing the maximum weighted latency given a fixed number of robots. In case the event stay durations are not fixed but are drawn from a known distribution, we consider the problem of maximizing the expected number of detected events. We motivate randomized patrolling paths for such scenarios and use Markov chains to represent those random patrolling paths. We characterize the expected number of detected events as a function of the Markov chains used for patrolling and show that the objective function is submodular for randomly arriving events. We propose an approximation algorithm for the case where the event durations for all the vertices is a constant. We also propose a centralized and an online distributed algorithm to find the random patrolling policies for the robots. We also consider the case where the events are adversarial and can choose where and when to appear in order to maximize their chances of remaining undetected. The last problem we study in this thesis considers events triggered by a learning adversary. The adversary has a limited time to observe the patrolling policy before it decides when and where events should appear. We study the single robot version of this problem and model this problem as a multi-stage two player game. The adversary observes the patroller’s actions for a finite amount of time to learn the patroller’s strategy and then either chooses a location for the event to appear or reneges based on its confidence in the learned strategy. We characterize the expected payoffs for the players and propose a search algorithm to find a patrolling policy in such scenarios. We illustrate the trade off between hard to learn and hard to attack strategies through simulations