A Survey on Delay-Aware Resource Control for Wireless Systems --- Large Deviation Theory, Stochastic Lyapunov Drift and Distributed Stochastic Learning

In this tutorial paper, a comprehensive survey is given on several major systematic approaches in dealing with delay-aware control problems, namely the equivalent rate constraint approach, the Lyapunov stability drift approach and the approximate Markov Decision Process (MDP) approach using stochastic learning. These approaches essentially embrace most of the existing literature regarding delay-aware resource control in wireless systems. They have their relative pros and cons in terms of performance, complexity and implementation issues. For each of the approaches, the problem setup, the general solution and the design methodology are discussed. Applications of these approaches to delay-aware resource allocation are illustrated with examples in single-hop wireless networks. Furthermore, recent results regarding delay-aware multi-hop routing designs in general multi-hop networks are elaborated. Finally, the delay performance of the various approaches are compared through simulations using an example of the uplink OFDMA systems.Comment: 58 pages, 8 figures; IEEE Transactions on Information Theory, 201

Two Timescale Convergent Q-learning for Sleep--Scheduling in Wireless Sensor Networks

In this paper, we consider an intrusion detection application for Wireless Sensor Networks (WSNs). We study the problem of scheduling the sleep times of the individual sensors to maximize the network lifetime while keeping the tracking error to a minimum. We formulate this problem as a partially-observable Markov decision process (POMDP) with continuous state-action spaces, in a manner similar to (Fuemmeler and Veeravalli [2008]). However, unlike their formulation, we consider infinite horizon discounted and average cost objectives as performance criteria. For each criterion, we propose a convergent on-policy Q-learning algorithm that operates on two timescales, while employing function approximation to handle the curse of dimensionality associated with the underlying POMDP. Our proposed algorithm incorporates a policy gradient update using a one-simulation simultaneous perturbation stochastic approximation (SPSA) estimate on the faster timescale, while the Q-value parameter (arising from a linear function approximation for the Q-values) is updated in an on-policy temporal difference (TD) algorithm-like fashion on the slower timescale. The feature selection scheme employed in each of our algorithms manages the energy and tracking components in a manner that assists the search for the optimal sleep-scheduling policy. For the sake of comparison, in both discounted and average settings, we also develop a function approximation analogue of the Q-learning algorithm. This algorithm, unlike the two-timescale variant, does not possess theoretical convergence guarantees. Finally, we also adapt our algorithms to include a stochastic iterative estimation scheme for the intruder's mobility model. Our simulation results on a 2-dimensional network setting suggest that our algorithms result in better tracking accuracy at the cost of only a few additional sensors, in comparison to a recent prior work

A Virtual Infrastructure for Mitigating Typical Challenges in Sensor Networks

Sensor networks have their own distinguishing characteristics that set them apart from other types of networks. Typically, the sensors are deployed in large numbers and in random fashion and the resulting sensor network is expected to self-organize in support of the mission for which it was deployed. Because of the random deployment of sensors that are often scattered from an overflying aircraft, the resulting network is not easy to manage since the sensors do not know their location, do not know how to aggregate their sensory data and where and how to route the aggregated data. The limited energy budget available to sensors makes things much worse. To save their energy, sensors have to sleep and wake up asynchronously. However, while promoting energy awareness, these actions continually change the underlying network topology and make the basic network protocols more complex. Several techniques have been proposed in different areas of sensor networks. Most of these techniques attempt to solve one problem in isolation from the others, hence protocol designers have to face the same common challenges again and again. This, in turn, has a direct impact on the complexity of the proposed protocols and on energy consumption. Instead of using this approach we propose to construct a lightweight backbone that can help mitigate many of the typical challenges in sensor networks and allow the development of simpler network protocols. Our backbone construction protocol starts by tiling the area around each sink using identical regular hexagons. After that, the closest sensor to the center of each of these hexagons is determined—we refer to these sensors as backbone sensors. We define a ternary coordinate system to refer to hexagons. The resulting system provides a complete set of communication paths that can be used by any geographic routing technique to simplify data communication across the network. We show how the constructed backbone can help mitigate many of the typical challenges inherent to sensor networks. In addition to sensor localization, the network backbone provides an implicit clustering mechanism in which each hexagon represents a cluster mud the backbone sensor around its center represents the cluster head. As cluster heads, backbone sensors can be used to coordinate task assignment, workforce selection, and data aggregation for different sensing tasks. They also can be used to locally synchronize and adjust the duty cycle of non-backbone sensors in their neighborhood. Finally, we propose “Backbone Switching”, a technique that creates alternative backbones and periodically switches between them in order to balance energy consumption among sensors by distributing the additional load of being part of the backbone over larger number of sensors

Reinforcement Learning in Self Organizing Cellular Networks

Self-organization is a key feature as cellular networks densify and become more heterogeneous, through the additional small cells such as pico and femtocells. Self- organizing networks (SONs) can perform self-configuration, self-optimization, and self-healing. These operations can cover basic tasks such as the configuration of a newly installed base station, resource management, and fault management in the network. In other words, SONs attempt to minimize human intervention where they use measurements from the network to minimize the cost of installation, configuration, and maintenance of the network. In fact, SONs aim to bring two main factors in play: intelligence and autonomous adaptability. One of the main requirements for achieving such goals is to learn from sensory data and signal measurements in networks. Therefore, machine learning techniques can play a major role in processing underutilized sensory data to enhance the performance of SONs. In the first part of this dissertation, we focus on reinforcement learning as a viable approach for learning from signal measurements. We develop a general framework in heterogeneous cellular networks agnostic to the learning approach. We design multiple reward functions and study different effects of the reward function, Markov state model, learning rate, and cooperation methods on the performance of reinforcement learning in cellular networks. Further, we look into the optimality of reinforcement learning solutions and provide insights into how to achieve optimal solutions. In the second part of the dissertation, we propose a novel architecture based on spatial indexing for system-evaluation of heterogeneous 5G cellular networks. We develop an open-source platform based on the proposed architecture that can be used to study large scale directional cellular networks. The proposed platform is used for generating training data sets of accurate signal-to-interference-plus-noise-ratio (SINR) values in millimeter-wave communications for machine learning purposes. Then, with taking advantage of the developed platform, we look into dense millimeter-wave networks as one of the key technologies in 5G cellular networks. We focus on topology management of millimeter-wave backhaul networks and study and provide multiple insights on the evaluation and selection of proper performance metrics in dense millimeter-wave networks. Finally, we finish this part by proposing a self-organizing solution to achieve k-connectivity via reinforcement learning in the topology management of wireless networks

Resource Management From Single-domain 5G to End-to-End 6G Network Slicing:A Survey

Network Slicing (NS) is one of the pillars of the fifth/sixth generation (5G/6G) of mobile networks. It provides the means for Mobile Network Operators (MNOs) to leverage physical infrastructure across different technological domains to support different applications. This survey analyzes the progress made on NS resource management across these domains, with a focus on the interdependence between domains and unique issues that arise in cross-domain and End-to-End (E2E) settings. Based on a generic problem formulation, NS resource management functionalities (e.g., resource allocation and orchestration) are examined across domains, revealing their limits when applied separately per domain. The appropriateness of different problem-solving methodologies is critically analyzed, and practical insights are provided, explaining how resource management should be rethought in cross-domain and E2E contexts. Furthermore, the latest advancements are reported through a detailed analysis of the most relevant research projects and experimental testbeds. Finally, the core issues facing NS resource management are dissected, and the most pertinent research directions are identified, providing practical guidelines for new researchers.<br/

Optimal Sensing and Transmission in Energy Harvesting Sensor Networks

Sensor networks equipped with energy harvesting (EH) devices have attracted great attentions recently. Compared with conventional sensor networks powered by batteries, the energy harvesting abilities of the sensor nodes make sustainable and environment-friendly sensor networks possible. However, the random, scarce and non-uniform energy supply features also necessitate a completely different approach to energy management. A typical EH wireless sensor node consists of an EH module that converts ambient energy to electrical energy, which is stored in a rechargeable battery, and will be used to power the sensing and transmission operations of the sensor. Therefore, both sensing and transmission are subject to the stochastic energy constraint imposed by the EH process. In this dissertation, we investigate optimal sensing and transmission policies for EH sensor networks under such constraints. For EH sensing, our objective is to understand how the temporal and spatial variabilities of the EH processes would affect the sensing performance of the network, and how sensor nodes should coordinate their data collection procedures with each other to cope with the random and non-uniform energy supply and provide reliable sensing performance with analytically provable guarantees. Specifically, we investigate optimal sensing policies for a single sensor node with infinite and finite battery sizes in Chapter 2, status updating/transmission strategy of an EH Source in Chapter 3, and a collaborative sensing policy for a multi-node EH sensor network in Chapter 4. For EH communication, our objective is to evaluate the impacts of stochastic variability of the EH process and practical battery usage constraint on the EH systems, and develop optimal transmission policies by taking such impacts into consideration. Specifically, we consider throughput optimization in an EH system under battery usage constraint in Chapter 5

Resource Management in E-health Systems

E-health systems are the information and communication systems deployed to improve quality and efficiency of public health services. Within E-health systems, wearable sensors are deployed to monitor physiology information not only in hospitals, but also in our daily lives under all types of activities; wireless body area networks (WBANs) are adopted to transmit physiology information to smartphones; and cloud servers are utilized for timely diagnose and disease treatment. The integrated services provided by E-health systems could be more convenient, reliable, patient centric and bring more economic healthcare services. Despite of many benefits, e-health systems face challenges among which resource management is the most important one as wearable sensors are energy and computing capability limited, and medical information has stringent quality of service (QoS) requirements in terms of delay and reliability. This thesis presents resource management mechanisms, including transmission power allocation schemes for wearable sensors, Medium Access Control (MAC) for WBANs, and resource sharing schemes among cloud networks, that can efficiently exploit the limited resources to achieve satisfactory QoS. First, we address how wearable sensors could energy efficiently transmit medical information with stringent QoS requirements to a smart phone. We first investigate how to provide worst-case delay provisioning for vital physiology information. Sleep scheduling and opportunistic channel access are exploited to reduce energy consumption in idle listening and increase energy efficiency. Considering dynamic programming suffers from curse of dimensionality, Lyapunov optimization formulation is established to derive a low complexity two-step transmission power allocation algorithm. We analyze the conditions under which the proposed algorithm could guarantee worst-case delay. We then investigate the impacts of peak power constraint and statistical QoS provisioning. An optimal transmission power allocation scheme under a peak power constraint is derived, and followed by an efficient calculation method. Applying duality gap analysis, we characterize the upper bound of the extra average transmission power incurred due a peak power constraint. We demonstrate that when the peak power constraint is stringent, the proposed constant power scheme is suitable for wearable sensors for its performance is close to optimal. Further, we show that the peak power constraint is the bottleneck for wearable sensors to provide stringent statistical QoS provisioning. Second, WBANs can provide low-cost and timely healthcare services and are expected to be widely adopted in hospitals. We develop a centralized MAC layer resource management scheme for WBANs, with a focus on inter-WBAN interference mitigation and sensor power consumption reduction. Based on the channel state and buffer state information reported by smart phones deployed in each WBAN, channel access allocation is performed by a central controller to maximize the network throughput. Note that sensors have insufficient energy and computing capability to timely provide all the necessary information for channel resource management, which deteriorates the network performance. We exploit the temporal correlation of body area channel such that channel state reports from sensors are minimized. We then formulate the MAC design problem as a partially observable optimization problem and develop a myopic policy accordingly. Third, cloud computing is expected to meet the rising computing demands. Both private clouds, which aim at patients in their regions, and public clouds, which serve general public, are adopted. Reliability control and QoS provisioning are the core issues of private clouds and public clouds, respectively. A framework, which exploits the abundant resource of private clouds in time domain, to enable cooperation among private clouds and public clouds, is proposed. Considering the cost of service failure in e-health system, the first time failure probability is adopted as reliability measures for private clouds. An algorithm is proposed to minimize the failure probability, and is proven to be optimal. Then, we propose an e-health monitoring system with minimum service delay and privacy preservation by exploiting geo-distributed clouds. In the system, the resource management scheme enables the distributed cloud servers to cooperatively assign the servers to the requested users under a load balance condition. Thus, the service delay for users is minimized. In addition, a traffic shaping algorithm is proposed, which converts the user health data traffic to the non-health data traffic such that the capability of traffic analysis attacks is largely reduced. In summary, we believe the research results developed in this dissertation can provide insights for efficient transmission power allocation for wearable sensor, can offer practical MAC layer solutions for WBANs in hospital environment, and can improve the QoS provisioning provided by cloud networks in e-health systems

Security and privacy for the internet of medical things enabled healthcare systems: a survey

With the increasing demands on quality healthcare and the raising cost of care, pervasive healthcare is considered as a technological solutions to address the global health issues. In particular, the recent advances in Internet of Things have led to the development of Internet of Medical Things (IoMT). Although such low cost and pervasive sensing devices could potentially transform the current reactive care to preventative care, the security and privacy issues of such sensing system are often overlooked. As the medical devices capture and process very sensitive personal health data, the devices and their associated communications have to be very secured to protect the user's privacy. However, the miniaturized IoMT devices have very limited computation power and fairly limited security schemes can be implemented in such devices. In addition, with the widespread use of IoMT devices, managing and ensuring the security of IoMT systems are very challenging and which are the major issues hindering the adoption of IoMT for clinical applications. In this paper, the security and privacy challenges, requirements, threats, and future research directions in the domain of IoMT are reviewed providing a general overview of the state-of-the-art approaches