Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Dynamic Recofiguration Techniques for Wireless Sensor Networks

The need to achieve extended service life by battery powered Wireless Sensor Networks (WSNs) requires new concepts and technqiues beyond the state-of-the-art low-power designs based on fixed hardware platforms or energy-efficient protocols. This thesis investigates reconfiguration techniques that enable sensor hardware to adapt its energy consumption to external dynamics, by means of Dynamic Voltage Scaling (DVS), Dynamic Modulation Scaling (DMS), and other related concepts. For sensor node-level reconfiguration, an integration of DVS and DMS techniques was proposed to minimize the total energy consumption. A dynamic time allocation algorithm was developed, demonstrating an average of 55% energy reduction. For network-level reconfiguration, a node activation technique was presented to reduce the cost of recharging energy-depleted sensor nodes. Network operation combined with node activation was modeled as a stochastic decision process, where the activation decisions directly affected the energy efficiency of the network. An experimental test bed based on the Imote2 sensor node platform was realized, which demonstrated energy reduction of up to 50%. Such energy saving can be effectively translated into prolonged service life of the sensor network

Markov decision processes with applications in wireless sensor networks: A survey

Ministry of Education, Singapore under its Academic Research Funding Tier

Learning and Communications Co-Design for Remote Inference Systems: Feature Length Selection and Transmission Scheduling

In this paper, we consider a remote inference system, where a neural network is used to infer a time-varying target (e.g., robot movement), based on features (e.g., video clips) that are progressively received from a sensing node (e.g., a camera). Each feature is a temporal sequence of sensory data. The learning performance of the system is determined by (i) the timeliness and (ii) the temporal sequence length of the features, where we use Age of Information (AoI) as a metric for timeliness. While a longer feature can typically provide better learning performance, it often requires more channel resources for sending the feature. To minimize the time-averaged inference error, we study a learning and communication co-design problem that jointly optimizes feature length selection and transmission scheduling. When there is a single sensor-predictor pair and a single channel, we develop low-complexity optimal co-designs for both the cases of time-invariant and time-variant feature length. When there are multiple sensor-predictor pairs and multiple channels, the co-design problem becomes a restless multi-arm multi-action bandit problem that is PSPACE-hard. For this setting, we design a low-complexity algorithm to solve the problem. Trace-driven evaluations suggest that the proposed co-designs can significantly reduce the time-averaged inference error of remote inference systems.Comment: 41 pages, 8 figures. The manuscript has been submitted to IEEE Journal on Selected Areas in Information Theor

A survey on intelligent computation offloading and pricing strategy in UAV-Enabled MEC network: Challenges and research directions

The lack of resource constraints for edge servers makes it difficult to simultaneously perform a large number of Mobile Devices’ (MDs) requests. The Mobile Network Operator (MNO) must then select how to delegate MD queries to its Mobile Edge Computing (MEC) server in order to maximize the overall benefit of admitted requests with varying latency needs. Unmanned Aerial Vehicles (UAVs) and Artificial Intelligent (AI) can increase MNO performance because of their flexibility in deployment, high mobility of UAV, and efficiency of AI algorithms. There is a trade-off between the cost incurred by the MD and the profit received by the MNO. Intelligent computing offloading to UAV-enabled MEC, on the other hand, is a promising way to bridge the gap between MDs' limited processing resources, as well as the intelligent algorithms that are utilized for computation offloading in the UAV-MEC network and the high computing demands of upcoming applications. This study looks at some of the research on the benefits of computation offloading process in the UAV-MEC network, as well as the intelligent models that are utilized for computation offloading in the UAV-MEC network. In addition, this article examines several intelligent pricing techniques in different structures in the UAV-MEC network. Finally, this work highlights some important open research issues and future research directions of Artificial Intelligent (AI) in computation offloading and applying intelligent pricing strategies in the UAV-MEC network

Resource Management in Green Wireless Communication Networks

The development of wireless technologies has been stimulated by the ever increasing network capacity and the diversity of users' quality of service (QoS) requirements. It is widely anticipated that next-generation wireless networks should be capable of integrating wireless networks with various network architectures and wireless access technologies to provide diverse high-quality ubiquitous wireless accesses for users. However, the existing wireless network architecture may not be able to satisfy explosive wireless access request. Moreover, with the increasing awareness of environmental protection, significant growth of energy consumption caused by the massive traffic demand consequently raises the carbon emission footprint. The emerging of green energy technologies, e.g., solar panel and wind turbine, has provided a promising methodology to sustain operations and management of next-generation wireless networks by powering wireless network devices with eco-friendly green energy. In this thesis, we propose a sustainable wireless network solution as the prototype of next-generation wireless networks to fulfill various QoS requirements of users with harvested energy from natural environments. The sustainable wireless solution aims at establishing multi-tier heterogeneous green wireless communication networks to integrate different wireless services and utilizing green energy supplies to sustain the network operations and management. The solution consists of three steps, 1) establishing conventional green wireless networks, 2) building multi-tier green wireless networks, and 3) allocating and balancing network resources. In the first step, we focus on cost-effectively establishing single-tier green wireless networks to satisfy users' basic QoS requirements by designing efficient network planning algorithm. We formulate the minimum green macro cell BS deployment problem as an optimization problem, which aims at placing the minimum number of BSs to fulfill the basic QoS requirements by harvested energy. A preference level is defined as the guidance for efficient algorithm design to solve the minimum green macro cell BSs deployment problem. After that, we propose a heuristic algorithm, called two-phase constrained green BS placement (TCGBP) algorithm, based on Voronoi diagram. The TCGBP algorithm jointly considers the rate adaptation and power allocation to solve the formulated optimization problem. The performance is verified by extensive simulations, which demonstrate that the TCGBP algorithm can achieve the optimal solution with significantly reduced time complexity. In the second step, we aim at efficiently constructing multi-tier green heterogeneous networks to fulfill high-end QoS requirements of users by placing green small cell BSs. We formulate the green small cell BS deployment and sub-carrier allocation problem as a mixed-integer non-linear programming (MINLP) problem, which targets at deploying the minimum number of green small cell BSs as relay nodes to further improve network capacities and provide high-quality QoS wireless services with harvested energy under the cost constraint. We propose the sub-carrier and traffic over rate (STR) metric to evaluate the contribution of deployed green small cell BSs in both energy and throughput aspects. Based on the metric, two algorithms are designed, namely joint relay node placement and sub-carrier allocation with top-down/bottom-up (RNP-SA-t/b) algorithms. Extensive simulations demonstrate that the proposed algorithms provide simple yet efficient solutions and offer important guidelines on network planning and resource management in two-tier heterogeneous green wireless networks. In the last step, we intend to allocate limited network resources to guarantee the balance of charging and discharging processes. Different from network planning based on statistical historical data, the design of resource allocation algorithm generally concerns relatively short-term resources management, and thus it is essential to accurately estimate the instantaneous energy charging and discharging rates of green wireless network devices. Specifically, we investigate the energy trading issues in green wireless networks, and try to maximize the profits of all cells by determining the optimal price and quantity in each energy trading transaction. Finally, we apply a two-stage leader-follower Stackelberg game to formulate the energy trading problem. By using back induction to obtain the optimal price and quantity of traded energy, we propose an optimal algorithm, called optimal profits energy trading (OPET) algorithm. Our analysis and simulation results demonstrate the optimality performance of OPET algorithm. We believe that our research results in this dissertation can provide insightful guidance in the design of next-generation wireless communication networks with green energy. The algorithms developed in the dissertation offer practical and efficient solutions to build and optimize multi-tier heterogeneous green wireless communication networks