An Index Policy for Minimizing the Uncertainty-of-Information of Markov Sources

This paper focuses on the information freshness of finite-state Markov sources, using the uncertainty of information (UoI) as the performance metric. Measured by Shannon's entropy, UoI can capture not only the transition dynamics of the Markov source but also the different evolutions of information quality caused by the different values of the last observation. We consider an information update system with M finite-state Markov sources transmitting information to a remote monitor via m communication channels. Our goal is to explore the optimal scheduling policy to minimize the sum-UoI of the Markov sources. The problem is formulated as a restless multi-armed bandit (RMAB). We relax the RMAB and then decouple the relaxed problem into M single bandit problems. Analyzing the single bandit problem provides useful properties with which the relaxed problem reduces to maximizing a concave and piecewise linear function, allowing us to develop a gradient method to solve the relaxed problem and obtain its optimal policy. By rounding up the optimal policy for the relaxed problem, we obtain an index policy for the original RMAB problem. Notably, the proposed index policy is universal in the sense that it applies to general RMABs with bounded cost functions.Comment: 55 page

Learning-based Scheduling for Information Accuracy and Freshness in Wireless Networks

We consider a system of multiple sources, a single communication channel, and a single monitoring station. Each source measures a time-varying quantity with varying levels of accuracy and one of them sends its update to the monitoring station via the channel. The probability of success of each attempted communication is a function of the source scheduled for transmitting its update. Both the probability of correct measurement and the probability of successful transmission of all the sources are unknown to the scheduler. The metric of interest is the reward received by the system which depends on the accuracy of the last update received by the destination and the Age-of-Information (AoI) of the system. We model our scheduling problem as a variant of the multi-arm bandit problem with sources as different arms. We compare the performance of all $4$ standard bandit policies, namely, ETC, $\epsilon$-greedy, UCB, and TS suitably adjusted to our system model via simulations. In addition, we provide analytical guarantees of $2$ of these policies, ETC, and $\epsilon$-greedy. Finally, we characterize the lower bound on the cumulative regret achievable by any policy.Comment: 21 pages, 5 figure

Uncertainty-of-Information Scheduling: A Restless Multi-armed Bandit Framework

This paper proposes using the uncertainty of information (UoI), measured by Shannon's entropy, as a metric for information freshness. We consider a system in which a central monitor observes multiple binary Markov processes through a communication channel. The UoI of a Markov process corresponds to the monitor's uncertainty about its state. At each time step, only one Markov process can be selected to update its state to the monitor; hence there is a tradeoff among the UoIs of the processes that depend on the scheduling policy used to select the process to be updated. The age of information (AoI) of a process corresponds to the time since its last update. In general, the associated UoI can be a non-increasing function, or even an oscillating function, of its AoI, making the scheduling problem particularly challenging. This paper investigates scheduling policies that aim to minimize the average sum-UoI of the processes over the infinite time horizon. We formulate the problem as a restless multi-armed bandit (RMAB) problem, and develop a Whittle index policy that is near-optimal for the RMAB after proving its indexability. We further provide an iterative algorithm to compute the Whittle index for the practical deployment of the policy. Although this paper focuses on UoI scheduling, our results apply to a general class of RMABs for which the UoI scheduling problem is a special case. Specifically, this paper's Whittle index policy is valid for any RMAB in which the bandits are binary Markov processes and the penalty is a concave function of the belief state of the Markov process. Numerical results demonstrate the excellent performance of the Whittle index policy for this class of RMABs.Comment: 28 pages, 5 figure

AoI-minimal Power Adjustment in RF-EH-powered Industrial IoT Networks : A Soft Actor-Critic-Based Method

This paper investigates the radio-frequency-energy-harvesting-powered (RF-EH-powered) wireless Industrial Internet of Things (IIoT) networks, where multiple sensor nodes (SNs) are first powered by a wireless power station (WPS), and then collect status updates from the industrial environment and finally transmit the collected data to the monitor with their harvested energy. To enhance the timeliness of data, age of information (AoI) is used as a metric to optimize the system. Particularly, an expected sum AoI (ESA) minimization problem is formulated by optimizing the power adjustment policy for the SNs under multiple practical constraints, including the EH, the minimal signal-to-noise-plus-interference ratio (SINR) and the battery capacity constraints. To solve the non-convex problem with no explicit AoI expression, we transform it into a Markov decision problem (MDP) with continuous state space and action space. Then, inspired by the Soft Actor-Critic (SAC) framework in deep reinforcement learning, a SAC-based age-aware power adjustment (SAPA) method is proposed by modeling the power adjustment as a stochastic strategy. Furthermore, to reduce the communication overhead of SAPA, a multi-agent version of SAPA, i.e., MSAPA, is proposed, with which each SN is able to adjust its transmit power based on its local observations. The communication overhead of SAPA and MSAPA is also analyzed theoretically. Simulation results show that the proposed SAPA and MSAPA converge well with different numbers of SNs. It is also shown that the ESA achieved by the proposed SAPA and MSAPA is lower than that achieved by the baseline methods

A game theoretic approach to wireless body area networks interference control

In this paper we consider a scenario where there are two wireless body area networks (WBANs) interfere with each other from a game theoretic perspective. In particular, we envision two WBANs playing a potential game to enhance their performance by decreasing interference to each other. Decreasing interference extends the sensors' batteries life time and reduces the number of re‐transmissions. We derive the required conditions for the game to be a potential game and its associated the Nash equilibrium (NE). Specifically, we formulate a game where each WBAN has three strategies. Depending on the payoff of each strategy, the game can be designed to achieve a desired NE. Furthermore, we employ a learning algorithm to achieve that NE. In particular, we employ the Fictitious play (FP) learning algorithm as a distributed algorithm that WBANs can use to approach the NE. The simulation results show that the NE is mainly a function of the power cost parameter and a reliability factor that we set depending on each WBAN setting (patient). However, the power cost factor is more dominant than the reliability factor according to the linear cost function formulation that we use throughout this work

Leveraging 6G Technologies to Optimize Information Freshness for Time-Sensitive Applications

Next-generation wireless networks (Beyond 5G, 6G) aim to provide tremendous improvements over previous generations by promising a massive connectivity, ultra-reliable and low-latency communications, and soaring broadband speeds. Such transformation will give rise to a wide range of propitious Internet-of-Things (IoT) applications such as intelligent transportation systems (ITS), tactile internet, augmented/virtual reality, industry 4.0, etc. These applications possess stringent requirements of fresh and timely information updates to make critical decisions. Out-dated or stale information updates are highly undesirable for these applications as they may call forth unreliable or erroneous decisions. The conventional performance metrics such as delay and latency may not fully characterize the freshness of information for time-critical IoT applications. Recently, information freshness has been investigated through defining a new performance metric termed as Age of Information (AoI). AoI offers a rigorous way to quantify the information freshness as compared to other performance metrics and is deemed suitable for real-time IoT applications. In reality, the limited energy and computing resources of IoT devices (IoTDs) is a significant challenge towards realizing the timely delivery of information updates. To address this challenge, the first aim of this dissertation is to examine the capability of multi-access edge computing (MEC) towards minimizing the AoI. In fact, MEC offers an expedited computation of resource-intensive tasks, which, if processed locally at the IoTDs, may experience excessive computational latency. In this context, an optimization problem is setup to determine the optimal scheduling policy with the goal of minimizing the expected sum AoI of multiple IoTDs, while considering the combined impact of unreliable channel conditions and random packet arrivals. Another acute challenge is the high randomness and uncontrollable behaviour of wireless communication environments, which may severely impede the timely and reliable delivery of information updates. Towards addressing this challenge, reconfigurable intelligent surface (RIS) is leveraged to mitigate the propagation-induced impairments of the wireless environment and enhance the quality of wireless links to preserve the information freshness. First, a wireless network consisting of a base station (BS) that is forwarding information updates of multiple real-time traffic streams to their destinations is studied. The considered multiple access technique is frequency division multiple access (FDMA), which is an orthogonal multiple access (OMA) technique. A joint user scheduling and phase-shift matrix (passive beamforming) optimization problem is formulated with the objective of minimizing the expected sum AoI of the coexisting multiple traffic streams. The resulting problem is a mixed integer non-convex optimization problem. To evade the high coupling of the invoked optimization variables, the bi-level optimization technique is utilized, where the original problem is decomposed into an outer traffic stream scheduling problem and an inner RIS phase-shift matrix problem. Owing to the stochastic nature of packet arrivals, a deep reinforcement learning (DRL) solution is employed to solve the outer problem. To do so, the traffic stream scheduling is modeled as a Markov Decision Process (MDP) and Proximal Policy Optimization (PPO) is invoked to solve it. On the other hand, the inner problem that determines the RIS configuration is solved through semi-definite relaxation (SDR). Due to the limitations of OMA techniques in terms of the number of served IoTDs and the spectral efficiency, the focus of this dissertation shifts to explore non-orthogonal multiple access (NOMA) scheme towards achieving the goal of minimizing the AoI in an uplink setting. In this context, an optimization problem is formulated to optimize the RIS configuration, the transmit power of IoTDs and their clustering policy. To solve this mixed-integer non-convex problem, the RIS configuration is obtained first by resorting to difference-of-convex (DC) along with successive convex approximation (SCA). On the other hand, the bi-level optimization is used to solve the power allocation and the clustering problems. Optimal closed-form expressions are derived for the power control scheme and the one-to-one matching is employed to solve the clustering problem. Aiming to further improve the information freshness in time-critical IoT applications, an extended version of NOMA, termed as Cooperative-NOMA (C-NOMA), is adopted. In C-NOMA, the cooperation between IoTDs through device-to-device (D2D) communication and full-duplex (FD) relaying is invoked within the NOMA scheme. In this context, the integration of RIS and C-NOMA is investigated towards achieving the goal of minimizing the average sum AoI. Precisely, it is investigated how much performance gain in terms of AoI reduction can be brought by the RIS-enabled uplink C-NOMA system compared to the conventional C-NOMA and NOMA schemes, both with and without RIS. Results elucidate the superiority of our proposed approaches against other baseline schemes. The findings in this dissertation shed light on the choice of effective design of wireless communication networks leveraging the core future enabling technologies