Minimizing the AoI in Resource-Constrained Multi-Source Relaying Systems: Dynamic and Learning-based Scheduling

We consider a multi-source relaying system where the independent sources randomly generate status update packets which are sent to the destination with the aid of a relay through unreliable links. We develop transmission scheduling policies to minimize the sum average age of information (AoI) subject to transmission capacity and long-run average resource constraints. We formulate a stochastic control optimization problem. To solve the problem, a constrained Markov decision process (CMDP) approach and a drift-plus-penalty method are proposed. The CMDP problem is solved by transforming it into an MDP problem using the Lagrangian relaxation method. We theoretically analyze the structure of optimal policies for the MDP problem and subsequently propose a structure-aware algorithm that returns a practical near-optimal policy. By the drift-plus-penalty method, we devise a dynamic near-optimal low-complexity policy. We also develop a model-free deep reinforcement learning policy, which does not require the full knowledge of system statistics. To do so, we employ the Lyapunov optimization theory and a dueling double deep Q-network. Simulation results are provided to assess the performance of our policies and validate the theoretical results. The results show up to 91% performance improvement compared to a baseline policy.Comment: 30 Pages, preliminary results of this paper were presented at IEEE Globecom 2021, https://ieeexplore.ieee.org/document/968594

Statistical Age-of-Information Optimization for Status Update over Multi-State Fading Channels

Age of information (AoI) is a powerful metric to evaluate the freshness of information, where minimization of average statistics, such as the average AoI and average peak AoI, currently prevails in guiding freshness optimization for related applications. Although minimizing the statistics does improve the received information's freshness for status update systems in the sense of average, the time-varying fading characteristics of wireless channels often cause uncertain yet frequent age violations. The recently-proposed statistical AoI metric can better characterize more features of AoI dynamics, which evaluates the achievable minimum peak AoI under the certain constraint on age violation probability. In this paper, we study the statistical AoI minimization problem for status update systems over multi-state fading channels, which can effectively upper-bound the AoI violation probability but introduce the prohibitively-high computing complexity. To resolve this issue, we tackle the problem with a two-fold approach. For a small AoI exponent, the problem is approximated via a fractional programming problem. For a large AoI exponent, the problem is converted to a convex problem. Solving the two problems respectively, we derive the near-optimal sampling interval for diverse status update systems. Insightful observations are obtained on how sampling interval shall be tuned as a decreasing function of channel state information (CSI). Surprisingly, for the extremely stringent AoI requirement, the sampling interval converges to a constant regardless of CSI's variation. Numerical results verify effectiveness as well as superiority of our proposed scheme

Peak Age of Information Distribution for Edge Computing with Wireless Links

Age of Information (AoI) is a critical metric for several Internet of Things (IoT) applications, where sensors keep track of the environment by sending updates that need to be as fresh as possible. The development of edge computing solutions has moved the monitoring process closer to the sensor, reducing the communication delays, but the processing time of the edge node needs to be taken into account. Furthermore, a reliable system design in terms of freshness requires the knowledge of the full distribution of the Peak AoI (PAoI), from which the probability of occurrence of rare, but extremely damaging events can be obtained. In this work, we model the communication and computation delay of such a system as two First Come First Serve (FCFS) queues in tandem, analytically deriving the full distribution of the PAoI for the M/M/1 - M/D/1 and the M/M/1 - M/M/1 tandems, which can represent a wide variety of realistic scenarios.Comment: Preprint version of the paper accepted for publication in the Transactions on Communication

Radio resource management for V2V multihop communication considering adjacent channel interference

This paper investigates schemes for multihop scheduling and power control for vehicle-to-vehicle (V2V) multicast communication, taking into account the effects of both co-channel interference and adjacent channel interference, such that requirements on latency or age of information (AoI) are satisfied. Optimal performance can be achieved by formulating and solving mixed Boolean linear programming (MBLP) optimization problems for various performance metrics, including network throughput and connectivity. Fairness among network nodes (vehicles) is addressed by considering formulations that maximizes the worst-case network node performance. Solving the optimization problem comes at the cost of significant computational complexity for large networks and requires that (slow) channel state information is gathered at a central point. To address these issues, a clustering method is proposed to partition the optimization problem into a set of smaller problems, which reduces the overall computational complexity, and a decentralized algorithm that does not need channel state information is provided

Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs