45 research outputs found
Average age of information with hybrid ARQ under a resource constraint
Scheduling the transmission of status updates over an error-prone communication channel is studied in order to minimize the long-term average age of information (AoI) at the destination under a constraint on the average number of transmissions at the source node. After each transmission, the source receives an instantaneous ACK/NACK feedback, and decides on the next update without prior knowledge on the success of future transmissions. First, the optimal scheduling policy is studied under different feedback mechanisms when the channel statistics are known; in particular, the standard automatic repeat request (ARQ) and hybrid ARQ (HARQ) protocols are considered. Then, for an unknown environment, an average-cost reinforcement learning (RL) algorithm is proposed that learns the system parameters and the transmission policy in real time. The effectiveness of the proposed methods are verified through numerical simulations
Hierarchical Deep Reinforcement Learning for Age-of-Information Minimization in IRS-aided and Wireless-powered Wireless Networks
In this paper, we focus on a wireless-powered sensor network coordinated by a
multi-antenna access point (AP). Each node can generate sensing information and
report the latest information to the AP using the energy harvested from the
AP's signal beamforming. We aim to minimize the average age-of-information
(AoI) by adapting the nodes' transmission scheduling and the transmission
control strategies jointly. To reduce the transmission delay, an intelligent
reflecting surface (IRS) is used to enhance the channel conditions by
controlling the AP's beamforming vector and the IRS's phase shifting matrix.
Considering dynamic data arrivals at different sensing nodes, we propose a
hierarchical deep reinforcement learning (DRL) framework to for AoI
minimization in two steps. The users' transmission scheduling is firstly
determined by the outer-loop DRL approach, e.g. the DQN or PPO algorithm, and
then the inner-loop optimization is used to adapt either the uplink information
transmission or downlink energy transfer to all nodes. A simple and efficient
approximation is also proposed to reduce the inner-loop rum time overhead.
Numerical results verify that the hierarchical learning framework outperforms
typical baselines in terms of the average AoI and proportional fairness among
different nodes.Comment: 31 pages, 6 figures, 2 tables, 3 algorithm
AoI Minimization in Status Update Control with Energy Harvesting Sensors
Information freshness is crucial for time-critical IoT applications, e.g.,
monitoring and control systems. We consider an IoT status update system with
multiple users, multiple energy harvesting sensors, and a wireless edge node.
The users receive time-sensitive information about physical quantities, each
measured by a sensor. Users send requests to the edge node where a cache
contains the most recently received measurements from each sensor. To serve a
request, the edge node either commands the sensor to send a status update or
retrieves the aged measurement from the cache. We aim at finding the best
actions of the edge node to minimize the age of information of the served
measurements. We model this problem as a Markov decision process and develop
reinforcement learning (RL) algorithms: model-based value iteration and
model-free Q-learning methods. We also propose a Q-learning method for the
realistic case where the edge node is informed about the sensors' battery
levels only via the status updates. The case under transmission limitations is
also addressed. Furthermore, properties of an optimal policy are analytically
characterized. Simulation results show that an optimal policy is a
threshold-based policy and that the proposed RL methods significantly reduce
the average cost compared to several baselines.Comment: 31 pages, 6 figures, submitted journa
AoI-optimal Joint Sampling and Updating for Wireless Powered Communication Systems
This paper characterizes the structure of the Age of Information
(AoI)-optimal policy in wireless powered communication systems while accounting
for the time and energy costs of generating status updates at the source nodes.
In particular, for a single source-destination pair in which a radio frequency
(RF)-powered source sends status updates about some physical process to a
destination node, we minimize the long-term average AoI at the destination
node. The problem is modeled as an average cost Markov Decision Process (MDP)
in which, the generation times of status updates at the source, the
transmissions of status updates from the source to the destination, and the
wireless energy transfer (WET) are jointly optimized. After proving the
monotonicity property of the value function associated with the MDP, we
analytically demonstrate that the AoI-optimal policy has a threshold-based
structure w.r.t. the state variables. Our numerical results verify the
analytical findings and reveal the impact of state variables on the structure
of the AoI-optimal policy. Our results also demonstrate the impact of system
design parameters on the optimal achievable average AoI as well as the
superiority of our proposed joint sampling and updating policy w.r.t. the
generate-at-will policy
Optimal Scheduling Policy for Minimizing Age of Information with a Relay
We consider IoT sensor network where multiple sensors are connected to
corresponding destination nodes via a relay. Thus, the relay schedules sensors
to sample and destination nodes to update. The relay can select multiple
sensors and destination nodes in each time. In order to minimize average
weighted sum AoI, joint optimization of sampling and updating policy of the
relay is investigated. For errorless and symmetric case where weights are
equally given, necessary and sufficient conditions for optimality is found.
Using this result, we obtain that the minimum average sum AoI in a closed-form
expression which can be interpreted as fundamental limit of sum AoI in a single
relay network. Also, for error-prone and symmetric case, we have proved that
greedy policy achieves the minimum average sum AoI at the destination nodes.
For general case, we have proposed scheduling policy obtained via reinforcement
learning.Comment: 30 page