Towards AoI-aware Smart IoT Systems

Age of Information (AoI) has gained importance as a Key Performance Indicator (KPI) for characterizing the freshness of information in information-update systems and time-critical applications. Recent theoretical research on the topic has generated significant understanding of how various algorithms perform in terms of this metric on various system models and networking scenarios. In this paper, by the help of the theoretical results, we analyzed the AoI behavior on real-life networks, using our two test-beds, addressing IoT networks and regular computers. Excessive number of AoI measurements are provided for variations of transport protocols such as TCP, UDP and web-socket, on wired and wireless links. Practical issues such as synchronization and selection of hardware along with transport protocol, and their effects on AoI are discussed. The results provide insight toward application and transport layer mechanisms for optimizing AoI in real-life networks

Timely Monitoring of Dynamic Sources with Observations from Multiple Wireless Sensors

Age of Information (AoI) has recently received much attention due to its relevance in IoT sensing and monitoring applications. In this paper, we consider the problem of minimizing the AoI in a system in which a set of sources are observed by multiple sensors in a many-to-many relationship, and the probability that a sensor observes a source depends on the state of the source. This model represents many practical scenarios, such as the ones in which multiple cameras or microphones are deployed to monitor objects moving in certain areas. We formulate the scheduling problem as a Markov Decision Process, and show how the age-optimal scheduling policy can be obtained. We further consider partially observable variants of the problem, and devise approximate policies for large state spaces. Our evaluations show that the approximate policies work well in the considered scenarios, and that the fact that sensors can observe multiple sources is beneficial, especially when there is high uncertainty of the source states.Comment: Submitted for publicatio

A Reinforcement Learning Framework for Optimizing Age-of-Information in RF-powered Communication Systems

In this paper, we study a real-time monitoring system in which multiple source nodes are responsible for sending update packets to a common destination node in order to maintain the freshness of information at the destination. Since it may not always be feasible to replace or recharge batteries in all source nodes, we consider that the nodes are powered through wireless energy transfer (WET) by the destination. For this system setup, we investigate the optimal online sampling policy (referred to as the age-optimal policy) that jointly optimizes WET and scheduling of update packet transmissions with the objective of minimizing the long-term average weighted sum of Age-of-Information (AoI) values for different physical processes (observed by the source nodes) at the destination node, referred to as the sum-AoI. To solve this optimization problem, we first model this setup as an average cost Markov decision process (MDP). Due to the extreme curse of dimensionality in the state space of the formulated MDP, classical reinforcement learning algorithms are no longer applicable to our problem. Motivated by this, we propose a deep reinforcement learning (DRL) algorithm that can learn the age-optimal policy in a computationally-efficient manner. We further characterize the structural properties of the age-optimal policy analytically, and demonstrate that it has a threshold-based structure with respect to the AoI values for different processes. We extend our analysis to characterize the structural properties of the policy that maximizes average throughput for our system setup, referred to as the throughput-optimal policy. Afterwards, we analytically demonstrate that the structures of the age-optimal and throughput-optimal policies are different. We also numerically demonstrate these structures as well as the impact of system design parameters on the optimal achievable average weighted sum-AoI

Age Minimization of Multiple Flows using Reinforcement Learning

Age of Information (AoI) is a recently proposed performance metric measuring the freshness of data at the receiving side of a flow. This metric is particularly suited to status-update type information flows, like those occurring in machine-type communication (MTC), remote monitoring and similar applications. In this paper, we consider the problem of AoI-optimal scheduling of multiple flows served by a single server. The performance of scheduling algorithms proposed in previous literature has been shown under limited assumptions, due to the analytical intractability of the problem. The goal of this paper is to apply reinforcement learning methods to achieve scheduling decisions that are resilient to network conditions and packet arrival processes. Specifically, Policy Gradients and Deep Q-Learning methods are employed. These can adapt to the network without a priori knowledge of its parameters. We study the resulting performance relative to a benchmark, the MAF algorithm, which is known to be optimal under certain conditions