60,796 research outputs found
Goal-oriented Estimation of Multiple Markov Sources in Resource-constrained Systems
This paper investigates goal-oriented communication for remote estimation of
multiple Markov sources in resource-constrained networks. An agent selects the
update order of the sources and transmits the packet to a remote destination
over an unreliable delay channel. The destination is tasked with source
reconstruction for the purpose of actuation. We utilize the metric cost of
actuation error (CAE) to capture the significance (semantics) of error at the
point of actuation. We aim to find an optimal sampling policy that minimizes
the time-averaged CAE subject to average resource constraints. We formulate
this problem as an average-cost constrained Markov Decision Process (CMDP) and
transform it into an unconstrained MDP by utilizing Lyapunov drift techniques.
Then, we propose a low-complexity drift-plus-penalty(DPP) policy for systems
with known source/channel statistics and a Lyapunov optimization-based deep
reinforcement learning (LO-DRL) policy for unknown environments. Our policies
achieve near-optimal performance in CAE minimization and significantly reduce
the number of uninformative transmissions
Dynamic Server Allocation over Time Varying Channels with Switchover Delay
We consider a dynamic server allocation problem over parallel queues with
randomly varying connectivity and server switchover delay between the queues.
At each time slot the server decides either to stay with the current queue or
switch to another queue based on the current connectivity and the queue length
information. Switchover delay occurs in many telecommunications applications
and is a new modeling component of this problem that has not been previously
addressed. We show that the simultaneous presence of randomly varying
connectivity and switchover delay changes the system stability region and the
structure of optimal policies. In the first part of the paper, we consider a
system of two parallel queues, and develop a novel approach to explicitly
characterize the stability region of the system using state-action frequencies
which are stationary solutions to a Markov Decision Process (MDP) formulation.
We then develop a frame-based dynamic control (FBDC) policy, based on the
state-action frequencies, and show that it is throughput-optimal asymptotically
in the frame length. The FBDC policy is applicable to a broad class of network
control systems and provides a new framework for developing throughput-optimal
network control policies using state-action frequencies. Furthermore, we
develop simple Myopic policies that provably achieve more than 90% of the
stability region. In the second part of the paper, we extend our results to
systems with an arbitrary but finite number of queues.Comment: 38 Pages, 18 figures. arXiv admin note: substantial text overlap with
arXiv:1008.234
A High Reliability Asymptotic Approach for Packet Inter-Delivery Time Optimization in Cyber-Physical Systems
In cyber-physical systems such as automobiles, measurement data from sensor
nodes should be delivered to other consumer nodes such as actuators in a
regular fashion. But, in practical systems over unreliable media such as
wireless, it is a significant challenge to guarantee small enough
inter-delivery times for different clients with heterogeneous channel
conditions and inter-delivery requirements. In this paper, we design scheduling
policies aiming at satisfying the inter-delivery requirements of such clients.
We formulate the problem as a risk-sensitive Markov Decision Process (MDP).
Although the resulting problem involves an infinite state space, we first prove
that there is an equivalent MDP involving only a finite number of states. Then
we prove the existence of a stationary optimal policy and establish an
algorithm to compute it in a finite number of steps.
However, the bane of this and many similar problems is the resulting
complexity, and, in an attempt to make fundamental progress, we further propose
a new high reliability asymptotic approach. In essence, this approach considers
the scenario when the channel failure probabilities for different clients are
of the same order, and asymptotically approach zero. We thus proceed to
determine the asymptotically optimal policy: in a two-client scenario, we show
that the asymptotically optimal policy is a "modified least time-to-go" policy,
which is intuitively appealing and easily implementable; in the general
multi-client scenario, we are led to an SN policy, and we develop an algorithm
of low computational complexity to obtain it. Simulation results show that the
resulting policies perform well even in the pre-asymptotic regime with moderate
failure probabilities
- …