6 research outputs found
Pathwise Performance of Debt Based Policies for Wireless Networks with Hard Delay Constraints
Hou et al have introduced a framework to serve clients over wireless channels
when there are hard deadline constraints along with a minimum delivery ratio
for each client's flow. Policies based on "debt," called maximum debt first
policies (MDF) were introduced, and shown to be throughput optimal. By
"throughput optimality" it is meant that if there exists a policy that fulfils
a set of clients with a given vector of delivery ratios and a vector of channel
reliabilities, then the MDF policy will also fulfill them. The debt of a user
is the difference between the number of packets that should have been delivered
so as to meet the delivery ratio and the number of packets that have been
delivered for that client. The maximum debt first (MDF) prioritizes the clients
in decreasing order of debts at the beginning of every period. Note that a
throughput optimal policy only guarantees that \begin{small} \liminf_{T \to
\infty} \frac{1}{T}\sum_{t=1}^{T} \mathbbm{1}\{\{client nt} \} \geq q_{i} \end{small}, where the right hand side
is the required delivery ratio for client . Thus, it only guarantees that
the debts of each user are , and can be otherwise arbitrarily large. This
raises the interesting question about what is the growth rate of the debts
under the MDF policy. We show the optimality of MDF policy in the case when the
channel reliabilities of all users are same, and obtain performance bounds for
the general case. For the performance bound we obtain the almost sure bounds on
for all , where
Index Policies for Optimal Mean-Variance Trade-Off of Inter-delivery Times in Real-Time Sensor Networks
A problem of much current practical interest is the replacement of the wiring
infrastructure connecting approximately 200 sensor and actuator nodes in
automobiles by an access point. This is motivated by the considerable savings
in automobile weight, simplification of manufacturability, and future
upgradability.
A key issue is how to schedule the nodes on the shared access point so as to
provide regular packet delivery. In this and other similar applications, the
mean of the inter-delivery times of packets, i.e., throughput, is not
sufficient to guarantee service-regularity. The time-averaged variance of the
inter-delivery times of packets is also an important metric.
So motivated, we consider a wireless network where an Access Point schedules
real-time generated packets to nodes over a fading wireless channel. We are
interested in designing simple policies which achieve optimal mean-variance
tradeoff in interdelivery times of packets by minimizing the sum of
time-averaged means and variances over all clients. Our goal is to explore the
full range of the Pareto frontier of all weighted linear combinations of mean
and variance so that one can fully exploit the design possibilities. We
transform this problem into a Markov decision process and show that the problem
of choosing which node's packet to transmit in each slot can be formulated as a
bandit problem. We establish that this problem is indexable and explicitly
derive the Whittle indices. The resulting Index policy is optimal in certain
cases. We also provide upper and lower bounds on the cost for any policy.
Extensive simulations show that Index policies perform better than previously
proposed policies
A High Reliability Asymptotic Approach for Packet Inter-Delivery Time Optimization in Cyber-Physical Systems
In cyber-physical systems such as automobiles, measurement data from sensor
nodes should be delivered to other consumer nodes such as actuators in a
regular fashion. But, in practical systems over unreliable media such as
wireless, it is a significant challenge to guarantee small enough
inter-delivery times for different clients with heterogeneous channel
conditions and inter-delivery requirements. In this paper, we design scheduling
policies aiming at satisfying the inter-delivery requirements of such clients.
We formulate the problem as a risk-sensitive Markov Decision Process (MDP).
Although the resulting problem involves an infinite state space, we first prove
that there is an equivalent MDP involving only a finite number of states. Then
we prove the existence of a stationary optimal policy and establish an
algorithm to compute it in a finite number of steps.
However, the bane of this and many similar problems is the resulting
complexity, and, in an attempt to make fundamental progress, we further propose
a new high reliability asymptotic approach. In essence, this approach considers
the scenario when the channel failure probabilities for different clients are
of the same order, and asymptotically approach zero. We thus proceed to
determine the asymptotically optimal policy: in a two-client scenario, we show
that the asymptotically optimal policy is a "modified least time-to-go" policy,
which is intuitively appealing and easily implementable; in the general
multi-client scenario, we are led to an SN policy, and we develop an algorithm
of low computational complexity to obtain it. Simulation results show that the
resulting policies perform well even in the pre-asymptotic regime with moderate
failure probabilities
Distributed Operation of Uncertain Dynamical Cyberphysical Systems
In this thesis we address challenging issues that are faced in the operation of important cyber-physical systems of great current interest. The two particular systems that we address are communication networks and the smart grid. Both systems feature distributed agents making decisions in dynamic uncertain environments. In communication networks, nodes need to decide which packets to transmit, while in the power grid individual generators and loads need to decide how much to pro-duce or consume in a dynamic uncertain environment. The goal in both systems, which also holds for other cyber-physical systems, is to develop distributed policies that perform efficiently in uncertain dynamically changing environments. This thesis proposes an approach of employing duality theory on dynamic stochastic systems in such a way as to develop such distributed operating policies for cyber-physical systems.
In the first half of the thesis we examine communication networks. Many cyber-physical systems, e.g., sensor networks, mobile ad-hoc networks, or networked control systems, involve transmitting data over multiple-hops of a communication network. These networks can be unreliable, for example due to the unreliability of the wireless medium. However, real-time applications in cyber-physical systems often require that requisite amounts of data be delivered in a timely manner so that it can be utilized for safely controlling physical processes. Data packets may need to be delivered within their deadlines or at regular intervals without large gaps in packet deliveries when carrying sensor readings. How such packets with deadlines can be scheduled over networks is a major challenge for cyber-physical systems.
We develop a framework for routing and scheduling such data packets in a multi-hop network. This framework employs duality theory in such a way that actions of nodes get decoupled, and results in efficient decentralized policies for routing and scheduling such multi-hop communication networks. A key feature of the scheduling policy derived in this work is that the scheduling decisions regarding packets can be made in a fully distributed fashion. A decision regarding the scheduling of an individual packet depend only on the age and location of the packet, and does not require sharing of the queue lengths at various nodes.
We examine in more detail a network in which multiple clients stream video packets over shared wireless networks. We are able to derive simple policies of threshold type which maximize the combined QoE of the users.
We turn to another important cyber-physical system of great current interest – the emerging smarter grid for electrical power. We address some fundamental problems that arise when attempting to increase the utilization of renewable energy sources. A major challenge is that renewable energy sources are unpredictable in their availability. Utilizing them requires adaptation of demand to their uncertain availability. We address the problem faced by the system operator of coordinating sources of power and loads to balance stochastically time varying supply and demand while maximizing the total utilities of all agents in the system. We develop policies for the system operator that is charged with coordinating such distributed entities through a notion of price. We analyze some models for such systems and employ a combination of duality theory and analysis of stochastic dynamic systems to develop policies that maximize the total utility function of all the agents.
We also address the issue of how the size of energy storage facilities should scale with respect to the stochastic behavior of renewables in order to mitigate the unreliability of renewable energy sources