Optimal Local and Remote Controllers with Unreliable Communication

We consider a decentralized optimal control problem for a linear plant controlled by two controllers, a local controller and a remote controller. The local controller directly observes the state of the plant and can inform the remote controller of the plant state through a packet-drop channel. We assume that the remote controller is able to send acknowledgments to the local controller to signal the successful receipt of transmitted packets. The objective of the two controllers is to cooperatively minimize a quadratic performance cost. We provide a dynamic program for this decentralized control problem using the common information approach. Although our problem is not a partially nested LQG problem, we obtain explicit optimal strategies for the two controllers. In the optimal strategies, both controllers compute a common estimate of the plant state based on the common information. The remote controller's action is linear in the common estimated state, and the local controller's action is linear in both the actual state and the common estimated state

An Optimal Transmission Strategy for Kalman Filtering over Packet Dropping Links with Imperfect Acknowledgements

This paper presents a novel design methodology for optimal transmission policies at a smart sensor to remotely estimate the state of a stable linear stochastic dynamical system. The sensor makes measurements of the process and forms estimates of the state using a local Kalman filter. The sensor transmits quantized information over a packet dropping link to the remote receiver. The receiver sends packet receipt acknowledgments back to the sensor via an erroneous feedback communication channel which is itself packet dropping. The key novelty of this formulation is that the smart sensor decides, at each discrete time instant, whether to transmit a quantized version of either its local state estimate or its local innovation. The objective is to design optimal transmission policies in order to minimize a long term average cost function as a convex combination of the receiver's expected estimation error covariance and the energy needed to transmit the packets. The optimal transmission policy is obtained by the use of dynamic programming techniques. Using the concept of submodularity, the optimality of a threshold policy in the case of scalar systems with perfect packet receipt acknowledgments is proved. Suboptimal solutions and their structural results are also discussed. Numerical results are presented illustrating the performance of the optimal and suboptimal transmission policies.Comment: Conditionally accepted in IEEE Transactions on Control of Network System

Optimal estimation and control for lossy network: stability, convergence, and performance

In this paper, we study the problems of optimal estimation and control, i.e., the linear quadratic Gaussian (LQG) control, for systems with packet losses but without acknowledgment. Such acknowledgment is a signal sent by the actuator to inform the estimator of the incidence of control packet losses. For such system, which is usually called as a user datagram protocol (UDP)-like system, the optimal estimation is nonlinear and its calculation is time-consuming, making its corresponding optimal LQG problem complicated. We first propose two conditions: 1) the sensor has some computation abilities; and 2) the control command, exerted to the plant, is known to the sensor. For a UDP-like system satisfying these two conditions, we derive the optimal estimation. By constructing the finite and infinite product probability measure spaces for the estimation error covariances (EEC), we give the stability condition for the expected EEC, and show the existence of a measurable function to which the EEC converges in distribution, and propose some practical methods to evaluate the estimation performance. Finally, the LQG controllers are derived, and the conditions for the mean square stability of the closed-loop system are established

Optimal Two Player LQR State Feedback With Varying Delay

This paper presents an explicit solution to a two player distributed LQR problem in which communication between controllers occurs across a communication link with varying delay. We extend known dynamic programming methods to accommodate this varying delay, and show that under suitable assumptions, the optimal control actions are linear in their information, and that the resulting controller has piecewise linear dynamics dictated by the current effective delay regime.Comment: Extended version of IFAC '14 submissio

Control over adversarial packet-dropping communication networks revisited

We revisit a one-step control problem over an adversarial packet-dropping link. The link is modeled as a set of binary channels controlled by a strategic jammer whose intention is to wage a `denial of service' attack on the plant by choosing a most damaging channel-switching strategy. The paper introduces a class of zero-sum games between the jammer and controller as a scenario for such attack, and derives necessary and sufficient conditions for these games to have a nontrivial saddle-point equilibrium. At this equilibrium, the jammer's optimal policy is to randomize in a region of the plant's state space, thus requiring the controller to undertake a nontrivial response which is different from what one would expect in a standard stochastic control problem over a packet dropping channel.Comment: This paper has been accepted for presentation at the 2014 American Control Conference, Portland, Orego

Stochastic Estimation and Control of Queues within a Computer Network

An extended Kalman filter is used to estimate size and packet arrival rate of network queues. These estimates are used by a LQG steady state linear perturbation PI controller to regulate queue size within a computer network. This paper presents the derivation of the transient queue behavior for a system with Poisson traffic and exponential service times. This result is then validated for ideal traffic using a network simulated in OPNET. A more complex OPNET model is then used to test the adequacy of the transient queue size model when non-Poisson traffic is combined. The extended Kalman filter theory is presented and a network state estimator is designed using the transient queue behavior model. The equations needed for the LQG synthesis of a steady state linear perturbation PI controller are presented. These equations are used to develop a network queue controller based on the transient queue model. The performance of the network state estimator and network queue controller was investigated and shown to provide improved control when compared to other simplistic control algorithms