Minimum-Information LQG Control - Part I: Memoryless Controllers

With the increased demand for power efficiency in feedback-control systems, communication is becoming a limiting factor, raising the need to trade off the external cost that they incur with the capacity of the controller's communication channels. With a proper design of the channels, this translates into a sequential rate-distortion problem, where we minimize the rate of information required for the controller's operation under a constraint on its external cost. Memoryless controllers are of particular interest both for the simplicity and frugality of their implementation and as a basis for studying more complex controllers. In this paper we present the optimality principle for memoryless linear controllers that utilize minimal information rates to achieve a guaranteed external-cost level. We also study the interesting and useful phenomenology of the optimal controller, such as the principled reduction of its order

Memory Resilient Gain-scheduled State-Feedback Control of Uncertain LTI/LPV Systems with Time-Varying Delays

The stabilization of uncertain LTI/LPV time delay systems with time varying delays by state-feedback controllers is addressed. At the difference of other works in the literature, the proposed approach allows for the synthesis of resilient controllers with respect to uncertainties on the implemented delay. It is emphasized that such controllers unify memoryless and exact-memory controllers usually considered in the literature. The solutions to the stability and stabilization problems are expressed in terms of LMIs which allow to check the stability of the closed-loop system for a given bound on the knowledge error and even optimize the uncertainty radius under some performance constraints; in this paper, the $\mathcal{H}_\infty$ performance measure is considered. The interest of the approach is finally illustrated through several examples

Longitudinal Dynamic versus Kinematic Models for Car-Following Control Using Deep Reinforcement Learning

The majority of current studies on autonomous vehicle control via deep reinforcement learning (DRL) utilize point-mass kinematic models, neglecting vehicle dynamics which includes acceleration delay and acceleration command dynamics. The acceleration delay, which results from sensing and actuation delays, results in delayed execution of the control inputs. The acceleration command dynamics dictates that the actual vehicle acceleration does not rise up to the desired command acceleration instantaneously due to dynamics. In this work, we investigate the feasibility of applying DRL controllers trained using vehicle kinematic models to more realistic driving control with vehicle dynamics. We consider a particular longitudinal car-following control, i.e., Adaptive Cruise Control (ACC), problem solved via DRL using a point-mass kinematic model. When such a controller is applied to car following with vehicle dynamics, we observe significantly degraded car-following performance. Therefore, we redesign the DRL framework to accommodate the acceleration delay and acceleration command dynamics by adding the delayed control inputs and the actual vehicle acceleration to the reinforcement learning environment state, respectively. The training results show that the redesigned DRL controller results in near-optimal control performance of car following with vehicle dynamics considered when compared with dynamic programming solutions.Comment: Accepted to 2019 IEEE Intelligent Transportation Systems Conferenc

Upper Bounds on the Capacities of Noncontrollable Finite-State Channels with/without Feedback

Noncontrollable finite-state channels (FSCs) are FSCs in which the channel inputs have no influence on the channel states, i.e., the channel states evolve freely. Since single-letter formulae for the channel capacities are rarely available for general noncontrollable FSCs, computable bounds are usually utilized to numerically bound the capacities. In this paper, we take the delayed channel state as part of the channel input and then define the {\em directed information rate} from the new channel input (including the source and the delayed channel state) sequence to the channel output sequence. With this technique, we derive a series of upper bounds on the capacities of noncontrollable FSCs with/without feedback. These upper bounds can be achieved by conditional Markov sources and computed by solving an average reward per stage stochastic control problem (ARSCP) with a compact state space and a compact action space. By showing that the ARSCP has a uniformly continuous reward function, we transform the original ARSCP into a finite-state and finite-action ARSCP that can be solved by a value iteration method. Under a mild assumption, the value iteration algorithm is convergent and delivers a near-optimal stationary policy and a numerical upper bound.Comment: 15 pages, Two columns, 6 figures; appears in IEEE Transaction on Information Theor

Energy Optimal Transmission Scheduling in Wireless Sensor Networks

One of the main issues in the design of sensor networks is energy efficient communication of time-critical data. Energy wastage can be caused by failed packet transmission attempts at each node due to channel dynamics and interference. Therefore transmission control techniques that are unaware of the channel dynamics can lead to suboptimal channel use patterns. In this paper we propose a transmission controller that utilizes different "grades" of channel side information to schedule packet transmissions in an optimal way, while meeting a deadline constraint for all packets waiting in the transmission queue. The wireless channel is modeled as a finite-state Markov channel. We are specifically interested in the case where the transmitter has low-grade channel side information that can be obtained based solely on the ACK/NAK sequence for the previous transmissions. Our scheduler is readily implementable and it is based on the dynamic programming solution to the finite-horizon transmission control problem. We also calculate the information theoretic capacity of the finite state Markov channel with feedback containing different grades of channel side information including that, obtained through the ACK/NAK sequence. We illustrate that our scheduler achieves a given throughput at a power level that is fairly close to the fundamental limit achievable over the channel.Comment: Accepted for publication in the IEEE Transactions on Wireless Communication