Optimal Control for LQG Systems on Graphs---Part I: Structural Results

In this two-part paper, we identify a broad class of decentralized output-feedback LQG systems for which the optimal control strategies have a simple intuitive estimation structure and can be computed efficiently. Roughly, we consider the class of systems for which the coupling of dynamics among subsystems and the inter-controller communication is characterized by the same directed graph. Furthermore, this graph is assumed to be a multitree, that is, its transitive reduction can have at most one directed path connecting each pair of nodes. In this first part, we derive sufficient statistics that may be used to aggregate each controller's growing available information. Each controller must estimate the states of the subsystems that it affects (its descendants) as well as the subsystems that it observes (its ancestors). The optimal control action for a controller is a linear function of the estimate it computes as well as the estimates computed by all of its ancestors. Moreover, these state estimates may be updated recursively, much like a Kalman filter

Rate-cost tradeoffs in control

Consider a distributed control problem with a communication channel connecting the observer of a linear stochastic system to the controller. The goal of the controller is minimize a quadratic cost function. The most basic special case of that cost function is the mean-square deviation of the system state from the desired state. We study the fundamental tradeoff between the communication rate r bits/sec and the limsup of the expected cost b, and show a lower bound on the rate necessary to attain b. The bound applies as long as the system noise has a probability density function. If target cost b is not too large, that bound can be closely approached by a simple lattice quantization scheme that only quantizes the innovation, that is, the difference between the controller's belief about the current state and the true state

A Decoupling Principle for Simultaneous Localization and Planning Under Uncertainty in Multi-Agent Dynamic Environments

Simultaneous localization and planning for nonlinear stochastic systems under process and measurement uncertainties is a challenging problem. In its most general form, it is formulated as a stochastic optimal control problem in the space of feedback policies. The Hamilton-Jacobi-Bellman equation provides the theoretical solution of the optimal problem; but, as is typical of almost all nonlinear stochastic systems, optimally solving the problem is intractable. Moreover, even if an optimal solution was obtained, it would require centralized control, while multi-agent mobile robotic systems under dynamic environments require decentralized solutions. In this study, we aim for a theoretically sound solution for various modes of this problem, including the single-agent and multi-agent variations with perfect and imperfect state information, where the underlying state, control and observation spaces are continuous with discrete-time models. We introduce a decoupling principle for planning and control of multi-agent nonlinear stochastic systems based on a small noise asymptotics. Through this decoupling principle, under small noise, the design of the real-time feedback law can be decoupled from the off-line design of the nominal trajectory of the system. Further, for a multi-agent problem, the design of the feedback laws for different agents can be decoupled from each other, reducing the centralized problem to a decentralized problem requiring no communication during execution. The resulting solution is quantifiably near-optimal. We establish this result for all the above-mentioned variations, which results in the following variants: Trajectory-optimized Linear Quadratic Regulator (T-LQR), Multi-agent T-LQR (MT-LQR), Trajectory-optimized Linear Quadratic Gaussian (T-LQG), and Multi-agent T-LQG (MT-LQG). The decoupling principle provides the conditions under which a decentralized linear Gaussian system with a quadratic approximation of the cost, obtained by linearization around an optimally designed nominal trajectory can be utilized to control the nonlinear system. The resulting decentralized feedback solution at runtime, being decoupled with respect to the mobile agents, requires no communication between the agents during the execution phase. Moreover, the complexity of the solution vis-a-vis the computation of the nominal trajectory as well as the closed-loop gains is tractable with low polynomial orders of computation. Experimental implementation of the solution shows that the results hold for moderate levels of noise with high probability. Further optimizing the performance of this approach we show how to design a special cost function for the problem with imperfect state measurement that takes advantage of the fact that the estimation covariance of a linear Gaussian system is deterministic and not dependent on the observations. This design, which corresponds in our overall design to “belief space planning”, incorporates the consequently deterministic cost of the stochastic feedback system into the deterministic design of the nominal trajectory to obtain an optimal nominal trajectory with the best estimation performance. Then, it utilizes the T-LQG approach to design an optimal feedback law to track the designed nominal trajectory. This iterative approach can be used to further tune both the open loop as well as the decentralized feedback gain portions of the overall design. We also provide the multi-agent variant of this approach based on the MT-LQG method. Based on the near-optimality guarantees of the decoupling principle and the TLQG approach, we analyze the performance and correctness of a well-known heuristic in robotic path planning. We show that optimizing measures of the observability Gramian as a surrogate for estimation performance may provide irrelevant or misleading trajectories for planning under observation uncertainty. We then consider systems with non-Gaussian perturbations. An alternative heuristic method is proposed that aims for fast planning in belief space under non- Gaussian uncertainty. We provide a special design approach based on particle filters that results in a convex planning problem implemented via a model predictive control strategy in convex environments, and a locally convex problem in non-convex environments. The environment here refers to the complement of the region in Euclidean space that contains the obstacles or “no fly zones”. For non-convex dynamic environments, where the no-go regions change dynamically with time, we design a special form of an obstacle penalty function that incorporates non-convex time-varying constraints into the cost function, so that the decoupling principle still applies to these problems. However, similar to any constrained problem, the quality of the optimal nominal trajectory is dependent on the quality of the solution obtainable for the nonlinear optimization problem. We simulate our algorithms for each of the problems on various challenging situations, including for several nonlinear robotic models and common measurement models. In particular, we consider 2D and 3D dynamic environments for heterogeneous holonomic and non-holonomic robots, and range and bearing sensing models. Future research can potentially extend the results to more general situations including continuous-time models

On Control and Estimation of Large and Uncertain Systems

This thesis contains an introduction and six papers about the control and estimation of large and uncertain systems. The first paper poses and solves a deterministic version of the multiple-model estimation problem for finite sets of linear systems. The estimate is an interpolation of Kalman filter estimates. It achieves a provided energy gain bound from disturbances to the point-wise estimation error, given that the gain bound is feasible. The second paper shows how to compute upper and lower bounds for the smallest feasible gain bound. The bounds are computed via Riccati recursions. The third paper proves that it is sufficient to consider observer-based feedback in output-feedback control of linear systems with uncertain parameters, where the uncertain parameters belong to a finite set. The paper also contains an example of a discrete-time integrator with unknown gain. The fourth paper argues that the current methods for analyzing the robustness of large systems with structured uncertainty do not distinguish between sparse and dense perturbations and proposes a new robustness measure that captures sparsity. The paper also thoroughly analyzes this new measure. In particular, it proposes an upper bound that is amenable to distributed computation and valuable for control design. The fifth paper solves the problem of localized state-feedback L2 control with communication delay for large discrete-time systems. The synthesis procedure can be performed for each node in parallel. The paper combines the localized state-feedback controller with a localized Kalman filter to synthesize a localized output feedback controller that stabilizes the closed-loop subject to communication constraints. The sixth paper concerns optimal linear-quadratic team-decision problems where the team does not have access to the model. Instead, the players must learn optimal policies by interacting with the environment. The paper contains algorithms and regret bounds for the first- and zeroth-order information feedback

Optimal Control of a Large Space Telescope Using an Annular Momentum Control Device

Application of a new development in the field of momentum storage devices, the Annular Momentum Control Device (AMCD), to the twin problems of large angle maneuvers and fine pointing control is considered. The basic concept of the AMCD consists of a spinning rim, with no central hub area, suspended by a minimum of three magnetic bearings, and driven by a noncontacting electromagnetic spin motor. The dissertation considers in detail the design of an optimal controller to achieve both large angle maneuvers and the fine pointing control of a Large Telescope (LST) with a single configuration, consisting of a single AMCD mounted in a single gimbal

Discrete-time optimal preview control

There are many situations in which one can preview future reference signals, or future disturbances. Optimal Preview Control is concerned with designing controllers which use this preview to improve closed-loop performance. In this thesis a general preview control problem is presented which includes previewable disturbances, dynamic weighting functions, output feedback and nonpreviewable disturbances. It is then shown how a variety of problems may be cast as special cases of this general problem; of particular interest is the robust preview tracking problem and the problem of disturbance rejection with uncertainty in the previewed signal. . (', The general preview problem is solved in both the Fh and Beo settings. The H2 solution is a relatively straightforward extension ofpreviously known results, however, our contribution is to provide a single framework that may be used as a reference work when tackling a variety of preview problems. We also provide some new analysis concerning the maximum possible reduction in closed-loop H2 norm which accrues from the addition of preview action. / Name of candidate: Title of thesis: I DESCRIPTION OF THESIS Andrew Hazell Discrete-Time Optimal Preview Control The solution to the Hoo problem involves a completely new approach to Hoo preview control, in which the structure of the associated Riccati equation is exploited in order to find an efficient algorithm for computing the optimal controller. The problem tackled here is also more generic than those previously appearing in the literature. The above theory finds obvious applications in the design of controllers for autonomous vehicles, however, a particular class of nonlinearities found in typical vehicle models presents additional problems. The final chapters are concerned with a generic framework for implementing vehicle preview controllers, and also a'case study on preview control of a bicycle.Imperial Users onl