2,977 research outputs found
Synthesis of Minimal Error Control Software
Software implementations of controllers for physical systems are at the core
of many embedded systems. The design of controllers uses the theory of
dynamical systems to construct a mathematical control law that ensures that the
controlled system has certain properties, such as asymptotic convergence to an
equilibrium point, while optimizing some performance criteria. However, owing
to quantization errors arising from the use of fixed-point arithmetic, the
implementation of this control law can only guarantee practical stability:
under the actions of the implementation, the trajectories of the controlled
system converge to a bounded set around the equilibrium point, and the size of
the bounded set is proportional to the error in the implementation. The problem
of verifying whether a controller implementation achieves practical stability
for a given bounded set has been studied before. In this paper, we change the
emphasis from verification to automatic synthesis. Using synthesis, the need
for formal verification can be considerably reduced thereby reducing the design
time as well as design cost of embedded control software.
We give a methodology and a tool to synthesize embedded control software that
is Pareto optimal w.r.t. both performance criteria and practical stability
regions. Our technique is a combination of static analysis to estimate
quantization errors for specific controller implementations and stochastic
local search over the space of possible controllers using particle swarm
optimization. The effectiveness of our technique is illustrated using examples
of various standard control systems: in most examples, we achieve controllers
with close LQR-LQG performance but with implementation errors, hence regions of
practical stability, several times as small.Comment: 18 pages, 2 figure
Towards a Theoretical Foundation of Policy Optimization for Learning Control Policies
Gradient-based methods have been widely used for system design and
optimization in diverse application domains. Recently, there has been a renewed
interest in studying theoretical properties of these methods in the context of
control and reinforcement learning. This article surveys some of the recent
developments on policy optimization, a gradient-based iterative approach for
feedback control synthesis, popularized by successes of reinforcement learning.
We take an interdisciplinary perspective in our exposition that connects
control theory, reinforcement learning, and large-scale optimization. We review
a number of recently-developed theoretical results on the optimization
landscape, global convergence, and sample complexity of gradient-based methods
for various continuous control problems such as the linear quadratic regulator
(LQR), control, risk-sensitive control, linear quadratic
Gaussian (LQG) control, and output feedback synthesis. In conjunction with
these optimization results, we also discuss how direct policy optimization
handles stability and robustness concerns in learning-based control, two main
desiderata in control engineering. We conclude the survey by pointing out
several challenges and opportunities at the intersection of learning and
control.Comment: To Appear in Annual Review of Control, Robotics, and Autonomous
System
Evolutionary Poisson Games for Controlling Large Population Behaviors
Emerging applications in engineering such as crowd-sourcing and
(mis)information propagation involve a large population of heterogeneous users
or agents in a complex network who strategically make dynamic decisions. In
this work, we establish an evolutionary Poisson game framework to capture the
random, dynamic and heterogeneous interactions of agents in a holistic fashion,
and design mechanisms to control their behaviors to achieve a system-wide
objective. We use the antivirus protection challenge in cyber security to
motivate the framework, where each user in the network can choose whether or
not to adopt the software. We introduce the notion of evolutionary Poisson
stable equilibrium for the game, and show its existence and uniqueness. Online
algorithms are developed using the techniques of stochastic approximation
coupled with the population dynamics, and they are shown to converge to the
optimal solution of the controller problem. Numerical examples are used to
illustrate and corroborate our results
- …